AI hallucination

Why it happens

Language models are built to produce plausible text. Asked a question they can't ground in data, they may still produce something that reads like an answer — a sales figure, a client name, a date — because plausible is what they're good at.

Why it's dangerous in sales

A hallucinated figure doesn't look wrong. A rep repeats it to a client ("you've spent about €40,000 with us this year"), the client knows it's €25,000, and trust is gone — in the rep, and in the tool. In finance, an invented overdue amount can start the wrong conversation with a good customer.

How to prevent it

• Answer from a real query on your own data, not from the model's general knowledge.
• Treat an empty result as "I don't know", deterministically, before any wording is generated.
• Keep the wording strictly tied to the numbers returned.
• Test it: ask about a client that doesn't exist. The right answer is that there's no such client.

Where emily fits

emily never invents numbers: every figure comes from your data, and when the data isn't there she says so. See reliable numbers and emily vs ChatGPT and Copilot.