What it means in practice
An AI model (typically an LLM such as ChatGPT or Claude) gives you an answer that sounds confident and looks like a fact, but is wrong or entirely invented. The best-known examples: made-up sources, laws that do not exist, fictional numbers.
The reason is technical. An LLM does not know the truth. It predicts the most likely continuation of the text from its training data. When it is unsure, it does not say “I do not know”, it says something that sounds right.
Why companies need to know this
- Customer communication. An AI agent in support hallucinates a price, a term or a deadline, and you have to honour it.
- Internal decisions. A manager has AI do the research, gets invented numbers, and decides on the basis of them.
- Compliance and law. The AI produces a clause that does not exist and the lawyer does not catch it in time.
- Marketing and content. The AI writes a “fact” that is invented and you publish it under the company name.
How to protect yourself
- Do not ask AI for facts you cannot verify. AI is an excellent assistant for structure, concept and a first draft. Not for the truth of a number.
- Use RAG (Retrieval-Augmented Generation). The AI answers only from your documents, not from its training data.
- Demand sources. Have the AI cite where each fact came from. Check at least one in three.
- Human in the loop. For anything with consequences (a customer, the law, money), a person approves it before it goes out.
- Log the outputs. So you can trace back where the hallucination came from.
Hallucinations are not a reason not to use AI. They are a reason to use it properly. In the workshop we show the specific processes where hallucinations do not matter, the ones where they are dangerous, and how to keep the two apart.