Hallucinations, plainly

In the last lesson you met the rule: AI drafts, you check. This lesson explains the why behind it, because once you understand why AI makes things up, checking stops feeling like paranoia and starts feeling like the obvious thing to do.

The one bit of jargon worth knowing

The word is hallucination. It sounds dramatic, but it just means this: the AI states something false as though it were certainly true. It will invent a statistic, a quote, a policy, a web address or a fact, and present it with the same calm confidence it uses for things that are correct.

That is the whole idea. There is no wobble, no “I think”, no visible difference between the AI knowing and the AI guessing. It all comes out sounding sure.

Why it happens (and why you cannot switch it off)

Cast your mind back to lesson one. The machine works by predicting plausible next words. That is not a flaw bolted onto an otherwise perfect system. It is the whole mechanism. And here is the catch: plausible is not the same as true. Most of the time the most plausible words are also the correct ones, which is why AI is useful at all. But sometimes the most plausible-sounding answer is simply wrong, and the machine has no separate sense of truth to catch it. It is not lying, because lying needs you to know the truth first. It genuinely does not know it is wrong.

So this is not a bug that a future update quietly fixes. Newer models hallucinate less, and that is real progress, but “less” is not “never”. Treat it as a permanent feature of the tool, the way you treat a knife as permanently sharp, and you will handle it fine.

When it is most likely to happen

Hallucinations are not random. They cluster in predictable places, and knowing where lets you raise your guard at the right moments:

• Specific facts and figures: dates, prices, statistics, phone numbers, legal clauses. Anything precise that it would have to recall exactly.
• Your business: anything about your prices, policies, staff or customers, which it has never seen.
• Recent events: things that happened after its training, which it may not know but will still gamely guess at.
• Sources and links: it can invent a study, a book, a case or a web address that looks completely real.
• Anything it does not know but wants to be helpful about: it would rather give you a confident answer than say “I am not sure”.

Notice that ordinary writing work, drafting an email, softening a message, summarising text you gave it, sits well away from this list. That is why words-in, words-out jobs are the safe heartland of everyday AI use.

Two habits that handle it

You do not need to become a fact-checking machine yourself. Two small habits cover most of it.

First, match your checking to the stakes. A tidied-up internal note barely needs a glance. A figure going into a customer quote needs verifying against your real numbers. Spend your attention where a mistake would actually cost you.

Second, give it the facts up front. A huge share of hallucinations happen because the AI is filling a gap you left. Paste in the real document, the real numbers, the real policy, and ask it to work from that. It hallucinates far less when it is not forced to guess. Giving your AI the right context to work from is a big enough idea that we build a whole later course around it.

Understand this and the safety rule is no longer a rule you follow on trust. It is just common sense. Next, the fun part: five real jobs to hand to an AI this week.