Ask a language model a question and the answer arrives in clean, assured sentences. That polish is the trap. A model writes fluently whether or not it has the facts, and fluent text feels trustworthy even when a detail is invented.
The industry word for an invented or unsupported statement is a hallucination. Provider documentation says plainly that even very capable models can produce text that is factually incorrect or inconsistent with the context they were given. This is a property of how the tools work, not a rare glitch you can assume away.
Why it happens#
Recall from the earlier article that a language model predicts likely text from context. Likely is not the same as true. Several ordinary conditions make errors more probable:
- Missing context. If the facts are not in the request, the model fills the gap with something plausible.
- Stale or absent information. The model learned from data collected up to some point. Prices, dates, laws and company details move on.
- Ambiguous instructions. If you leave a choice open, the model picks one and does not tell you it was a choice.
- Invented references. A model asked for sources may produce a title, a quotation or a link that looks right and does not exist.
- Pressure to answer. Unless told it may say "I do not know", a model tends to give an answer.
None of this means the tools are useless. It means the output is a candidate, and the person using it is responsible for what is accepted.
An illustration#
The example is fictional. A small company asks an assistant: "Write the launch announcement for our new app." The brief mentions the app's name and features. It does not mention a release date, because the date has not been decided.
The announcement comes back: "We are delighted to announce that Orbit launches on 14 March!" The date reads naturally. Nobody supplied it. It was filled in.
If that line is published, the company has announced a date it never chose. The right treatment is to see that the date is unsupported and turn it into a question for a person: what is the real launch date? Until someone answers, the announcement has a gap, not a fact.
A way to check: claim, evidence, source, confirm#
Step 1: Claim · Overview
Pick one statement that matters.
Read all steps as a list
- Claim. Pick one statement that matters. A date, a figure, a name, a promise, a legal or medical statement. You do not need to verify every adjective, only the claims that would hurt if wrong.
- Evidence available?. Was it supplied or invented? Look at what you gave the model. If the claim is not in your own material, it came from the model and needs outside support.
- Check the source. Go to a primary or authoritative record. An official page, the original document, your own system of record. If a link was provided, open it and confirm it says what is claimed.
- Confirm or leave unknown. Keep it or mark it open. If the source supports it, keep it. If not, remove it or record it as an open question. Do not keep a claim because it sounds right.
Asking for sources helps, but check them#
Asking for sources and quotes is one of the better habits. Anthropic's guidance suggests having the model cite supporting quotes for its claims, and retract any claim it cannot support. That makes an answer easier to audit.
It does not make the answer safe on its own. A citation can be wrong, mismatched or invented. Open the link. Read the passage. Make sure it says what the answer says it does.
Make the task smaller and give evidence#
You can lower the odds of a wrong answer, not remove them:
- Provide the evidence. Paste the relevant text and ask the model to work only from it, not from general knowledge.
- Ask for quotes first. For long documents, extract the relevant sentences, then answer using only those.
- Allow an honest unknown. Tell it to say when the material does not contain the answer.
- Narrow the question. "What does clause 4 say about notice?" is safer than "Is this contract fine?"
- Compare runs. Different answers to the same question are a warning sign.
The documentation for these techniques says that they significantly reduce hallucinations but do not eliminate them, and that critical information should always be validated, especially for high-stakes decisions.
Separate facts, assumptions and suggestions#
When reading an output, sort each statement into one of three bins:
- Facts taken from a source you can name.
- Assumptions the model made to fill a gap. These should be labelled or removed.
- Suggestions and ideas which are creative and fine, as long as nobody mistakes them for fact.
A brief that asks the model to mark assumptions (see How to give AI a useful brief) makes this sorting easier.
Put a review before consequential actions#
How much checking is enough depends on the cost of being wrong and whether the action can be undone.
- Drafting an internal note: a quick read is probably enough.
- Sending an email to a customer: check names, amounts and promises.
- Changing a record, paying an invoice or giving legal, medical or financial advice: a named person approves it, against the real source.
A useful rule: AI may prepare. A person decides. This is how we think about workflow design too. The first-workflow guide shows where to draw the line: How to choose your first business AI workflow.
Takeaway#
Treat fluent text as a lead, not a finding. Provide the evidence, allow "unknown", check the claims that matter against a source and keep a person in charge of the decision.