AI & TECHNOLOGY / PRACTICAL GUIDE

AI Hallucinations: Definition and How to Check Them

An AI hallucination is an error a model presents as fact. NIST calls this confabulation. To verify an AI-generated citation, find the original source, open the relevant section and check whether it supports the exact claim. Try it below with a real reference and two claims that look equally well sourced.

By Mika vs Guru Editorial Team · Updated 13 November 2026 · About 7 min read

How to verify AI-generated citations

Imagine an AI answer cites NIST AI 600-1 to make two claims. One holds up. The other falls apart as soon as you read the source.

1. Find the original. Follow the DOI for NIST AI 600-1. The cover identifies NIST as the publisher and gives the title Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile and the date, July 2024. If a link is broken, search for the exact title and report number on the publisher’s site.
2. Check the first claim. “NIST uses confabulation for errors that generative AI presents confidently.” Open section 2.2, titled Confabulation. It describes AI systems producing and confidently presenting false or erroneous content. Supported.
3. Check the second claim. “NIST makes this profile mandatory for every organisation using generative AI.” Section 1 says the framework is intended for voluntary use and describes the profile as offering suggested actions. The cited document does not support a universal requirement.

For the second claim, the corrected wording would be “NIST published suggested actions for managing generative AI risks.” If you can’t locate the source or the relevant passage, mark the claim as unverified and leave it out of anything you plan to share as fact.

Confidence is a style signal, not a fact-check

Grammar, specific-sounding numbers and even a neatly formatted citation can make an answer look authoritative regardless of whether the underlying claim is true. Presentation and evidence come from two separate processes, and only one of them is actually checkable.

The case: one generated answer, five claims

Fictional worked example: Northfield Robotics, its people, figures and verification outcomes below are invented to demonstrate the checking method.

Say you ask an AI assistant to summarize the background of a company we'll call Northfield Robotics, and it returns a tidy five-sentence bio. Before repeating any of it, break the paragraph into individual, checkable claims — one row per claim, and one plan for how you'd confirm each.

What the check actually turned up

• Founding year and founder — supported: Matched an independent filing and the company's own materials, so this claim held up.
• Funding figure — unsupported: No such round appears in any funding record. The number was plausible-sounding and simply invented to fill a gap in what the model actually knew.
• Headquarters — uncertain: One official source said Austin; an older one said Dallas. The honest verdict is "uncertain, pending a current source," not a coin-flip guess at which is right.

Three different verdicts from one short paragraph — supported, unsupported and uncertain — is a normal outcome, not a rare one. The point of the exercise is catching that spread before any of it gets repeated as settled fact.

Why this happens

A generative model produces the statistically likely next piece of text given everything it has seen and the prompt it was given. When a real answer is thin, unclear or missing from its training, a fluent, wrong answer is often just as easy for it to produce as a correct one — the process behind it doesn't distinguish the two the way a person checking a fact would.

Five ways a wrong answer gets made

"The model made it up" is true but not useful. Different mechanisms produce different kinds of error, and knowing which one you are probably looking at tells you where to check first.

• Thin training data on a niche topic: A small company, a local regulation or an obscure paper gives the model little to draw on, so it fills the gap with whatever is typical. Example: the Northfield funding round above, a normal-sounding number attached to a company it barely knew.
• Stale data in the present tense: Training stops at a cutoff, but the answer reads as if it were written this morning. Example: "The CEO is Elena Cho" when she stepped down last year and the model never saw the announcement.
• A leading prompt: Ask "Why did Northfield recall the R-9 in 2022?" and the model will often explain a recall that never happened, because the question already asserted it. Example: any prompt beginning "Explain why" about an event you have not confirmed.
• Long context, lost details: Paste a forty-page document and ask about page three, and the answer may blend page three with page thirty. Example: a summary of a long transcript that attributes a quote to the wrong speaker.
• Citations that look real and are not: The model has seen millions of references and can produce the shape of one perfectly. Example: "Smith and Patel (2019), Journal of Robotics Safety, 14(2)," where the journal may exist but the article does not.

The last one defeats the usual instinct, because a citation feels like evidence. Treat it as one more claim: open the source rather than noting that one was named.

A verification routine you can actually run

1. Split the answer into claims: One factual statement per line — names, dates, numbers and anything presented as settled.
2. Find an independent source for each one: Not another AI-generated summary, and not a source that could have copied the same original error.
3. Check the exact wording: A source can support the general topic while contradicting the specific number or date given.
4. Note the source's date: A true claim from 2019 can be a false one today.
5. Mark the verdict: Supported, unsupported or still uncertain — and keep the uncertain ones labeled that way instead of rounding them up to true.

Triage, document, escalate

The routine above handles one answer. Three additions keep regular checking from becoming either a full-time job or a formality. First, triage by what is riding on the claim, because not every line deserves the same effort.

Second, document what you checked, briefly. One line per claim is plenty: the claim, the source used, the date you looked, the verdict. "Funding round: searched two funding databases on 14 May, nothing found, unsupported." Six months later that line tells you whether the check is still current, and it lets someone else see what was actually confirmed instead of taking your word for it.

Third, escalate instead of resolving it yourself when the claim sits in a domain where being wrong has real consequences and you are not qualified to judge the sources. A drug interaction, a tax treatment, a contract clause: the right move is not a third search but handing the specific claim and your notes to a pharmacist, an accountant or a lawyer, with the honest label "an AI tool said this and I could not confirm it." That handoff is a verification step, not a failure of one.

Set a stopping rule

If a claim is consequential and you cannot trace it to a suitable source or a qualified person, the answer is not "probably fine" — it is "not yet checked." Record the uncertainty and act only on what you have actually confirmed.

What to do next

1. Pick one AI-generated answer already sitting in a chat history and split it into individual claims before trusting any of it.
2. For each claim, find one independent source that isn't itself another AI summary, and check the exact wording, not just the general topic.
3. Label every claim supported, unsupported or uncertain, and only share or act on the ones marked supported.

Sources and further reading

• NIST: Generative AI Profile ↗

Continue learning