An AI hallucination is an answer from a chatbot or other AI system that sounds right but is false: a wrong date, a misattributed quote, a study or court case that does not exist. OpenAI, which makes ChatGPT, defines hallucinations as “plausible but false statements generated by language models.” The U.S. National Institute of Standards and Technology prefers the term confabulation, which its generative AI risk profile describes as “the production of confidently stated but erroneous or false content (known colloquially as ‘hallucinations’ or ‘fabrications’).”
The confidence is the problem. A hallucinated answer usually reads exactly like a correct one, with the same fluent tone and, sometimes, a citation attached.
What a hallucination looks like
OpenAI’s help center lists the common forms in ChatGPT: incorrect definitions, dates or facts; fabricated quotes, studies, citations or references to sources that do not exist; and overconfident answers to ambiguous or complex questions. It adds a blunt line: “Confidence isn’t reliability.”
NIST’s definition is wider. It also counts answers that drift away from the prompt or the material supplied with it, and answers that contradict something the system said earlier in the same conversation. It warns that a model can produce confabulated reasoning or citations that make a wrong answer look justified, laying out logical-looking steps “even when the answer itself is incorrect.”
A small example comes from OpenAI’s own researchers. In a 2025 paper, they asked a state-of-the-art open-source model for the birthday of one of the paper’s authors, Adam Tauman Kalai, and requested an answer only if it knew. On three attempts it gave three different dates, all wrong. Asked for the title of his PhD dissertation, widely used chatbots gave confident answers that were also incorrect.
Why language models hallucinate
A large language model learns by predicting the next word across huge amounts of text. OpenAI’s researchers point out that this text carries no “true” or “false” labels: the model sees fluent language and learns its patterns. Patterns that are consistent, such as spelling or matching parentheses, are learned reliably, which is why these models rarely misspell words. Arbitrary facts that appear rarely, like a particular person’s birthday, cannot be inferred from patterns, so the model fills the gap with something that merely fits.
NIST describes the same mechanism from a risk angle: confabulations “are a natural result of the way generative models are designed,” because the models generate outputs that approximate the statistical patterns of their training data. It notes the risk is higher for open-ended, long answers and for subjects that require specialist knowledge.
The second reason is how models are graded. OpenAI argues that most benchmarks score only accuracy, so a model that guesses is rewarded over one that admits it does not know. Its analogy is a multiple-choice exam: a blank answer scores zero, a wild guess sometimes scores a point. If a model is asked for a birthday it does not know, guessing gives it a 1-in-365 chance of being right, while “I don’t know” guarantees zero points.
The company’s own test results show the trade-off. On SimpleQA, a set of short factual questions, its gpt-5-thinking-mini model declined to answer 52% of the time, answered correctly 22% of the time and gave a wrong answer 26% of the time. The older o4-mini almost never declined (1%) and scored slightly higher on accuracy, 24%, but was wrong 75% of the time. OpenAI says GPT-5 has significantly fewer hallucinations than earlier models, “but they still occur.”
Other causes are more ordinary. A model’s knowledge stops at its training cutoff unless it can search, and OpenAI notes that it may not be able to reach a web page because of technical problems, paywalls or a site’s robots.txt settings. Errors and gaps in the training data are learned along with everything else.
When it goes wrong in public
The best-documented case is a U.S. court ruling. In Mata v. Avianca, a federal judge in the Southern District of New York found that two lawyers and their firm had submitted “non-existent judicial opinions with fake quotes and citations created by the artificial intelligence tool ChatGPT,” according to the opinion and order on sanctions of June 22, 2023. The court imposed a $5,000 penalty and ordered the lawyers to send letters to each judge falsely named as the author of a fake opinion.
One detail is worth remembering. According to the court’s findings, one of the lawyers asked ChatGPT whether one of the cases was real, and the chatbot replied that it “does indeed exist” and could be found on legal research databases. Asking a chatbot to confirm its own answer is not a check.
How to check an AI answer
No setting removes hallucinations. Anthropic, which makes Claude, says in its developer guide that its techniques reduce them but “don’t eliminate them entirely,” and advises validating critical information. These habits, drawn from the AI companies’ own guidance, catch most problems.
Treat the answer as a first draft. OpenAI’s advice is to use ChatGPT “as a first draft, not a final source” and to verify quotes, data, technical information and references to outside documents.
Open the sources. When a chatbot cites a page, follow the link and find the claim on that page. OpenAI recommends checking sources by visiting links directly; Google’s Gemini help center explains that Gemini can show a Sources button with related links, and that not every response includes them. A citation that does not exist, or a page that says something different, is the clearest warning sign.
Ask for quotes, not summaries. When you give a chatbot a document, Anthropic suggests asking it to pull word-for-word quotes before it answers, and to withdraw any claim it cannot support with a quote. A quote can be found in the original; a paraphrase is harder to verify.
Give it permission to say “I don’t know.” Anthropic says explicitly allowing the model to admit uncertainty “can drastically reduce false information.”
Ask the same question more than once. Anthropic’s guide notes that inconsistencies between several answers to the same prompt can indicate a hallucination. The three different birthdays in OpenAI’s example would have exposed the problem at once.
Be most careful with names, numbers, dates and citations. These are the arbitrary, low-frequency facts OpenAI’s research identifies as most prone to error, and they are also the details most likely to be repeated without checking.
Does search or RAG fix it?
It helps. A chatbot that searches the web, or a system that looks up passages in a set of documents before answering, a design called retrieval-augmented generation (RAG), gives the model real text to work from and lets you see where an answer came from. But retrieval can pick the wrong passage, and the sources themselves can be wrong, so the answer still needs the same checks.
The risk grows when an AI system acts rather than just answers. An AI agent that books, buys or edits files on the basis of a hallucinated detail turns a wrong sentence into a wrong action. If you paste documents into a chatbot so it can answer from them, what the service keeps is a separate question: see what happens to files you upload to an AI tool and our AI privacy guide for how nine assistants handle your data.
