Why AI Gives Wrong Answers

An AI chatbot gave a lawyer a list of case citations for a court filing. The citations looked real. They had case names, court names, and dates. They were entirely made up. The lawyer submitted them to the court, the court discovered they did not exist, and the lawyer was fined. The story, from 2023, is the most famous example of a problem called hallucination, and it is the reason this post exists. AI gives wrong answers, confidently, and the confidence is the danger. This post explains, in simple words, why AI gives wrong answers, the four ways it goes wrong, and the checklist that protects you.

The Core Reason: Prediction, Not Knowledge

The core reason AI gives wrong answers is that it does not know what is true. A large language model predicts the next token — the next chunk of text — based on patterns in the enormous amount of text it was trained on. It has no internal sense of what is true and what is false. It has a sense of what sounds likely, given the words that came before. IBM’s guide to AI hallucinations, updated in July 2026, puts it plainly: the model predicts the next token based on patterns, without an internal sense of truth, and the text it produces can be fluent and confident while still being wrong.

That distinction — fluent versus true — is the whole problem. A human who does not know an answer usually hesitates, says “I don’t know”, or guesses with visible uncertainty. A language model does none of those things by default. It produces the most likely continuation of the text, with the same confidence whether the continuation is true or false. The result reads like an answer from someone who knows, even when it is an answer from a machine that is guessing. The Everyday AI Guide, checked in October 2026, says it directly: AI is not lying, because lying requires intent, but it predicts likely-sounding words, and likely-sounding is not the same as correct.

Hallucinations: Plausible, Wrong, Confident

The word used for AI’s wrong answers is hallucination. IBM defines an AI hallucination as an output that is plausible but factually wrong, irrelevant, or fabricated. The word is a metaphor, borrowed from human experience, and it is not perfect, because a machine does not perceive anything. But it captures the experience of the user: the answer looks real, reads smoothly, and is wrong in a way that is hard to spot without checking.

Hallucinations come in types. Factual hallucinations invent facts: a date, a name, a number, or an event that never happened. Contextual hallucinations give an answer that is true in general but wrong for the specific context, such as answering a question about one country with facts about another. Consistency hallucinations contradict themselves, saying one thing in one paragraph and the opposite in the next. The lawyer’s fake citations were factual hallucinations: plausible, detailed, and entirely invented. The danger is not that AI is often wrong. It is that when it is wrong, it is wrong in a way that looks right.

Four Problems Behind Wrong Answers

The Everyday AI Guide lists four problems that sit behind AI’s wrong answers. The first is hallucinations: the model invents facts that sound plausible. The second is outdated information: the model’s training data ends at a cut-off date, so it does not know about events after that date, and it may answer a question about this week with information from last year. The third is made-up sources: the model invents citations, links, books and articles that do not exist, because a citation looks like a plausible next piece of text. The fourth is missing context: the model does not know what you meant, what you already know, or what details matter, so it answers the question it thinks you asked, not the one you meant.

Each problem has a different fix. Hallucinations are reduced by checking facts against reliable sources. Outdated information is fixed by retrieval, where the system fetches current information and puts it into the model’s context. Made-up sources are caught by clicking the link or searching for the title before you trust it. Missing context is fixed by giving the model more detail in your question: what you need, what you already know, and what the answer is for. No single fix solves all four, which is why checking matters more than any one trick.

Why Training Data Causes Errors

The model’s training data is the root of many errors. The data is massive, drawn from books, articles, websites, and other text, and it contains errors, gaps, biases, and outdated information, because human text contains all of those things. IBM lists flawed data as the first cause of hallucinations: if the training data contains a wrong fact, the model may learn it as a pattern and repeat it. If the data contains a gap, the model may fill the gap with a plausible invention. If the data is biased, the model’s answers will be biased in the same direction.

Two other technical causes add to the problem. Overfitting happens when the model learns the training data too closely, including its errors, and repeats them instead of generalising. Design limits mean that the model, which works by predicting tokens, has no built-in mechanism for checking facts against the world. It cannot look something up on its own, unless a tool is provided for that purpose. And prompt injection, where text in a document or web page tells the model to do something the user did not ask for, can push the model towards wrong or unwanted answers. The causes are technical, but the result is human: a wrong answer that looks right.

The Famous Case: Fake Citations In Court

The 2023 case of the American lawyer who submitted fake ChatGPT citations is worth telling in full, because it shows every part of the problem. The lawyer asked ChatGPT for cases supporting his argument. ChatGPT returned a list of cases with names, courts, and summaries. The lawyer submitted them in a court filing. The opposing lawyers could not find the cases. The judge asked the lawyer to provide copies. The lawyer asked ChatGPT whether the cases were real, and ChatGPT said they were. They were not. The cases did not exist. The court fined the lawyer and his firm.

The case matters because the lawyer did what many users do: he trusted the fluency. The citations looked like real citations, with the right format and the right kind of detail. The model had no reason to invent them, from the lawyer’s point of view, so he assumed they were real. The lesson is not that the lawyer was foolish. It is that the model’s fluency is designed to produce exactly that trust, and that trust must be checked, every time, against a source that exists outside the model.

How To Reduce Hallucinations

Hallucinations can be reduced, but not fully eliminated, according to IBM. The methods that help are practical and well tested. Quality data: training and fine-tuning on accurate, well-checked text reduces the errors the model learns. Retrieval-augmented generation, known as RAG: fetching relevant documents and putting them into the model’s context, so that it bases its answer on real text rather than on patterns alone. Prompt engineering: writing clearer questions, with more context, so that the model has a better chance of predicting the right answer. Human in the loop: having a person check the answer before it is used for anything important.

None of these methods makes the model know the truth. They make it more likely to produce a correct answer, by giving it better material to work with and by checking its work. The honest summary, from IBM, is that hallucinations can be reduced but not fully eliminated. Any system that claims to have solved hallucination entirely is making a claim the research does not support. The practical answer is to use AI with checking, not to wait for a version that never needs checking.

An Eight-Point Checklist Before You Trust An Answer

The Everyday AI Guide offers an eight-point checklist for checking AI answers, and it is worth using every time the answer matters. First, check the facts against a reliable source, especially names, dates, and numbers. Second, click the links and check that the sources exist. Third, ask whether the answer is up to date, and whether the model could know about recent events. Fourth, check whether the answer fits the context of your question, or whether it has answered a different question. Fifth, look for internal contradictions in the answer. Sixth, ask the model to show its sources, and then check those sources yourself. Seventh, compare the answer with a second source, such as a search engine or a reference book. Eighth, use your own judgement: if an answer seems too neat, too confident, or too convenient, it is worth a second look.

The checklist is not fast, and that is the point. Checking takes time, and the time is the cost of using a tool that is fluent but not reliably true. For a casual question, a quick check is enough. For a court filing, a medical question, a financial decision, or anything that affects other people, the full checklist is the minimum. The AI is a drafting tool, not a source of truth. Treating it as a source of truth is how the lawyer’s case happened.

When To Trust AI, And When Not To

AI is trustworthy for drafting, brainstorming, explaining, translating, and summarising text you already have. It is useful for generating ideas, for rewording, for outlining, and for helping you think through a problem. It is not trustworthy, on its own, for facts that matter: names, dates, statistics, citations, medical advice, legal advice, or financial decisions. For those, it is a starting point, not an answer.

The rule is simple. If the answer will be used by other people, or will affect a decision that matters, check it. If the answer is for your own thinking, and a mistake costs nothing, a quick answer is fine. The difference between those two situations is the difference between a tool and a trap. AI is a powerful tool. Used with checking, it makes work faster and thinking clearer. Used without checking, it makes mistakes faster, and with more confidence than any human would dare.

Frequently Asked Questions

Why does AI give wrong answers?

AI gives wrong answers because it predicts the next token based on patterns in its training data, without an internal sense of what is true. It produces text that sounds likely, given the words that came before, with the same confidence whether the continuation is true or false. Its training data contains errors, gaps, biases, and outdated information, and the model can repeat those errors or fill gaps with plausible inventions. The result is fluent and confident text that can be factually wrong, which is why checking against a reliable source matters.

What is an AI hallucination?

An AI hallucination is an output that is plausible but factually wrong, irrelevant, or fabricated. IBM defines it that way, and lists three types: factual hallucinations, which invent facts such as dates, names, or events that never happened; contextual hallucinations, which give an answer that is true in general but wrong for the specific context; and consistency hallucinations, which contradict themselves within the same answer. The most famous example is the 2023 case of a lawyer who submitted fake ChatGPT case citations to a court, which did not exist and led to a fine.

Can AI hallucinations be completely stopped?

No. IBM states that hallucinations can be reduced but not fully eliminated. Methods that help include training on quality data, retrieval-augmented generation, which fetches real documents for the model to base its answer on, clear prompt engineering, and keeping a human in the loop to check answers before they are used. None of these methods makes the model know the truth. They make a correct answer more likely, by giving the model better material and by checking its work. Any claim that hallucination has been solved entirely is not supported by the research.

How can I check an AI answer?

Use a checklist: check names, dates, and numbers against a reliable source; click the links and confirm the sources exist; ask whether the answer is up to date; check whether it fits the context of your question; look for contradictions within the answer; ask the model to show its sources and check them yourself; compare the answer with a second source; and use your own judgement if an answer seems too neat or too confident. For facts that matter, such as medical, legal, or financial decisions, the full checklist is the minimum. AI is a drafting tool, not a source of truth.

Is AI lying when it gives a wrong answer?

No, not in the usual sense. Lying requires intent, and a language model has no intent. It predicts likely-sounding words based on patterns, and likely-sounding is not the same as correct. The Everyday AI Guide puts it directly: AI is not lying, but it predicts likely-sounding words. The distinction matters, because it means the problem is not dishonesty but design: a system built to produce fluent text will produce fluent text whether or not the text is true, and the user must supply the checking that the system does not do itself.

Sources

• IBM — What Are AI Hallucinations?: https://www.ibm.com/think/topics/ai-hallucinations
• Everyday AI Guide — Why AI Gets Things Wrong: https://www.everydayaiguide.com/why-ai-gets-things-wrong/