How to Check AI Answers for Fake Sources, Quotes and Statistics
An easy fact-checking workflow for AI-generated citations, quotes and statistics before they end up in a report or article.
The dangerous part of a wrong AI answer is often the confidence. A fabricated report can arrive with an impressive title, a believable author and a URL that almost looks right. If you are racing to finish a presentation, that packaging is easy to mistake for evidence.
The fix is not an elaborate research method. You need a repeatable check that takes a few minutes and does not depend on the AI grading its own homework.
What counts as an AI hallucination?
A hallucination is a claim generated without adequate grounding in the information that supports it. It might be an invented publication, a genuine paper with the wrong conclusion, a correct statistic assigned to the wrong year, or a quotation that nobody actually said. Some mistakes come from outdated information; others arise even when the model sounds certain.
A source link is not proof that the sentence next to it is true. The link might be broken, lead to an unrelated page or point to material that does not support the claim. That is why the unit of checking is claim plus evidence, not just the existence of a hyperlink.
The five checks I use as an editorial framework
- Existence: Does the source or document exist on the publisher's actual website? Search the exact title or DOI independently.
- Identity: Are the author, publisher, date and document type correct? An opinion column is not a government statistic.
- Support: Does the linked page actually say what the draft claims? Read the relevant section, not just the snippet.
- Context: Were caveats, countries, sample sizes, dates or uncertainty removed?
- Currency: Is the fact still relevant today? Policies, product features and prices change.
Do these checks on the most consequential claims first: numbers, quotations, safety instructions, legal statements and 'first ever' or 'best' claims. A harmless transition sentence does not deserve the same scrutiny as a statistic in a board report.
A worked example: an impressive but unsupported number
Suppose an AI answer says, 'A 2025 study found that 78% of UK small businesses use AI every day.' Do not copy it into an article simply because it sounds plausible.
Search the exact phrase in quotation marks. If you cannot find it, remove the quotation marks and search for a report on UK SME AI adoption. Open the actual research. Does it measure *daily use*, *some use*, or *planned use*? What businesses were surveyed? When? If the survey only measured whether businesses had used AI at least once, the stronger statement is unsupported even if the percentage happens to match.
The responsible final copy may be: 'Surveys report varying levels of AI adoption among small businesses, depending on how use is defined.' Better still, cite a source with its precise question and date, or leave the number out entirely.
This example is deliberately hypothetical; it is not a claim about any real survey.
Check quotations differently from summaries
Direct quotations require exact words. Open the original interview, transcript, speech or document and search for the phrase. Check that editing has not reversed the meaning or merged separate answers. If the quote is not verifiable, paraphrase only what the accessible source actually supports—or omit it.
For research papers, follow the DOI, publisher page or institutional repository. Abstracts help identify a paper, but they rarely settle a nuanced question about limitations, methods or findings.
Treat AI search results as leads, not footnotes
AI-assisted search can be useful for discovering a topic and possible sources. It is a poor final citation system unless someone independently opens and checks each reference. Even when the model has browsing or retrieval tools, an extraction can miss qualifiers or combine several documents into one conclusion.
Try this prompt to reduce unsupported assertions:
Give me a table with three columns: claim, original primary source URL, and the exact section that supports it. If you cannot verify a claim directly, write 'not verified'. Do not invent a source or quote. Separate your inference from the source's statement.
Then open the URLs yourself. The prompt makes the uncertainty more visible, but it doesn't remove the verification step.
A faster review process for long articles
For a 1,200-word draft, highlight every statistic, named study, quote and claim about product capabilities. Put them in a scratch table. Add the original URL and the date checked. If one claim cannot be established in a reasonable time, replace it with a narrower statement or remove it. Finally read the article again to see whether the argument still works.
Where a story relies on one company's announcement, call it what it is: a company claim. For important technology topics, look for independent analysis or direct demonstrations, not only marketing pages.
Mistakes that make an article look researched when it isn't
- Linking to a homepage rather than the specific page that supports the sentence.
- Citing a news article for a statistic that came from a different primary report.
- Treating an outdated pricing page as today's price.
- Using 'experts agree' with no named expert, method or evidence.
- Asking the same AI to 'confirm' its answer and treating its second answer as independent verification.
- Quietly replacing a source when the original claim was actually false.
Good research sometimes produces a shorter article. That's fine. Removing a dubious claim increases the value of everything that remains.
Questions readers often ask
Are AI citations always fake?
No. Many are legitimate, and tools connected to retrieval systems can return accurate links. Each important claim still needs a supporting source check.
Can I use Wikipedia to verify a claim?
It can help you find a topic and references. For consequential facts, follow its citations to appropriate primary or authoritative sources.
Can AI check its own facts?
It can identify claims worth reviewing and sometimes find useful links. It cannot substitute for independent examination of the original evidence.
The test is simple: can another reader click the link and see why the claim is justified? If not, it is not ready to publish.