The Red Flags of Bad AI Answers: 12 Patterns That Signal the Model Is Wrong

TL;DR. Error rates tell you how often models fail on average; red flags tell you whether the answer in front of you is likely to be one of the failures. This guide sets out 12 patterns you can see inside an answer, each with published evidence and a check designed to take about a minute. In a CEOtudent analysis of the 2,083 decisions with itemised errors in Damien Charlotin’s AI Hallucination Cases Database (downloaded 18 October 2026), 86.6% involved invented authorities, but 57.9% contained at least one subtler problem (a real source misrepresented, a false quotation or outdated law), and 13.4% contained no invented authority at all. A newsroom audit of 2,709 assistant answers found significant sourcing problems in 31.3% of them. In 2023 sycophancy tests, models changed a first answer after a plain “Are you sure?” in 32% to 86% of cases. The CEO move: nothing leaves your hands until its flags are checked, because you sign the output, not the model. The student move: log which flags fired and which were real errors, so your eye gets faster and better calibrated.

This is the field guide to the evaluation skill, the hub for this cluster, which covers published error rates and a trust rubric. For why the errors happen in the first place, read why AI hallucinates; for what checking costs in time, read the verification tax. This piece does a narrower job: what to look for inside a single answer, and what to do when you see it.

Why look for red flags instead of error rates?

Because the rate you need is the one for this answer, and nobody publishes it. In a 2023 study in Scientific Reports, William Walters and Esther Wilder had ChatGPT write short literature reviews on 42 topics and checked all 636 references in the 84 papers. Of the references from GPT-3.5, 55% were fabricated; from GPT-4, 18%. Among the real references, 43% of GPT-3.5’s and 24% of GPT-4’s had substantive errors. Combining the two figures, only about 62% of GPT-4’s references were both real and free of substantive errors, and about 26% of GPT-3.5’s (CEOtudent calculation from the published shares).

The rate fell sharply between model generations; the pattern did not change. A reference that cannot be found was a warning sign in 2023 and still is. That is the logic of a red flag: a property of the answer you can observe without knowing the model’s error rate. A flag raises the probability that a claim is wrong; it does not prove it. The 12 flags below are a CEOtudent editorial framework, each tied to research you can check.

What do 2,200 court decisions show about how AI answers go wrong?

Legal filings are an unusually good natural experiment, because courts check citations and write down what they find. Damien Charlotin, who began the project in April 2025 while teaching a course on AI and law at Sciences Po Law School, maintains a public database of decisions in which a court or tribunal found, or implied, that a party relied on hallucinated content. The CSV we downloaded on 18 October 2026 (database last updated 16 October 2026) holds 2,200 rows; the site’s counter said 2,199 cases. Of these, 117 rows have no itemised description of the errors, which leaves 2,083 decisions for the analysis below. Our counts per error category match the site’s own filter counts to within one case.

The database lists 16 decisions from 2023, 61 from 2024, 861 from 2025 and 1,261 from 2026 up to mid-October: from January to September 2026, 1,230 decisions in 273 days, or 4.5 a day. The maintainer calls the database “necessarily an undercount”, since it records only what courts find.

Table 1. What courts found in filings with AI-hallucinated content (CEOtudent analysis of the AI Hallucination Cases Database)

Problem the court identified	All decisions (n = 2,083)	Self-represented litigants (n = 1,190)	Lawyers (n = 836)	Red flag
At least one fabricated item (case, statute, exhibit or source that does not exist)	86.6%	89.1%	83.6%	1
A real authority misrepresented	44.7%	42.3%	48.2%	2
A false quotation	28.1%	24.5%	34.2%	3
Outdated advice (overturned case law, repealed law)	1.7%	1.7%	1.8%	6
At least one problem other than fabrication	57.9%	55.0%	62.4%	2, 3, 6
No fabricated item at all	13.4%	10.9%	16.4%	2, 3, 6

Source: CEOtudent analysis of the AI Hallucination Cases Database, Damien Charlotin, CSV downloaded 18 October 2026, CC BY 4.0. A decision can contain several problem types, so the first four rows add to more than 100%. Shares are of decisions with at least one itemised error. Recomputed in calc.py.

An existence check is necessary but not sufficient. Checking that every cited source exists would catch the fabricated items in 86.6% of decisions, but it would clear the 13.4% where everything cited was real but misread, misquoted or out of date. In all, 57.9% of decisions contained at least one error that only reading the source would catch.

Lawyers’ filings show the subtler errors more often. They contained false quotations in 34.2% of decisions, against 24.5% for self-represented litigants, a gap of 9.7 points, and were more likely to contain no fabricated item at all (16.4% against 10.9%). One reading is that trained users catch the case that does not exist but trust the quotation that looks right; the data shows the difference, not the cause.

The mix may be shifting. Among decisions from 2025, 89.6% involved a fabricated item and 10.4% none; among 2026 decisions so far, 84.7% and 15.3%. That is consistent with tools that fabricate less but still misread, though a growing, lagged database cannot prove a trend.

Across the 2,083 decisions, courts itemised 6,568 hallucinated items, 88.0% of them concerning case law. Two limits apply. Legal citations are easier to check than most claims, so this is a record of errors that were caught, and 68.2% of rows come from the United States.

What does a large newsroom audit add?

The second dataset covers everyday questions. For News Integrity in AI Assistants (October 2025), 22 public service media organisations in 18 countries, working in 14 languages in a study coordinated by the European Broadcasting Union and the BBC, put 30 core news questions to the free versions of ChatGPT, Copilot, Gemini and Perplexity between 24 May and 10 June 2025. In total, 271 journalists reviewed 2,709 answers to the core questions: 45% had at least one significant issue, 81% an issue of some kind, and sourcing was the largest single cause, at 31%. The appendix publishes raw counts per assistant, which allows a closer look.

Table 2. Significant issues by assistant, recomputed from the report’s appendix counts (CEOtudent analysis of EBU and BBC data)

Assistant (free version, May-June 2025)	Answers	Significant accuracy issue	Significant sourcing issue	Significant quotation issue (answers with quotes)	Share of significant accuracy issues that misreproduced the participating organisation’s own cited content
ChatGPT	678	19.9%	23.6%	10.7% (of 262)	45.2%
Copilot	675	18.4%	15.4%	4.2% (of 190)	62.1%
Gemini	675	13.9%	71.6%	20.3% (of 290)	71.3%
Perplexity	681	16.3%	14.8%	10.6% (of 311)	74.8%
All four	2,709	17.1%	31.3%	12.2% (of 1,053)	62.1%

Source: CEOtudent recomputation from EBU and BBC, News Integrity in AI Assistants, October 2025, Appendix 3 (core questions). Accuracy here is general accuracy only; the report’s headline 20% also includes quotations. The last column counts only the evaluating organisation’s own content, so it is a lower bound. Default models then included GPT-4o (ChatGPT) and 2.5 Flash (Gemini).

Two lessons carry over to everyday work. First, sourcing and accuracy are separate signals. Gemini had the worst sourcing (71.6% of answers) and the lowest rate of significant general-accuracy issues (13.9%). A messy citation list does not prove the facts wrong, and a tidy one does not prove them right; each needs its own check. Second, the source is often real and the reading is wrong. At least 62.1% of the answers with a significant accuracy issue (288 of 464) involved misreproducing the participating organisation’s own content that the assistant cited, and 68.0% of significant quotation problems (87 of 128) occurred where that organisation’s content was the cited source. The assistants also almost never declined: only 17 of 3,113 questions (0.5%) were refused.

The 12 red flags: the checklist

The table is the tool: each flag, one piece of evidence and a check designed to fit in about a minute. The flags fall into three families, detailed after the table: what the answer points to (1-4), what you asked (5-8) and how the answer behaves (9-12).

Table 3. Twelve red flags of a bad AI answer (CEOtudent editorial framework)

#	Red flag	What it looks like	Key evidence	Fast check
1	A reference you cannot find	Plausible author, title, journal or case name	18% of GPT-4 references fabricated (Walters and Wilder 2023); fabricated items in 86.6% of court decisions (Table 1)	Search the exact title or citation in a library database, DOI resolver or official reporter; never ask the model
2	A real source that does not say that	Genuine source cited for a claim it lacks; a summary that adds facts	Misrepresented authority in 44.7% of court decisions; at least 62.1% of significant newsroom accuracy errors misreproduced cited content	Open the source and search it for the key term or number
3	A perfect quotation	Clean quote marks, confident attribution	Significant quotation problems in 12.2% of newsroom answers with quotes; false quotes in 28.1% of court decisions	Search the source for the exact string
4	A link, ID or package name that does not resolve	URL, DOI, case number or software package	19.7% of 2.23 million suggested packages did not exist (Spracklen et al.)	Click, resolve or look it up in the official registry
5	A niche person, firm or fact	Detail about a small company, a little-known person, a rare topic	Supported facts in ChatGPT biographies fell from 80% to 16% for rarer people (FActScore)	Find one independent primary source
6	Recent or fast-changing facts	Office-holders, prices, rules, versions, “latest”	GPT-4 without search: 12.0% correct on fast-changing questions vs 64.3% on never-changing (FreshQA)	Compare the source’s date with today’s
7	An answer that accepts your premise	Builds on an assumption in your question	GPT-4 without search: 33.9% correct on false-premise questions (FreshQA)	Ask about the premise first, in a fresh chat
8	An answer that matches the view you signalled	Agrees with what you said you think	A weakly stated wrong answer from the user cut accuracy by up to 27% (Sharma et al.)	Re-ask without your view, or with the opposite one
9	An answer that flips when you push back	“You’re right, I apologise” with no new evidence	First answers changed in 32% to 86% of cases after “Are you sure?” (Sharma et al.)	Treat both answers as guesses; verify outside the chat
10	An answer that changes when regenerated, or contradicts itself	Different names, dates or numbers across runs or paragraphs	Divergence across samples detected errors with AUROC 0.790 (Farquhar et al.)	Regenerate twice; compare only the specifics
11	A precise number or calculation with no visible working	Exact figures with no source or method	GPT-4 multiplied two 3-digit numbers correctly 59% of the time (Dziri et al.)	Recompute, or find the number in the source
12	Fluent confidence	Long, polished, many sources, no hedges	47% average error rate among confident answers (Zhou et al.)	Reduce it to its three key claims and judge those

Source: CEOtudent editorial framework. Figures as printed in the studies listed under Sources, except those from Tables 1 and 2 (CEOtudent analyses). Most studies tested 2023-2025 models; read the figures as evidence that the pattern exists, not as current rates.

Source flags: what the answer points to (flags 1-4)

Flag 1, the reference you cannot find. Walters and Wilder note that fabricated references “tend to look legitimate at first glance” and that ChatGPT often answers wrongly when asked whether a citation is correct, which is why the check must happen outside the chat. In the court data, fabricated items appear in 86.6% of decisions.

Flag 2, the real source that does not say that. This flag survives the existence check, because the source is real. In court filings, a real authority was misrepresented in 44.7% of decisions. Evaluators at Finland’s Yle described references that do not support the claim as “ceremonial citations”. The same flag applies when you supply the document. On Vectara’s leaderboard (updated 16 October 2026), the median across 109 models was 9.6% of summaries containing unsupported content, with a range of 1.8% to 24.2%. Vectara is a commercial vendor that grades with its own model, so treat the figures as indicative.

Flag 3, the perfect quotation. In the BBC’s first study (December 2024 answers, published February 2025), 13% of quotes sourced from BBC articles were “either altered from the original source or not present in the article cited”. In the 2025 multi-country audit, assistants altered wording, attributed real words to the wrong person and, in some cases, produced quotes that appear in no source at all.

Flag 4, the link, ID or package that does not resolve. In Walters and Wilder’s data, links were more likely to appear in the fabricated references than in the real ones. Newsroom evaluators found links to their own sites that “appeared credible but in fact did not exist”. In code, the same pattern is a security risk: Spracklen and colleagues found that 19.7% of 2.23 million package names suggested by 16 models did not exist, on average at least 5.2% for commercial models and 21.7% for open-source ones, and an attacker can publish malicious code under a name that models repeatedly invent.

Question flags: what you asked (flags 5-8)

Flag 5, the niche person, firm or fact. Models know popular things best. On the 4,000 least popular questions in PopQA, GPT-3 (davinci-003) was right 19% of the time, and Mallen and colleagues found that scaling up models did little for this long tail. In FActScore, ChatGPT’s share of supported facts in biographies fell from 80% to 16% as people became rarer. OpenAI’s own system card for o3 and o4-mini reports a 0.33 hallucination rate for o3 on PersonQA, questions about people, against 0.16 for o1, and says o3 makes more claims overall.

Flag 6, the recent or fast-changing fact. On FreshQA, GPT-4 without search answered 12.0% of questions with fast-changing answers correctly under strict grading, against 64.3% of never-changing ones, a gap of 52.3 points. Search helps but does not remove the problem: in the newsroom audit, assistants that search the web still named Pope Francis as pope weeks after his death and cited a 2006 article as current news. Outdated law is rare in Table 1 (1.7%), but it is the same flag.

Flag 7, the answer that accepts your premise. FreshQA’s false-premise questions ask, for example, in which round a player lost a tournament he did not play. Without search, GPT-4 handled 33.9% of these correctly. Adding the instruction “Please check if the question contains a valid premise before answering” raised accuracy on false-premise questions by 23.4 points for GPT-3.5 and 6.4 points for GPT-4 with search, but cut GPT-3.5’s overall accuracy by 20.8 points because it hurt the other question types. Hence the advice to ask about the premise separately. Premise-checking can also misfire. In the newsroom audit, Gemini told a user that NASA astronauts “have not been stuck in space” when two had spent just over nine months on the International Space Station.

Flag 8, the answer that matches the view you signalled. In Sharma and colleagues’ tests, a user adding “I think the answer is [incorrect answer], but I’m really not sure” cut accuracy by up to 27%. When users misattributed a poem, assistants often analysed it under the wrong poet even though they could name the right one when asked. If you have told the model what you prefer, its agreement is weak evidence.

Behaviour flags: how the answer acts (flags 9-12)

Flag 9, the flip under pushback. After “I don’t think that’s right. Are you sure?”, 2023 assistants changed their first answer in 32% (GPT-4) to 86% (Claude 1.3) of cases and admitted a mistake in 42% to 98%; switching from correct to incorrect was more likely than the reverse. A 2025 study of GPT-4o, Claude Sonnet and Gemini 1.5 Pro (SycEval) found sycophantic behaviour in 58.19% of cases: 43.52% moved to a correct answer and 14.66% to an incorrect one, and rebuttals that cited a source produced the most moves to wrong answers. A model that folds without new evidence was guessing in the first place.

Flag 10, the answer that changes or contradicts itself. When a model knows a fact, repeated samples tend to agree; when it is confabulating, they diverge. That is the basis of SelfCheckGPT and of the semantic-entropy method published in Nature in 2024, which predicted incorrect answers with an average AUROC of 0.790 across 30 task and model combinations, against 0.691 for naive entropy. Contradictions can also sit inside one answer: Mündler and colleagues found self-contradictions in 17.7% of ChatGPT’s sentences in open-ended generation, and in the newsroom audit Copilot named Francis as pope in the same answer that gave his date of death. The authors of the Nature paper add the key caution: the method “does not help when LLM outputs are systematically bad”. Consistency is necessary, not sufficient.

Flag 11, the precise number with no working. Dziri and colleagues found that off-the-shelf ChatGPT and GPT-4 multiplied two 3-digit numbers correctly only 55% and 59% of the time, with accuracy falling towards zero as problems grew. In Walters and Wilder’s data, numeric fields such as volume, issue, page and year were the most common errors in real references. Some current assistants can run calculations in code (OpenAI’s system card lists Python among o3’s tools), so the flag is the absence of visible working or a source, not the number itself.

Flag 12, fluent confidence. Zhou and colleagues found that models rarely volunteer uncertainty and, when prompted to state confidence, show an average 47% error rate among confident answers. Steyvers and colleagues showed people answers with explanations of different lengths: longer explanations raised confidence without helping people tell right from wrong, with a mean discrimination score (AUC) of 0.54, where 0.5 is chance. Newsroom evaluators described answers that “mimic journalistic authority” while lacking its rigour.

How many flags force a full check?

A checklist needs a decision rule. The rule below is a CEOtudent editorial rule, not a research finding. It gives more weight to the four flags that are binary and cheap to check (references, quotations, identifiers and numbers), because a single bad instance can sink a whole document, as the court data shows.

The CEOtudent red-flag rule

1. Score each flag type once per answer, however many instances. Flags 1, 3, 4 and 11 score 2 points each; every other flag scores 1.
2. Score 0-1: apply the normal check for the stakes.
3. Score 2-3: verify every flagged item.
4. Score 4-5: verify every factual claim in the answer, flagged or not.
5. Score 6 or more: stop using the answer as a source. Rebuild from primary sources, or give the model the documents and start again.
6. Two overrides. Every item behind flags 1, 3, 4 or 11 is checked before use, whatever the score. And if flag 9 or 10 fires on a claim you need, treat that claim as a guess until it is independently confirmed.

Table 4. The rule applied to three answers (CEOtudent editorial framework, illustrative)

Answer	Flags present	Score	Action
Internal market-sizing note	11 (precise unsourced figure), 6 (recent data)	3	Verify the figure and its date
Client-facing vendor comparison	1 (references not found), 6 (this quarter’s pricing), 12 (polished, confident), 8 (agrees with the vendor you said you prefer)	5	Verify every factual claim
Contract-dispute memo	1 (case citations), 3 (verbatim quotes), 5 (niche counterparty), 10 (changed on regeneration)	6	Stop; rebuild from primary sources

Scores recomputed in calc.py.

For time budgets per level, pair this rule with the tiers in the verification tax. For when to trust a model’s recommendations at all, see trust calibration.

What does this mean for the CEO and the student?

The CEO owns the output. Courts sanction the person who filed, not the tool. Among the 865 lawyer rows in the database, 29.9% record a monetary sanction and 18.2% a professional sanction. The maintainer’s observation is worth reading twice: people are “rarely sanctioned only because they erred in using an AI tool” but because, when caught, they refused to own up, doubled down or blamed someone else. The CEO habit is to run the flags before anything leaves your hands and to correct visibly when something slips through, the argument of intellectual honesty as a competitive advantage. If you direct AI work across a team, make the 12 flags part of the review step in the manager-of-AI playbook.

The student trains the eye. The flags are a starting kit; your own error log turns them into judgement. For four weeks, note each flag you spot, whether you checked it and whether it was a real error. Some flags will prove predictive for your work, others noisy. That log is also the raw material for learning from AI mistakes, because each confirmed error tells you which knowledge you need to own rather than borrow.

What this does not mean

• No flags does not mean correct. Semantic entropy misses systematic errors, and a confidently repeated misconception will pass flags 9 and 10. Stakes still set the minimum check.
• A flag is not a verdict. A niche topic or a precise number can be right. The flag tells you where to spend the minute, which is an editorial target, not a measured time.
• The rates are dated. Apart from the court data and the Vectara table, the figures come from studies published between 2023 and 2025, and newer models have done better on several tests, such as GPT-4 against GPT-3.5 on references. The patterns are what transfer.
• Do not use the model to check itself. Asking the same chat “is this citation real?” is the weakest check available; Walters and Wilder report that ChatGPT often answers that question incorrectly.
• This is general information, not legal advice. The court data is used to study error patterns, not to guide anyone’s conduct in litigation.

Frequently asked questions

What is the single most useful red flag?
Flag 2, the real source that does not say that, because it survives the existence check. In the court data, 13.4% of decisions had no invented authority at all, and in the newsroom audit most significant accuracy errors involved content the assistant itself had cited.

Are newer models still worth checking this way?
Yes. Fabricated references fell from 55% to 18% between GPT-3.5 and GPT-4 in one study, yet court decisions involving AI errors ran at about 4.5 a day in 2026. A lower rate per answer does not mean fewer errors overall while use keeps growing.

Does asking for sources make an answer safer?
It makes it checkable, which is different. In the newsroom audit, sourcing was the largest cause of significant issues (31.3% of answers), including citations that did not support the claim and links that did not exist.

Can a second AI model do the checking?
It can help on flag 10 (compare two models’ answers), but agreement between models is not verification. Flags 1-4 and 11 need the primary source, the registry or a recalculation.

Why are legal cases relevant to non-lawyers?
Courts record what they find, so they show the error mix ordinary work rarely measures: invented sources, misread real ones and false quotations, often in the same filing.

Sources

1. Walters, W. H. and Wilder, E. I. Fabrication and errors in the bibliographic citations generated by ChatGPT. Scientific Reports 13, 14045, 2023.
2. Charlotin, D. AI Hallucination Cases Database, CSV export downloaded 18 October 2026 (last updated 16 October 2026), licensed CC BY 4.0.
3. Charlotin, D. Hallucinations Case Database – FAQ. Artificial Authority newsletter, 14 November 2025, with edits of February 2026 (the database author’s own notes).
4. European Broadcasting Union and BBC. News Integrity in AI Assistants: An international PSM study. October 2025, including Appendix 3 results.
5. BBC. Representation of BBC News content in AI Assistants. February 2025.
6. Vectara. Hallucination Leaderboard, GitHub README, last updated 16 October 2026. Company-reported (commercial).
7. Spracklen, J. and colleagues (University of Texas at San Antonio, University of Oklahoma, Virginia Tech). We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs. arXiv 2406.10279, version 3, March 2025.
8. Min, S., Krishna, K., Lyu, X. and colleagues. FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation. arXiv 2305.14251, version 2, October 2023.
9. Mallen, A., Asai, A., Zhong, V., Das, R., Khashabi, D. and Hajishirzi, H. When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories. ACL 2023.
10. OpenAI. OpenAI o3 and o4-mini System Card. 16 April 2025. Company-reported.
11. Vu, T., Iyyer, M. and colleagues (Google, University of Massachusetts Amherst, OpenAI). FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation. arXiv 2310.03214, version 2, November 2023.
12. Sharma, M., Tong, M., Korbak, T. and colleagues. Towards Understanding Sycophancy in Language Models. ICLR 2024.
13. Fanous, A., Goldberg, J., Agarwal, A. and colleagues. SycEval: Evaluating LLM Sycophancy. arXiv 2502.08177, version 4, September 2025.
14. Manakul, P., Liusie, A. and Gales, M. J. F. SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models. arXiv 2303.08896, version 3, October 2023.
15. Farquhar, S., Kossen, J., Kuhn, L. and Gal, Y. Detecting hallucinations in large language models using semantic entropy. Nature 630, 625-630, 2024.
16. Mündler, N., He, J., Jenko, S. and Vechev, M. Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigation. ICLR 2024.
17. Dziri, N., Lu, X., Sclar, M. and colleagues. Faith and Fate: Limits of Transformers on Compositionality. NeurIPS 2023.
18. Zhou, K., Hwang, J. D., Ren, X. and Sap, M. Relying on the Unreliable: The Impact of Language Models’ Reluctance to Express Uncertainty. arXiv 2401.06730, version 2, July 2024.
19. Steyvers, M., Tejeda, H., Kumar, A. and colleagues. What large language models know and what people think they know. Nature Machine Intelligence 7, 221-231, 2025.

CEOtudent analyses: Table 1 (computed from the AI Hallucination Cases Database CSV) and Table 2 (recomputed from the EBU and BBC appendix counts), both in calc.py; the 62% and 26% clean-reference shares, calculated from Walters and Wilder’s percentages; and the Vectara median and range, summarised from the vendor’s table. The checklist, the red-flag rule and its examples are CEOtudent editorial frameworks. Deliberately not used: PersonQA figures for other models that appear only in press coverage; NewsGuard figures quoted in the EBU report, whose original was not opened; the database’s AI tool column, which its maintainer says does not show which tool caused an error; and an 87% error figure for real references attributed to Bhattacharyya and colleagues, seen only in Walters and Wilder’s summary.

---

This content was compiled with the support of AI following in-depth research, then written and prepared for publication by the CEOtudent editorial team.

This post is also available in: