How to Check an AI-Generated Market Map for Hallucinated Data
Bret van Putten5 min readUpdated October 8, 2026
On this page
- Three collapses, one root cause
- KPMG: 40 of 45 citations failed
- EY Canada: broken links dressed as sources
- Deloitte Australia: a quote from a judge who never said it
- What the three have in common
- The checklist: five checks before a figure goes to a client
- What this means for a market map, specifically
- The through-line
- Sources
- FAQ
Three professional-services firms published a client-facing report in the past year that turned out to be substantially AI-hallucinated. Not a rough draft caught internally: a delivered, paid-for report, read by a client, before anyone checked the citations.
Our piece on how search firms use AI already covers the best-known of these, Deloitte’s Australian government refund. Two more happened since, and between the three, a pattern emerges that is more useful than any single story: the exact shape a hallucinated report takes, and the checklist that catches it before a client does.
Three collapses, one root cause
KPMG: 40 of 45 citations failed
KPMG published “Total Experience: Redefining Excellence in the Age of Agentic AI” in October 2025. GPTZero’s investigation, later confirmed by the Financial Times, checked all 45 citations. Five held up. Twenty-eight were hallucinations: real-sounding titles paraphrased or fabricated around a genuine source. Twelve were too vague to verify at all.
The report also cited case studies from UBS, the NHS, Swiss Federal Railways, and Transport for London describing AI deployments those organizations don’t recognize. NHS Greater Manchester said outright that the report’s description of its AI use didn’t match what the trust actually does. KPMG pulled the report in June 2026.
EY Canada: broken links dressed as sources
EY Canada published “Points of Attack: Uncovering Cyber Threats and Fraud in Loyalty Systems” in late 2025, credited to two partners and a senior manager. GPTZero’s investigation found 16 of its 27 citations were fabricated, misattributed, or linked to pages that returned nothing. Footnotes pointed to Forbes, McKinsey, Gartner, and WIRED articles that were never published.
The report also stated the loyalty-rewards market was worth $200 billion in one place and cited the same figure elsewhere as the value of unclaimed points, a contradiction nobody caught before release. EY withdrew the study and said it was reviewing how it was produced.
Deloitte Australia: a quote from a judge who never said it
Deloitte delivered a A$440,000 report to Australia’s Department of Employment and Workplace Relations reviewing the IT system behind the country’s welfare compliance penalties. A University of Sydney researcher found a quote wrongly attributed to a federal court judge and citations to academic papers that don’t exist.
Deloitte reissued the report and refunded A$97,000 of the contract in October 2025, about a quarter of the total fee.
Every figure in a MergeSearch Market Map names its source: a link where the record is public, the tool and retrieval date where the data is licensed.
Book a callWhat the three have in common
| KPMG | EY Canada | Deloitte Australia | |
|---|---|---|---|
| Citations checked | 45 | 27 | Not fully disclosed |
| Found fabricated or unverifiable | 40 | 16 | 12+ specific errors named |
| Named a real entity incorrectly | Yes (UBS, NHS, TfL) | No | Yes (a federal judge) |
| Caught internally before release | No | No | No |
| Caught by | Outside researchers (GPTZero) | Outside researchers (GPTZero) | A government researcher reading it |
Every failure was caught by someone outside the firm that produced it, after the report had already reached a client. None of the three had a step where a human traced a citation back to its source before publication. That single missing step, not the AI tool itself, is the actual defect.
The checklist: five checks before a figure goes to a client
None of these require special tooling. They require someone to actually do them, which is the step all three incidents skipped.
- Open every citation. Not the footnote text, the underlying page. If it 404s, doesn’t exist, or doesn’t say what the report claims, the citation fails. This single check is what caught both KPMG and EY.
- Check the entity’s own words. If a report names a company, agency, or person, confirm they said or did what’s attributed to them. NHS Greater Manchester’s public denial is what unraveled KPMG’s case studies.
- Cross-check any figure that appears twice. EY’s report cited the same $200 billion number as two different things. A number that can’t stay consistent within one document didn’t come from a checked source.
- Trace every number to a primary source, not a summary of one. A market size, a wage figure, or a comp band should point to an SEC filing, a BLS table, or a Census dataset, not a paraphrase of one.
- Label what’s a fact, an estimate, and an inference, separately. A filed compensation figure and an adjusted estimate for a private company are different kinds of claim. Collapsing them into one confident number is what makes an AI-drafted report read like it’s been checked when it hasn’t.
See the checklist applied to a real search: a worked market map, sourced end to end.
Read the worked exampleWhat this means for a market map, specifically
A market map carries the same categories of claim these three reports got wrong: a market size, a compensation range, a named company’s activity. The method behind every map we build runs on the same discipline as the checklist above, applied before a figure is ever drafted, not after a client asks where it came from.
What’s actually in a Market Map breaks down which module answers which question and from which source. The distinction that matters isn’t whether AI touched the draft.
Every firm above used AI to draft; none of them checked it. It’s whether every claim in the finished document can be traced back to something a client could open and confirm themselves.
The through-line
The Opportunity Monitor runs on the same standard: every fact sourced, every estimate labeled, checked against the primary record before it reaches you, not after.
Book a call: a sample on a brief of your choice, with every figure naming its source, and every public source linked.
Sources
Frequently asked questions
How do you know if an AI-generated report has hallucinated citations?
Open every footnote and confirm it resolves to a real, specific source that says what the report claims it says. GPTZero's investigations into the KPMG and EY reports both used this exact method: clicking through each citation rather than trusting the formatting.
Fabricated citations often look identical to real ones until someone actually follows the link.
What is the most common AI hallucination pattern in business research?
Paraphrasing a real source into a fake one. GPTZero's KPMG investigation found 28 of 45 citations followed this pattern: a real organization or publication attached to a claim that source never made. A citation that exists is not the same as a citation that says what it's cited for.
Can AI be trusted for a market map or comp benchmark?
For drafting and structure, yes. For the underlying facts, only once every figure is traced to a primary source: an SEC filing, a BLS wage table, a Census establishment count. The three incidents below all involved AI drafting a claim and nobody checking it against a primary source before it reached the client.