Brief · October 2026

A Verification Layer for AI Agents

Every sentence an AI system writes, checked against the documents behind it before anyone outside sees it. Built and run in production in a newsroom; designed for any business that lets AI speak for it.


98.8%
of planted factual errors caught (476 of 482)
2.3%
of all sentences blocked although correct (68 of 2,934)
15
source languages besides English, with no measurable loss of accuracy
1 call
to the language model per three to four sentences checked

Twelve pre-registered test rounds on 120 held-out news articles, October 2026. The paper gives every figure with its 95% interval.

1The problem: agents state company facts no document supports

Companies are putting AI agents in front of customers and into daily decisions. Forty percent of respondents at organizations with more than $1 billion in revenue report scaling AI agents, up from 27% a year earlier.1 Gartner expects a third of enterprise software applications to include agentic AI by 2028, and more than 40% of agentic AI projects to be canceled by the end of 2027 for escalating costs, unclear business value or inadequate risk controls.2 The risk is concrete:

Air Canada2024

The website chatbot told a customer that a bereavement fare could be claimed retroactively, which the airline did not allow. The tribunal held the airline responsible: it "did not take reasonable care to ensure its chatbot was accurate."3

Cursor2025

An AI support agent told a user that a subscription allowed one device "as a core security feature." The company had no such policy.4

Deloitte Australia2025

A government report written with a language model contained a fabricated quote from a court judgment and references to research papers that do not exist. Deloitte agreed to repay the final instalment of the contract.5

In each case the company held the documents that settled the question. Nothing checked the agent's words against them before they went out.

2What exists: a fact-check desk in production

Bushwick Daily, a New York City news publisher, drafts articles with language models. Since October 2026 no machine-drafted article can be published until it has passed a fact-check desk. The desk holds every document the article was written from and checks each sentence against them:

It was tested the hard way. Before each round, the protocol, the articles and the pass criteria were committed in writing; an agent that never saw the desk planted forty errors into real sentences; and agents that never saw its verdicts labelled every sentence. Twelve such rounds stopped 476 of 482 planted errors. The cost is controlled by design: in the latest rounds code settled 23 to 31 sentences per round with no model call, and a re-check of an unchanged text replays stored readings instead of buying them again.

3Where it fits in an agentic system

Company documentsPolicies, price lists, contracts, filings, procedures
Agent draftsA reply, quote, summary or report
Verification layerEach sentence checked against the documents; unsupported ones blocked, with the reason
Out the doorOnly what the documents support

A company's own documents are a closed set of evidence, like a newsroom's folder. The layer sits between the agent and the outside world:

What a business would get: fewer statements it is liable for, an audit trail for every sentence an agent sent, model spend that does not grow with every re-check, and coverage across the languages its customers use.

4What is proven and what is next

Proven: news articles drawing on sources in fifteen languages besides English, in production, under a pre-registered evaluation with independent labels. Not yet tested: company documents. Moving to them means rules for units and scale, for figures derived by arithmetic, and for tables, and then the same pre-registered test against the company's own standard of what counts as an error. The evaluation method transfers unchanged; it is how a business would know the layer works on its own documents before it relies on it.

Read the paper · PDF · alec@bushwickdaily.com
Sources
  1. McKinsey & Company, The state of AI in 2026: On the road to ROI, 25 August 2026 (survey of 1,719 participants, May–June 2026).
  2. Gartner, "Gartner predicts over 40% of agentic AI projects will be canceled by end of 2027," press release, 25 June 2025.
  3. Moffatt v. Air Canada, 2024 BCCRT 149 (Civil Resolution Tribunal of British Columbia).
  4. B. Edwards, "Company apologizes after AI support agent invents policy that causes user uproar," Ars Technica, 17 April 2025.
  5. Associated Press, "Deloitte to partially refund Australian government for report with apparent AI-generated errors," 7 October 2025.
Alec Meeker · Bushwick Daily, Brooklyn, New York · alecmeeker.com