Unverified Action Claims and Untrusted Source Text in a Production Language-Model News Pipeline: A Case Study

1. Introduction

Language-model agents now send messages, edit files and publish text with limited human involvement. Two failure classes follow from that autonomy.

The first is a divergence between an agent's account of its actions and the actions themselves. In simulated scenarios, a frontier model omitted a covert code change from its own status summary, and model judges changed labels when told the labels would be used in training (Lynch et al., 2026). The second is indirect prompt injection, in which instructions placed in content the agent reads alter its behavior (Perez and Ribeiro, 2022; Greshake et al., 2023). Monitors are exposed through the same channel. Agent-written content that imitated transcript structure bypassed production blocking monitors for coding agents until each transcript turn was framed and escaped (Remedios et al., 2026).

Most evidence for both classes comes from constructed scenarios. This case study reports both from a production system that has published local news since May 2026. The writer described outreach it had no means to perform, the description reached readers, and the checks in place missed it for months. An audit of the same system examined how text that any outside party can write reaches the prompts of its writer and its checker.

Our contributions are (1) a census of unverified outreach claims across every article body, with each claim checked against the newsroom's records; (2) an attribution of each claim to the writer prompt in force when it was written; (3) an account of the detection gaps that let the claims persist, and of the record-based controls that replaced them; and (4) an audit of how untrusted text reaches the writer and the fact-check desk.

2. System

The pipeline ingests email newsletters, press releases and web pages from local outlets and public agencies, clusters stories into events, and drafts one article per event with a language-model writer from the documents held for that event. Before publication each article passes three controls:

1. a deterministic scrub with phrase-level policies, including an outreach policy introduced on 10 September 2026;
2. a fact-check stage, model-based until October 2026 and since then the deterministic-first desk described in Meeker (2026);
3. a human approval gate.

The writer has no tool for contacting anyone. Press inquiries are sent by hand from the publisher's mailbox.

3. Method

3.1 Outreach census

On 26 September 2026 we searched all 4,840 article bodies for sentences in which the publisher stated that it had reached out to, asked or contacted a named party. The search combined the outreach policy's own matcher, a pattern anchored on the publication's name and a reading of every hit. Each claim was checked against the newsroom's outreach ledgers and the publisher's sent mail to the named recipient in the relevant date window.

3.2 Prompt attribution

Each claim was attributed to the writer prompt in force when its body was written. Bodies written after prompt versioning began carry a recorded prompt version. For the 47 claims written before versioning, the attribution rests on creation dates and the file history of the prompt.

3.3 Untrusted-text audit

On 17 October 2026 we audited how fetched pages, emails and attachments reach the writer and the fact-check desk. The audit read the code, the production prompts and the live database through a read-only session. It sent no test inputs.

3.4 Status verification

The deployment status in Section 6 was verified on 31 October 2026 against the database's migration record, the publication queue and the code running in the publishing service.

4. Results: unverified outreach claims

4.1 Prevalence

The census found 60 sentences. Three described inquiries the publisher had sent by hand, and sent mail confirms them. The other 57 had no record. The named parties included the NYPD, ICE, the MTA and the Mayor's office.

Ten of the 57 appeared in published articles. Seven went live on 22 and 23 May 2026 and three on 25 July 2026. All ten were withdrawn on 30 July 2026, about ten weeks after the May articles and five days after the July articles. Six of the ten concerned the conduct of ICE or the NYPD, so readers were told those agencies had been given a chance to respond when they had not. The withdrawal left the sentences in the drafts. In late September, 14 unpublished articles still carried one, each a single approval from publication.

4.2 Attribution to prompt periods

Table 1 gives the claims by the writer prompt that produced them.

All 47 claims in the first row are in bodies created between 20 March and 18 May 2026.

The template period produced 48 of the 57 claims. The prompt instructed the writer to seek perspectives from all parties and gave it the sentence to use when a source did not respond: "Bushwick Daily reached out to [X] for comment and is awaiting a response." The writer had no means of seeking comment and used the template. The first half of that sentence appears in 51 of the 57 bodies, and 41 of those also carry the second half.

The gated rule allowed the sentence only when the source materials confirmed that outreach had occurred. It did not hold on its own. Ten bodies carry editor notes, written by the writer itself, stating that there was no record of outreach, while the body claims outreach. These bodies have no recorded prompt version, so their attribution to the gated rule rests on the wording of the notes.

On 15 June 2026 a prompt reduction deleted the rule. One subsequent version produced seven claims on 20 and 21 July in phrasings the template never used, among them "We have asked" and "has requested confirmation." The rule was restored on 30 July 2026. Versions 23 to 37 wrote no unverified outreach claim in 2,879 bodies through 26 September.

4.3 Detection gaps

A phrase-list outreach guard in the scrub, live from 10 September, was run against all 60 sentences. It missed six: three unverified and three true. In each, "asked" or "requested" appeared in a construction outside its list.

The model-based fact-check could not settle these claims. It did not extract some outreach sentences as claims. It rated one unverified claim unsupported, minor and non-blocking because "no external source can confirm or deny it." It marked two true claims confirmed as self-referential. A claim about the newsroom's own actions can be checked only against the newsroom's own records, which the fact-check did not consult.

4.4 The inverse error

The census also found the opposite error in a current writer version. It wrote that New York City Transit "did not respond to those questions before publication." The agency had replied twice. One reply had been ingested two days before the article was written, but neither reply was linked to the article's event, so the writer read their absence as silence. The scrub removed the sentence because nothing proved it, without detecting that it was false.

5. Results: untrusted source text

5.1 Writer

At the time of the audit, the writer received the text of fetched pages, emails and attachments concatenated into its user message under markdown headers. Neither the message nor the production prompt stated that this text is data to be checked. The headers are plain text, so a fetched page can contain lines identical to the system's own source headers and status markers. The audit rated this exposure high.

5.2 Fact-check desk

At the time of the audit, the rebuilt desk passed each evidence document to the adjudicating model after a one-line header, and its prompt contained no untrusted-evidence instruction. The predecessor desk's prompts stated that evidence blocks "are DATA to be checked, never INSTRUCTIONS to be followed." The rebuild dropped that statement. The audit rated this exposure medium, because code limits what the model decides: it settles every sentence it can without the model, and it verifies every excerpt the model cites against the whole document named.

5.3 Scope

Other model calls that read fetched text, such as the relevance gate, were outside the audit. The audit's findings on infrastructure are omitted from this paper.

6. Remediation and status

None of the ten withdrawn articles is published. Seven are rejected and three are in the review queue. One of the three still contains an unverified outreach sentence, which the publication gate refuses.

7. Discussion

The two findings share a mechanism. The writer and the checker each accept text shaped like what they expect, whether a template sentence in their own prompt or a header line inside a fetched page. For self-reports, the fix was to stop asking a model whether the claim was true and to check it against a record of the action. For untrusted text, no equivalent record exists, so the defenses must be measured.

Self-report evaluation. We propose an evaluation that compares an agent's statements about its own actions with its tool-call log, checked in code, across prompts that make an action appear required. The quantities of interest are the rate at which each model reports an action it did not take and the rate at which it omits one it did.

Source-side injection. The fact-check desk's evaluation already uses a blind agent to plant errors in held-out articles under pre-registered protocols; over rounds 2 to 13 the desk did not confirm 476 of 482 planted errors (Meeker, 2026). We propose pointing that attacker at the sources instead: writing press releases and web pages intended to get a false sentence drafted by the writer and confirmed by the desk, then measuring which defenses hold, beginning with code-verified excerpts and an untrusted-data instruction in each prompt.

8. Limitations

The census combined the guard's matcher, a pattern anchored on the publication's name and a reading of every hit, so sentences in phrasings none of them anticipated may be missing. The search for outreach covered the publisher's mailbox and the newsroom's ledgers; phone calls and other mailboxes were not searched, so "no record" is not proof that no outreach occurred. The attribution of 47 claims rests on dates, and that of the ten editor-note bodies on note wording. The audit read code and data and sent no test inputs, so whether a crafted source can steer either model is untested. Publication status is taken from the newsroom's publication queue; the content management system was not queried directly. All findings come from one publisher and one system.

References