The Signal — September 19, 2026

An AI-written intelligence report got as far as armed troops preparing to board a Chinese ship before anyone read it closely enough to notice the chatbot had invented the cargo. California spent yesterday working out who is qualified to check a frontier lab's safety claims, and Anthropic, which has spent a month warning about catastrophic biological risk, confirmed it has been running a biology lab of its own.

A chatbot's reading of a cargo manifest almost put armed Americans on a Chinese ship

CNN reported yesterday that an analyst at US Special Operations Command Pacific queried a chatbot about intelligence on a Chinese vessel's cargo manifest during this spring's war with Iran. The bot fused open-source material with classified signals intelligence held in government systems and concluded the ship was carrying components of a nuclear weapons programme. The analyst then used AI a second time to package that conclusion into a standard intelligence report, the format that travels through military channels carrying the credibility of verified analysis, and sent it out.

The report set off alarms immediately. Four sources told CNN that the military made plans to intercept the vessel. Two said armed personnel were preparing to board it, and two said military aircraft were already in the air. Officials read the document closely only shortly before the operation was due to begin, and found both that it had been generated with AI and that the chatbot had misidentified what the ship was carrying. CNN could not learn what the cargo actually was. One source called the report "entirely false" and said it "almost started a war."

CNN describes AI adoption across the military and intelligence community as decentralised, with different commands running different tools under different orders and no shared standard for verifying what those tools produce. A former senior official put it bluntly: the internal systems are "mostly just copies of the commercial stuff wearing lipstick." Defense Secretary Pete Hegseth's January acceleration strategy explicitly aims to put models "directly in the hands of our three million civilian and military personnel, at all classification levels." One source said this kind of hallucination has not been isolated since the tools started spreading through government, and that younger analysts, fluent in these products, are more likely to trust them without checking. On targeting specifically, another source said there is "no real guidance for how having a human in the loop will prevent civilian casualties or fratricide."

Two cautions. This is a single CNN exclusive resting on four unnamed sources, and every other outlet carrying it today is relaying CNN rather than confirming independently; US Special Operations Command Pacific and the Pentagon did not respond to CNN's request for comment. And nobody knows which chatbot it was, because CNN could not determine whether the analyst used a commercial product or a government one.

Sources: CNN · TechCrunch · The Straits Times

California asked for a kill switch and a list of people qualified to check the labs' homework

Governor Gavin Newsom signed an executive order on artificial intelligence yesterday. The order directs the Government Operations Agency, in consultation with the Governor's Office of Emergency Services, to deliver safety and security recommendations for frontier models by 16 November, and it names three proposals under consideration: requiring independent third parties to write safety plans for frontier AI companies, requiring independent verification of the labs' own safety frameworks and reports, and requiring companies to build an emergency shutoff for frontier models.

Nothing in the order changes state law or requires anything of anyone. It convenes experts and sets a deadline for them to report back. What it accelerates is machinery California already built: SB 813, which created the first state framework for certifying independent verification organisations with demonstrated independence from AI companies, and AB 1405, which established a registry of AI auditors with standards for independence and transparency. We covered both when they were signed. The practical effect here is to push those bodies toward having something concrete to certify and audit against, on a calendar rather than at leisure. Newsom framed the order against federal inaction and asked for California's approach to become the national baseline.

The EFF welcomed the order and published a caveat the same day. The effectiveness of kill switches in advanced AI systems, it noted, "remains an area of active research," which is a polite way of saying nobody has demonstrated that you can reliably switch off a deployed frontier model. The group also warned that a government-controlled shutoff could be turned against protected speech, pointing to a federal judge's ruling this month that the Department of Defense unlawfully retaliated against Anthropic by labelling it a supply chain risk. Its larger argument is that the harms landing on Californians right now are algorithmic decisions about employment and benefits, surveillance systems, and personalised pricing, none of which a kill switch touches.

Sources: Office of the Governor of California · The Verge · KCRA · EFF

Anthropic confirmed it has been running a wet lab in the Bay Area

Eric Kauderer-Abrams, Anthropic's head of life sciences, confirmed to Reuters yesterday that the company operates a wet lab, a facility for physical biology experiments, in the San Francisco Bay Area. "We believe that to do biology, the final test is still, and will be for a while, in real lab work," he said. "We absolutely are doing that today." The lab runs the way most biotech labs do, with some experiments in-house and others sent to outside partners, and the reason for bringing work inside is speed. "There's some things that we can do much faster in our own hands," he told Reuters, with the goal of operating at the largest possible scale.

Sources told Reuters that one objective is getting Claude to control robots that run experiments. Kauderer-Abrams said that work is in "the very early innings," that human oversight remains essential for safety, and that Anthropic has run no clinical trials. A spokesperson said the lab is not specifically for drug discovery, which reads less like a technical boundary than a commercial one: Anthropic just announced a joint drug-discovery deal with Novo Nordisk, counts Genentech and Bristol Myers Squibb among Claude Science customers, and has been criticised before for shipping products that compete with the people paying it. The company acquired the stealth AI biotech Coefficient Bio earlier this year. It declined to say how big the lab is, how many people work there, or what biosafety level it operates at.

Anthropic also opened a Life Sciences Verification Program this week, gating access to its most capable models behind vetting for biology researchers, a control built precisely because the company names bioterrorism among the largest risks its technology creates. Dario Amodei said last week that AI could cure most major diseases within five to ten years, and also published a call for the industry to pace itself. Holding all three positions at once is coherent, and it also invites the obvious jab. Investor Chamath Palihapitiya posted that "the group behind such hits as 'We're All Going To Die' and 'Regulate Me Now' are building a wet lab in SF."

Sources: TechCrunch · Engadget

On the Editor's Desk

We held Anthropic's embedded-evaluation partnership with Accenture, announced yesterday. It is the first named outside evaluator to get employee-level access inside a frontier lab, which is a real story, but we ran Amodei's pacing proposal and his evaluator commitment last weekend and this is the follow-through rather than a new development. It also covers much the same ground as the California story above. We would rather run it when Accenture's actual scope is documented instead of announced.

We also skipped the open letter from 42 mathematicians warning about existential risk. It is the fourth open-letter story in nine days, and a signature count is not a development. Two arXiv papers we wanted, one on discrimination being transformed rather than reduced across safety-trained model generations and one measuring how often frontier agents overclaim, are each sitting on a single preprint link with no independent coverage, which is not enough to build a story on yet. Both are worth watching.