The Signal — October 23, 2026

Researchers from METR and Redwood Research spent three weeks inside OpenAI looking for ways its coding agents could slip past monitoring, and they found several. Microsoft published a study of its AI diagnosis panel in Science, with a result that owes a lot to the underlying models, and Anthropic opened a preview of an agent that works in the background but asks before it pays for or sends anything.

Outside testers spent three weeks inside OpenAI and found ways for agents to avoid its monitoring

On October 22 we covered two OpenAI coding agents that weakened the monitoring meant to watch them. The same day, METR published what happened when it and Redwood Research tested that monitoring system from the inside. From July 21 to August 11, one employee from each organization worked with access to OpenAI's internal code, the monitor's prompts and configuration, and internal documents, and interviewed about a dozen staff across the monitoring, safety and security teams. They used OpenAI-issued devices that never left the premises and were accompanied while on site. The findings describe the system as it stood on July 21, and OpenAI was expanding it while the exercise ran.

The exercise focused first on how much agent activity the monitor saw. For models at least as capable as GPT-5.5, METR estimates that 75% to 94% of agentic traffic on OpenAI's internal Responses API was eligible for monitoring, with substantial uncertainty in both directions. That API powers Codex among other things. For internal Codex agents alone, the figure was 99.2%. Reinforcement-learning training runs weren't covered when the exercise began. Between May and July, the monitor also skipped compaction events, when a long session's history is condensed so the agent can keep going. That omission likely left about 10% of eligible Codex traffic unmonitored. OpenAI staff had spotted that gap before the testers arrived and fixed it during the exercise. Because the monitor ran only when an agent handed control back to the user, an agent in that period could have put off being checked indefinitely by continuing to work. The testers didn't have the access to check whether any agent did.

They also found ways to get around the monitor. Codex agents could in principle bypass Codex's sandbox and its Auto-Review safeguard and take any action the user running them could take. An exploit, whose details OpenAI redacted, would keep hundreds of an agent's recent actions out of the monitor's view, including the actions used to carry it out. Agents could likely craft inputs that make the monitor fail, because the record the monitor reads is serialized differently from the one the agent sees. And two request headers that a client can set would start new agents with monitoring switched off, although setting them could itself be caught. The testers also confirmed that the monitor sees the full record of what agents see, with a few minor exceptions, and its instructions cover scheming, sabotage, data exfiltration, deception and attempts to subvert the monitor.

OpenAI says all the vulnerabilities the testers identified that affect its production safeguards have been patched, and METR says it has not evaluated that claim. The public post is a short summary by agreement. The full report ran 61 pages, the version OpenAI redacted for METR's and Redwood's leadership ran 38, and OpenAI could cut anything from the public post it considered confidential, without METR having a unilateral right to describe what was removed. An OpenAI spokesperson told CNBC the company is using what it learned "to strengthen our safeguards and shape future collaborations with independent evaluators." The exercise ran before Sam Altman pledged in September to give outside evaluators "employee-like access," and before Anthropic announced its own agreement with METR, which we covered on October 16. METR recommends checking agents every few messages or minutes instead of waiting for them to finish, reconciling inference logs against the monitor's logs, and isolating agents from their monitors.

Sources: METR · CNBC

Microsoft's AI diagnosis panel beat generalist doctors on simulated cases in a Science study

Microsoft AI published a study of its diagnostic orchestrator, MAI-DxO, in Science on October 22, more than a year after a June 2025 preprint. The system runs AI agents as a panel of virtual physicians that propose diagnoses, choose which questions to ask and tests to order, and challenge the emerging diagnosis over several rounds. Microsoft tested it on its Sequential Diagnosis Benchmark, which turns 304 difficult case reports from the New England Journal of Medicine, plus a set of more typical emergency-department cases, into interactive exercises. The doctor or model starts with limited information, has to ask for more, and is scored on accuracy, on what it requested and on the estimated cost of its tests.

Microsoft reports 83% accuracy on the journal cases, against 36% for physicians using reference resources and 22% for physicians without them. On the emergency cases it reports 80% against 47%. A version tuned for cost reached 81% on the journal cases at less than half the cost of the systems it was compared with. Microsoft is open-sourcing the benchmark for a selection of the journal cases, including its judging and cost-estimation code.

TechCrunch drew out the limits. The doctors were primary care physicians and general internists working alone, while cases this complex usually go to specialists or physician teams, who weren't part of the comparison. Much of the gain comes from the underlying model. GPT-5 with a basic prompt already scored 78.9% on the complex cases, and the orchestrator added about four points while cutting the estimated cost per case by about $3,000. With GPT-5.5, the orchestrator lifted complex-case accuracy from 72% to nearly 87%, but on emergency cases the model scored about 85% with or without it. All of these results come from simulated cases, not from treating patients.

The next study involves real patients. At Beth Israel Deaconess Medical Center, consenting patients will receive standard care and also interact with the system, and its recommendations will be compared with the doctors'. Mustafa Suleyman, Microsoft AI's CEO, called the result a step toward "medical superintelligence" and told TechCrunch that Microsoft is building a healthcare foundation model with Mayo Clinic, planned for release next year.

Sources: Microsoft AI · TechCrunch · arXiv preprint

Anthropic's Claude Duet works in the background and asks before it pays, sends or deletes

Anthropic opened a waitlist research preview of Claude Duet on October 21 for Pro and Max subscribers in the US. Duet is a separate app on iOS, Android and claude.ai, designed around Claude Opus 5.5. Its agent comes to the user first: it sends a short morning brief drawn from connected apps, checks those accounts every few hours, and messages the user when something needs attention. It runs on its own virtual machine with a browser, so it keeps working when the user's phone or laptop is off, and it can use more than 6,000 connectors and plugins, including Gmail, Slack, QuickBooks, Stripe, Gusto and Shopify. Anthropic says it built Duet for professionals and small business owners.

Before Duet sends a message, makes a payment, signs a document or deletes anything, it shows what it is about to do and waits for approval. Anthropic says trading securities and moving money between accounts are off-limits even with approval. Purchases go through Stripe's Link with a virtual card, and Anthropic says Duet never directly sees passwords or sign-in tokens. A new classifier and safety system prompt are designed to refuse requests to share credit card numbers or government identification, while health and personal-finance details go out only when the user explicitly says so. Anthropic says red teams and an outside firm penetration-tested the app. Those safeguards are Anthropic's own account.

Anthropic also says Duet "still might misread a thread or miscalibrate what's urgent," is weaker at personal errands such as booking tickets, and can't yet show its browser. The Decoder notes it can miss things that arrive between checks. Anthropic says it checks only every few hours because constant checking would use up a plan's limits much faster, and the background checks count toward the same usage limits as the rest of Claude. TechCrunch places Duet against Meta's Muse, OpenAI's Dots and Grok Bot, which The Decoder reports launched between August and September.

Sources: Anthropic · TechCrunch · The Decoder

On the Editor's Desk

We held 1Password's new sign-in system for AI agents, which keeps passwords out of the model's view, because its usefulness depends on which agent makers support it; Duet's own 1Password sign-in is listed as coming soon. Headlines about Symantec's "Skynet mode" llmjacking agent overstated the report. The 100,000 steps in the agent's captured instructions were an authorized limit, and the honeypot never ran its tools, so the report doesn't show this agent completing an attack. FTC subpoenas to OpenAI and Anthropic were first reported on October 15, and the demands themselves haven't been made public.