Better Models, Better Tools, or a Positive Feedback Loop?

METR, an independent nonprofit that evaluates AI systems, spent eleven weeks, from August 12 to October 29, going through Anthropic's internal data on how the company builds its models. In an assessment published November 5, most of the METR staff who worked on it say Anthropic is likely improving its models on well-specified tasks at around twice the rate it managed between March 2024 and October 2025. The team wouldn't be surprised if the rate were closer to 1.3 or 3 times as fast. Based on internal evidence and researcher interviews, METR estimates that AI accounts for most of that acceleration, perhaps 80 percent, though it is much less certain about the cause than the faster trend itself.

Better models and better systems for using them can help researchers try more ideas and spend less time implementing each one. Some of that work may then improve the models themselves, giving researchers more capable tools for the next round. Anthropic’s own technical reports had suggested that one exceptional model, Mythos Preview, produced a one-time jump in progress. METR’s newer evidence offers a way to examine whether the change goes further. Are we watching AI become a better research tool, or watching a research process begin to compound—and what would distinguish the two?

METR says the newest internal models "make a permanent slope change look more plausible than a one-time jump at Mythos Preview." Its benchmark index, AECI, combines many test results into a single score. A steadier, faster pace is what a much better tool would produce, so on its own it doesn't separate the tool from the loop. "The rate of growth could be undergoing continuous acceleration, e.g. hyperbolic growth," it writes. "While we do not see clear evidence of this in the AECI trends, there is other evidence that makes this seem more plausible which Anthropic redacted." In its amended IPO filing Anthropic says its internal estimates and expectations for future models, "rather than our current models," indicate that it "may imminently meet" its own policy's definition of "dramatically accelerated progress attributable to AI." It hasn't designated any current model as exceeding that threshold.

A better tool would already be a big deal

In Anthropic interview notes that METR published, one researcher described experiments people didn't bother attempting because each idea took one or two days to implement, but now cost almost nothing to implement with Mythos Preview. Another credited "very promising directions" to colleagues being able to "spam 10 ideas" at an internal agent system "and see what sticks." When implementation costs days, researchers ration their long shots. When it costs almost nothing, they try them, and some of the long shots pay off.

The models are also getting better at the experimental work itself. Anthropic gives each new model the same code that trains a small AI model and asks it to make the code run faster while passing the same correctness checks. Claude rewrites the code, runs it, times it and tries again. Opus 4 averaged about a threefold speedup in May 2025, and Mythos Preview reached about 52-fold by April 2026. Anthropic warns that the size of the multiple depends on how much room the starting code leaves, and that it shouldn't be read as a real-world training speedup. Across these generations, Claude became better at improving code against the same fixed goal.

Even with the same version of Claude, a lab could keep getting more productive as researchers learn to use it. The researcher who described implementation as nearly free guessed that the lab was “just starting to fully switch over” and expected further gains as it did. Trying more ideas and reorganizing experiments around the available tools could sustain that improvement without a more capable successor. Rising productivity alone therefore wouldn’t establish that the research process is compounding.

Hiring and compute complicate the attribution too. METR says it received convincing evidence that the acceleration didn't come from faster growth in training compute, which still leaves the compute spent on running agents, experiments and search. One Anthropic staff member told interviewers that hiring was probably the biggest reason for faster progress. OpenAI, describing its own research acceleration, reports that the number of experiments per active experimenter reached its highest level in August, in a series it began tracking in January 2025. It also notes that its available compute had grown significantly.

A sustained increase in the pace of model improvement would bring capabilities within reach sooner than last year’s trend suggested. Even if the gains came entirely from better research tools and wider adoption, they would change how much time everyone outside the labs has to understand and respond to each advance.

Where the loop could close

The clearest small loop is in Anthropic’s software development infrastructure. As Claude-assisted coding took off, the company went from merging about 2,500 code changes a day in May to more than 30,000 in September, according to its account of rebuilding source control. The copies of its code repository outgrew their disk and memory limits, and the conventional way of queuing changes for merging stopped being practical. Claude helped build the replacement, a system called Scion, while a seven-person team made the design decisions and reviewed every change. Anthropic says Claude's coding ability "was both what put us in this situation in the first place and how we were able to quickly build a high-quality, scalable system to deal with it." A model relieving pressure that its own use created is a modest loop, and one that runs through infrastructure rather than through the next model.

A paper posted this month examines how improvements to a model’s working environment can become improvements to the model itself. That environment, called a harness, includes the prompts, tools and rules used to carry out tasks. The researchers built a collection of harness variants designed to address different failures. When a small model failed a training task, they gave it further attempts under selected variants, then used successful attempts to train the model. To test whether it had learned something it could use without that extra support, they evaluated it under its original harness. On AppWorld, a benchmark involving multi-step tasks across apps, a nine-billion-parameter model’s average success per attempt rose from about 40 percent before training to 53 percent with standard training and 62 percent with the harness-assisted method. Some of the benefit survived after the extra help was removed.

As the model learned, the usefulness of that help changed too. A harness variant that helped an earlier version could hinder a later one, and the best variants changed between training checkpoints. Improving the model therefore changed which prompts, tools and rules worked best around it. The researchers kept the same pool of harness variants throughout training; they weren’t repeatedly building new harnesses for successive generations of models. These are author-reported results from small open models, rather than evidence of faster frontier development. They nevertheless show how the two sides can interact: better support can help train a better model, while the better model changes what support it needs.

Whether gains like these survive at larger scale is still an open question. Anthropic's automated weak-to-strong researcher set agents on a safety problem: how a weaker model can supervise a stronger one. In the test setup, the agents recovered 97 percent of the gap between the weak supervisor's performance and that of a strong model trained on correct answers. The best of four existing methods, which two of the authors spent seven days tuning, recovered 23 percent. The agents could check their test scores as often as they liked, so the test worked more like a practice set they could tune against than a sealed exam. When the researchers moved one of the strongest ideas into production-scale training on Sonnet 4.0, it gained half a point on a held-out evaluation of chat helpfulness, within the noise. They suspect the problem was how they drew out the training signal, and they haven't tried richer approaches yet.

METR identifies a broader concern: "acceleration from the use of AI could start to compound, as the development of more capable models leads to further acceleration." It believes AI's contribution has grown relatively more important over time, after a late-2025 speedup that non-AI factors likely helped drive. It also found no clear bottlenecks or steep diminishing returns standing in the way over the coming year, though its view of those bottlenecks changed several times and it isn't confident in the current read.

Recursive self-improvement needn't involve an AI rewriting itself while people watch. In these examples, people set the problem and judged the results, but their involvement doesn't prevent improvements from feeding into the next model. If the loop is starting, from the outside it looks like an ordinary productivity story.

What would tell them apart

To establish compounding, we’d need to follow an improvement through successive models. An earlier model helps develop a better way to train its successor, and that successor becomes better at finding further improvements to training. The harness paper shows that additional support during training can produce gains the model retains when that support is removed. It doesn’t show whether the improved model then becomes better at improving its own successor. Following that process across generations would let us ask whether each round produces more progress for the same research effort, or brings the next useful model sooner.

METR fit a one-time jump and a lasting slope change to the earlier benchmark record, then checked which better predicted the newest internal models' results. The lasting change had about half the prediction error. On AECI, Anthropic was gaining about 15 points a year before October 2025 and about 29 a year since. That supports a faster trend, but it can't show whether each successor arrived sooner or owed more to its predecessor. The newest models, which METR calls Models 2, 3 and 4, finished training between July and October 2026, and finish dates aren't development times. They don't say when work began or how much it overlapped.

METR also sees a model's results only after training ends, leaving its view roughly one training run behind the work. Finding sufficiently difficult benchmarks is becoming a problem too. Model 4 scored above 50 percent on all 15 benchmarks used to rate it, while Opus 4.6, nine months earlier, still fell below that line on many of them. METR says that leaves it "somewhat nervous about our ability to measure AECI now and in the future."

What the redactions imply

Anthropic redacted a majority of the evidence METR found most informative, and some of it bears directly on this question. METR says the withheld material gave it "some indication that acceleration could be above 2x and increasing over time," though it "would be surprised if it was significantly higher." Anthropic described the redactions as a whole as "aimed at removing confidential information that could accelerate other AI developers," and METR adds that most of the redacted information was very high-level. That rationale doesn't tell us what the acceleration evidence shows. It does say Anthropic believes some of what it has learned about its own research would help competitors move faster.

That cuts two ways. A lab can use its strongest internal model before it is publicly available, and METR dates Anthropic's unreleased models by when most employees could use them. If the loop is strong, that head start could widen a lead between public releases, the gap we described in The Permission Curve. If the methods transfer, other labs speed up too, which could narrow any one lab's lead while raising the pace of the whole field. The harness paper shows that some of these methods are already being developed in public. Either way, plans and forecasts built on last year's pace become less reliable.

When the next report comes out, the benchmark score alone won’t tell us whether the model before it helped cover more of the distance than its predecessor did.