Independent Review Finds Anthropic AI Progress May Be Twice as Fast

An independent assessment of Anthropic’s research operations suggests that the company’s AI models are improving on well-defined tasks at roughly twice their earlier pace. The conclusion is striking, but the researchers behind it are equally clear about its limits. Their estimate is uncertain, and Anthropic redacted much of the evidence that informed it.

The non-profit Model Evaluation and Threat Research group, known as METR, published its assessment on 5 November after spending 11 weeks examining internal data, interviewing staff and reviewing how Anthropic uses AI in its own research and development. The work is separate from the broader employee-like access arrangement Anthropic announced in October.

METR’s central judgment is that, as of October 2026, Anthropic was likely improving model capabilities on well-specified tasks at about twice the rate it did between March 2024 and October 2025. The researchers say a true rate closer to 1.3 times or three times the historical pace would not surprise them. This is an estimate of the speed of progress, not a statement that today’s models are twice as capable as last year’s.

What the researchers actually measured

One of the clearest public data points is a measure Anthropic uses for model performance across tasks with verifiable answers. METR found that scores for internal models on this index had grown at an estimated 1.9 times the earlier rate since October 2025. Its 95% confidence interval ran from 1.5 to 2.8 times. In practical terms, the group says the last year may have delivered about two years’ worth of progress at the older pace.

That result does not automatically translate into the same acceleration for every real-world job. Benchmarks often favour tasks that can be specified and graded neatly. Open-ended research, organisational decisions and messy practical work can be harder to measure. METR itself says the relationship between the index and wider impact remains uncertain.

The group also examined whether simply scaling training compute could explain the change. It says it is confident the acceleration was not caused by a faster increase in training compute. Interviews and internal surveys were consistent with AI tools helping researchers work faster, though those accounts cannot establish a precise causal share. METR’s judgment is that AI now accounts for a majority of the acceleration, while acknowledging that other factors also contributed.

Anthropic gave METR access to internal benchmarks, training data about the research process and interviews with staff. At the same time, the company retained the right to redact non-public material from the published report. METR says Anthropic removed most of the evidence it found most informative for its high-level risk conclusions.

The redactions do not make the assessment worthless. They do, however, limit independent scrutiny. Readers can examine the published method and some headline measurements, but cannot independently weigh every piece of evidence behind the most consequential judgments. METR says the public must rely on its interpretation more than it would ideally like. Anthropic says the redactions were intended to avoid revealing information that could help rival developers accelerate their own work.

There is also an important distinction between AI-assisted research and an AI system running a research laboratory on its own. METR says it has not seen evidence that AI can yet make highly significant, fully autonomous contributions to the most important parts of Anthropic’s model-training process. It reports signs that such contributions could become more important, but that is a possibility to watch, not an achieved milestone.

The implications reach beyond Anthropic. If more capable AI makes the people building the next generation of AI more productive, the development cycle could shorten. That would put more pressure on testing, outside review and governance to keep pace. METR did not identify a convincing near-term bottleneck in compute, data or algorithmic progress, but stressed that its understanding of those limits changed as it received more information.

For now, the careful reading is neither that runaway self-improvement has arrived nor that nothing unusual is happening. An outside group with unusually broad access has found evidence of faster progress at one frontier lab. Its report also shows why the quality of public evidence, and the freedom of independent evaluators to discuss it, will matter as much as the headline number. TechBooky has previously covered Claude’s role in Anthropic’s own research; METR’s assessment adds an external, more qualified test of how much that may be changing the pace.