METR: Anthropic is improving at about 2x its old pace, mostly because of AI

METR published an independent assessment on November 5 concluding that Anthropic is likely improving its models on well-specified tasks at around twice its historical rate, and that AI itself probably accounts for most of that speedup. In a companion review, METR said Anthropic’s public argument that it had not crossed its own 2x acceleration threshold was “overstated.”

Three days earlier, Anthropic’s amended IPO filing had said its internal estimates “indicate that we may imminently meet” the Responsible Scaling Policy’s definition of “dramatically accelerated progress attributable to AI.” Together, these are the most concrete public evidence so far on whether AI is speeding up its own development inside a frontier lab. They are also a lesson in how little of that evidence outsiders can see.

What the S-1/A changed

The November 2 S-1/A added language that the October 26 version did not have. It says Anthropic is monitoring risks “as we approach what has been termed ‘recursive self-improvement,’” and that using AI to build more capable AI “could also pose a considerable safety risk.” It also concedes that the company has “in the past not identified” significant acceleration until after it occurred.

Anthropic says it has not designated any current model as exceeding the threshold, does not expect to safely let Claude build successive frontier generations “without meaningful human supervision or understanding,” and is weighing policy updates to “keep humans in control of frontier model development.” The threshold, in the policy Gizmodo describes, is a doubling of the baseline rate of improvement plausibly attributable to AI.

There is a fair skeptical reading here. TechCrunch notes that an S-1 serves as both disclosure and marketing, and Gizmodo’s Mike Pearl suggests the language may work as a hype signal as well as a warning to investors. Anthropic’s IPO gives it reason to both disclose risk and sound close to something extraordinary. METR’s report is valuable because it is the closest thing to an outside check on that claim.

What METR measured

METR worked from August 12 to October 29, with 2.5 full-time staff, on a project scoped for four weeks that ran to eleven. It had extensive internal data and interviewed about 15 staff, working on Anthropic devices. It had no query access to unreleased models.

The core measure is AECI, Anthropic’s internal version of the Epoch Capabilities Index, which folds benchmark scores into a single latent capability score. METR found a trend break on October 7, 2025, with growth rising from about 15 AECI points a year to about 29. That is 1.9x, with a 95% confidence interval of 1.5x to 2.8x, or roughly two years of historical progress in one.

Estimate	Acceleration
METR main estimate (Anthropic models, including internal-only)	1.9x (1.5x to 2.8x)
Claude Opus 5.5 system card	about 1.53x
Haiku 5.5 system card	1.7x
All frontier models, including OpenAI, Google and xAI public models	1.4x
Full robustness range across link functions	1.1x to 2.9x
Staff self-reported uplift, Slack polls, October 2026	about 12x

Three of those rows are worth unpacking.

The model choice is the biggest swing. The range from 1.1x to 2.9x comes almost entirely from the link function, the mathematical choice of how to map benchmark scores to latent capability. Changing snapshots, benchmarks or regularization keeps results between 1.9x and 2.2x. So the estimate is stable to what you measure but sensitive to how you model it.

The 1.4x row is an open dispute. Include rivals’ models and the acceleration shrinks to 1.4x, going from +17 to +24 points a year. METR offers two opposing readings. If Anthropic’s models were subfrontier in 2025, its historical trend was understated and 1.4x is more accurate. If earlier non-Anthropic models were more overfit to benchmarks, 1.4x is too low and 1.9x stands. METR keeps 1.9x as its headline without resolving this. It does mean a large part of the headline may be Anthropic catching up, not the frontier speeding up.

The 12x figure is not a measurement. Staff polls rose from about 1.2x for Claude 3.7 Sonnet in January 2025 to about 12x for an internal model called Model 4, which finished training in September or October 2026. METR calls this “very loose evidence.” The distance between 12x felt and 2x measured is itself informative: perceived productivity and overall capability growth are different quantities.

There are measurement limits too. Model 4 scores above 50% on all 15 benchmarks used, raising saturation concerns, and AECI covers well-specified, automatically graded tasks, so it may not reflect research taste or real-world impact.

Is it AI, and is the threshold crossed?

METR is confident the acceleration was not caused by faster growth in training compute. Its subjective estimate is that about 80% of the October 2026 acceleration is due to AI, with substantial uncertainty, while the late-2025 break was likely influenced by non-AI factors. AI is not yet making highly significant, fully autonomous contributions to the most important training work, but METR reports “signs of life” that such contributions could come soon.

The sharper judgment is in the companion review of Anthropic’s August Risk Report, which rated risk from automated R&D as low and said acceleration was below a factor of 2. METR agrees catastrophic risk from the July 15 models was low and finds the argument against the separate Full Substitution Threshold broadly compelling. On the 2x threshold it disagrees with how the case was made:

• It considers crossing by July 15 plausible, though most researchers at METR and Anthropic lean toward it not having happened.
• The public report said “less than a factor of 2.” METR’s calculation from a redacted private figure gives 1.97, and it suggests “around 2” as the honest phrasing.
• It says the public text understates leading indicators, which it would describe as rates “between 1.3 and 2.8.”
• It says the public report leans on weak evidence for a one-time jump in AECI, when continued acceleration is the more parsimonious reading.
• The public report omitted a private note that Model 2 was still training and later snapshots were expected to improve.

So the same lab that told investors it has missed accelerations before also published a public risk report that its reviewer says rounded in a reassuring direction. That is an uncomfortable pairing, even though METR says it is “extremely unlikely” Anthropic actively misled it.

How far to trust a report you cannot check

METR is unusually candid about this. Anthropic held unilateral redaction rights and described them as “aimed at removing confidential information that could accelerate other AI developers.” METR says a majority of the evidence it found most informative was removed, including intermediate capability measures, algorithmic progress estimates, R&D inputs, bottleneck forecasts, breakdowns of AI’s contribution by pipeline stage, and forecasts for future models. A component verifying compute allocation between capabilities and safety was canceled midway by Anthropic. METR’s public text was approved by Anthropic, and it concedes that “the public has to rely on METR’s judgment much more than would be ideal.”

Other disclosures: METR received free tokens and laptops but no payment, one team member had job discussions with Anthropic and held competitor equity (their work was checked by an unconflicted colleague), and most of the analysis was done with Claude’s help. The last point is a small version of the larger problem. The tools used to measure AI-driven R&D are themselves the AI under assessment.

Even so, this is substantial access, and METR used it to publish a critique of its host. Whether that is a template or a ceiling depends on who follows. We are not aware of a comparable public assessment for OpenAI, Google or xAI, which leaves the 1.4x-versus-1.9x question harder to settle than it should be.

What to watch

METR’s forward-looking language is the part to take most seriously. It found no convincing bottlenecks in algorithms, compute or data over the next 12 months, though with low confidence, and wrote that “the lack of observed bottlenecks or strong diminishing returns means there is a significant risk” of faster acceleration. In the review, it said it “cannot rule out a substantial further acceleration” over the next six months, which “could rapidly lead to ‘high’ risks.”

Three concrete signals follow. First, whether Anthropic formally designates a model as past the threshold, and what the safeguards it has promised actually look like; the policy language points to “unprecedented levels of security” and internal visibility it calls “eyes on everything,” and Pearl’s point that this work needs to start now is hard to argue with. Second, whether the policy update it says it is considering tightens or loosens the definition, since changing the yardstick just before crossing it would be a telling move. Third, whether the separate independent assessment Anthropic announced on October 15, which METR says is distinct from this project, gets more access to models and fewer redactions.