If AI Speeds Up Research, What Exactly Is Getting Faster?

An AI assistant finishes the code for an experiment in half the usual time. That feels like a clear productivity win.

But the experiment still needs to run. Someone needs to check whether the result is real, decide what to try next, and turn the finding into something the team can use.

So how much faster did the research actually become?

On November 5, 2026, METR published an independent assessment of AI-assisted R&D at Anthropic. Its main benchmark analysis estimates that Anthropic's capabilities progress accelerated 1.9× after October 2025, with a reported 95% confidence interval of 1.5–2.8×.

That is a substantial finding. The question I find useful is what the measured improvement represents, and how we would recognize the same effect in an engineering team.

What exactly is getting faster?#

There are several different claims hiding inside “AI speeds up research.”

These are proposed engineering measurements, not four results from METR's assessment.

One can improve while another barely moves. Producing more experimental code might expose a GPU queue as the next bottleneck. Generating more results might overload the people who decide which results deserve follow-up.

This is why I would ask where the clock starts and stops before comparing two productivity numbers.

How does a benchmark become a measure of progress?#

The assessment uses AECI, Anthropic's adaptation of the Epoch Capabilities Index. Epoch's index combines results across benchmarks into a common capability scale. It helps compare models even when an individual test becomes too easy.

Epoch's methodology estimates model capability alongside benchmark difficulty and how sharply each benchmark separates models. A score increase therefore isn't interchangeable with the same percentage increase in accuracy on every task.

My mental picture is comparing students who took overlapping sets of exams. We want to account for differences in exam difficulty, rather than simply average every available percentage. That is an analogy for the measurement problem, not a claim that the model's abilities reduce to one perfectly complete number.

A steeper capability trend also doesn't mean each model runs faster. It concerns how quickly successive models improve on the chosen scale.

The public ECI and this internal adaptation aren't the same dataset. A public leaderboard alone cannot reproduce an analysis that includes private snapshots.

If the trend changed, did AI cause the change?#

That is a separate question from whether the trend changed at all.

METR judges that AI likely accounts for most of the current acceleration, with substantial uncertainty. This is an evidence-based assessment, not a randomized experiment that switches AI off for an otherwise identical Anthropic.

For a smaller team, the same attribution problem appears when several things change together. Suppose we adopt an assistant, clean up our datasets, and get access to more experiment capacity in the same month. A faster delivery rate doesn't identify the contribution from each change.

METR's earlier assessment proposal, dated October 14, explicitly separates capability growth, attribution to AI, and future bottlenecks. It calls for examining inputs such as compute and labour alongside results.

I find that separation more useful than collecting one impressive speedup multiplier. It forces us to ask what explanation the evidence can distinguish.

Why wouldn't halving coding time halve research time?#

Consider this invented, deliberately simple workflow. Its stages happen in sequence, with no overlap:

Coding became twice as fast. The complete cycle became 10% shorter. This is arithmetic, not an Anthropic measurement or a prediction for another team.

The gain could still matter. We might use the freed time to improve the next experiment or prepare several runs in parallel. But those are further changes to measure, rather than benefits already established by the coding-time result.

The reverse is possible too: a small improvement to a repeatedly blocking step can unlock much more useful work than its local timing suggests.

What happens when productivity is measured directly?#

There is a useful historical comparison, provided we keep its date attached.

In METR's July 10, 2025 randomized study, 16 experienced open-source developers worked on 246 real issues. Issues were randomly assigned to allow or disallow AI. Developers primarily used Cursor with Claude 3.5/3.7 Sonnet when assistance was allowed.

The AI-allowed work took 19% longer, even though participants believed it had helped. That result concerns those developers, repositories and early-2025 tools. It doesn't establish what current tools do for every programmer, or refute a later assessment of a different research environment.

The February 24, 2026 follow-up is just as important. METR warned that newer experimental data was unreliable for estimating the current effect: some developers didn't want to participate without AI, and concurrent-agent workflows complicated time measurement. Its raw results suggested improvement, but selection effects obscured the size.

That is a practical lesson for anyone running an internal pilot. The people willing to join a comparison may differ from those who decline. Recording time also becomes harder when one person supervises several tasks at once.

How independent is an assessment built on private evidence?#

METR reports substantial redactions of informative evidence, and it did not sample the non-public models itself. Its work provides outside scrutiny; public readers still cannot independently reconstruct the whole analysis.

For me, that changes how to use the report. It supports taking acceleration seriously while leaving room to dispute its magnitude and attribution. It doesn't justify treating a private model name or an interview estimate as a reproducible benchmark result.

The companion November 5 review of Anthropic's August risk report also distinguishes an early-warning threshold from evidence that a catastrophic threat has materialized. Its discussion concerns the earlier July coverage period; it isn't a blanket verdict on every later model.

I couldn't verify a public reproduction or original test video of the underlying internal runs. The chart above is original analysis, while the cover is METR's conceptual accounting diagram. Neither is footage of an autonomous research system building its successor.

What would I measure in my own team?#

I would start with a bounded outcome: a validated change that improves a fixed evaluation, rather than lines of generated code.

Then I would record implementation time, experiment wait and runtime, review effort, failed runs and accepted results. The evaluation would need consistent quality requirements and resource accounting. Otherwise, “more progress” might mean more spending or a looser acceptance standard.

METR's assessment makes this worth asking now. It doesn't remove the need for local evidence, and I haven't reproduced its analysis or run a productivity experiment for this article.

What interests me is the complete learning cycle: propose, implement, test, understand and decide. If AI accelerates that cycle, the benefit can extend well beyond code completion.

The measurement should tell us how much faster we reach a result we can trust.