Anthropic wants AI development to slow down globally, warns humans could lose control otherwise

Leading artificial intelligence (AI) company Anthropic, which is within kissing distance of a trillion-dollar valuation thanks to its AI systems, which have moved markets and changed the nature of white-collar jobs, now wants the technology to slow down.

In a long article on its website, titled "When AI builds itself", Anthropic said it would be good to have the option to slow or temporarily pause frontier AI development, citing safety risks and loss of control over the models as they self-improve.

The company claims that slowing down would be effective if done verifiably and globally.


Anthropic plans to organise conversations involving policymakers, researchers, civil society and other AI companies to address these questions.

The timing of this discussion seems strange, as the AI startup filed its draft papers to go public just last week. It had raised $65 billion in pre-IPO funding, at a valuation of about $965 billion.

Why does Anthropic want a slowdown?

The Claude maker said it is delegating a lot of its AI development to AI systems themselves, speeding up their work. Extrapolating from this trend, the company claims this could mean an AI capable of fully autonomously designing and developing its own successor, a process it termed "recursive self-improvement".

Anthropic cited multiple reasons why this is bad news:

The main fear is the loss of human control over these systems. If systems build their own successors, the way they are secured and monitored could go out of human hands.

Anthropic claims recursive self-improvement is not inevitable, but it is likely to come sooner than most organisations and institutions are prepared for.

The issues present in today's AI models, such as hallucinations, could compound if the models start building themselves, Anthropic warns. Each new generation could inherit and worsen flaws, making it harder for humans to trace their source and fix them, the company said.

What's the evidence?

This worry isn't without reason. Anthropic noted that as of May 2026, more than 80% of the code merged into its codebase was authored by Claude. On most open-ended coding tasks, Claude's success rate has increased by 50 percentage points in just six months, reaching 76% in May 2026.

When asked to pick a better next step in a research session than a human had chosen, the best model (Mythos Preview) did so 64% of the time in April, up from 51% in November 2025.

Some gaps remain, though, when Claude exercises judgment in choosing goals, both in engineering and research.

What's the possible future?

The first scenario, deemed unlikely by Anthropic, is that the trend stalls.

The second, most likely in the near term, is that AI labs will continue improving efficiency, with humans still controlling research directions.

The third picture shows AI systems achieving full recursive self-improvement and building their successors at a fast pace. In this case, humans would move to oversight and verification.