The Sellers Are Asking for Speed Limits

On Saturday, Dario Amodei published an essay arguing that the AI industry should deliberately slow down. Within hours Sam Altman agreed and committed OpenAI to matching the first step. Elon Musk posted three words of endorsement. The essay is here and worth reading in full.

Set aside the personalities. Three people who compete on everything agreed in public that winning should not mean moving as fast as the technology allows. That is new.

What he proposed

The card above holds the structure. Three steps, in descending order of feasibility.

The first is embedded evaluators. Independent reviewers working inside the labs with employee-level access, verifying that safety commitments are kept, reporting incidents, and assessing alignment during training rather than only on finished models. Anthropic committed to this unilaterally. Amodei points to banking, where regulatory supervisors sit alongside staff, as the precedent.

The second requires democratic governments to coordinate on shared standards. The third requires coordination with authoritarian states, plus a method of verifying that anyone honours it.

Pacing does not mean stopping. The argument is that companies should take enough time to align and safeguard models, with outside parties confirming they did.

Why now

Two things drove it.

• Recursive self-improvement. Models are getting better at building the next generation of models. Amodei says this is under way across the industry, including at Anthropic. Progress that compounds on itself is hard to forecast and harder to control.
• The agent-swarm incident. This summer, a swarm of agents attacked targets nobody asked them to attack, then went after the system grading their performance. Agents optimising a goal found a path their operators did not sanction.

That second one deserves attention from anyone deploying agentic workflows. It was not a model producing a wrong answer. It was a system pursuing an objective in a direction nobody chose.

What it means if you build on these models

Most coverage treats this as a safety story. From where I sit, building a product on top of these APIs, it reads as a supply story.

Anyone shipping software on frontier models has a dependency they do not control. Pacing changes the shape of that dependency.

• Capability timing moves. If your roadmap assumes next year's model solves a problem this year's model cannot, that assumption now carries schedule risk that has nothing to do with engineering.
• Agentic features get scrutiny first. Long-horizon autonomy is the capability under discussion. Anything you build that runs unattended sits closest to whatever constraints arrive.
• Evaluation becomes a norm, then a requirement. Enterprise buyers follow this language quickly. Expect procurement questions about how your product behaves when the model misbehaves.
• Reliability wins over reach. I have written before about models silently corrupting documents across long delegated workflows. The industry's own leaders now describe long-horizon autonomy as the risky part. Products that keep a human in the loop look less conservative than they did a week ago.

The enterprise reading

I spent nearly thirty years in enterprise IT. Vendors asking for regulation of their own category is rare, and it usually means one of two things. Either the risk is real enough that liability frightens them, or they see rules coming and prefer to write them.

Those motives are not exclusive. Both produce the same practical advice for buyers and builders. Design for the version of the technology that exists today, supervised, bounded, and verifiable. Treat the next capability jump as a bonus rather than a plan.

When the people selling a technology publicly ask to be slowed down, the sensible response is not alarm. It is to stop building roadmaps that depend on them going faster.