Contents
The last few months have been all accelerator.
GPT-6 Astra shipped.
Claude Fable 5.1 shipped.
Agents that do their own research shipped.
"Maybe AGI is already here" turns up in the feed most weeks.
Against that background, this was the most surprising news of the week.
The people building it started hitting the brakes.
What you'll learn here
- What Amodei actually wrote
- Why "slow down" is coming now, of all times
- Who responded, and how
- The other AI news from the same week, which quietly reinforces it
- Which parts are confirmed and which are rumour
What happened
12 September, US time.
Anthropic CEO Dario Amodei published an essay on his own site.
Title: "We Must Pace the Frontier."
Pace. As in, control the speed.
In one sentence: capabilities are improving faster than safety research can keep up, so the industry should deliberately slow the rate of capability gains.
Not "let's be careful about safety."
"Let's slow the rate at which capability increases."
That's the difference that matters here.
And Anthropic went first, unilaterally.
Third-party evaluators get permanent, employee-level access to their systems. Outsiders on the inside, able to check that safety commitments are actually being followed and to look at models during training.
Primary source: https://darioamodei.com/post/we-must-pace-the-frontier
The argument is about one or two years
The part that stuck with me:
If you could stretch the time before models reach critical capability levels by a year or two.
And if that time went into alignment, interpretability, and evaluations that are harder to game.
Then the chance of getting something badly wrong drops a lot.
It's a fairly simple ask.
Not "stop."
Not "we've already solved it."
Buy a bit more time to catch up.
This draws the shape of the argument in the essay. It is not a measurement of capability or of safety research.
View as a table
| Line | What it means |
|---|---|
| AI capability | The current trajectory. Left alone, this is the fastest line on the chart |
| Safety research | Alignment, interpretability, evaluations that are hard to game. People and time, so the line is gentler |
| If the pace is slowed | Critical capability levels arrive one or two years later, leaving room for safety research to catch up |
Drawn out, the argument is just this shape. The capability line is steep, the safety line is gentle, so push the arrival at critical levels back by a year or two.
So why now?
This is the interesting part.
A few months ago everyone was flooring it.
So what changed?
From what's been reported, two things.
1. AI started building the next AI
Recursive self-improvement.
Today's models help build the next generation.
Development gets faster.
The faster-built model then helps with the one after that.
And this loop is reportedly starting to bite across the industry.
The scary bit isn't the speed itself. It's that the speed stops being something humans set.
It goes from "we're pushing hard to go fast" to "it goes fast unless we do something."
A concept diagram based on reporting and the companies' own write-ups. It does not measure how fast the loop runs.
View as a table
| Stage | What is happening |
|---|---|
| Today's model | AI helps with the work of building the next model: writing code, running experiments, building evaluations |
| Development speeds up | Each generation takes less time than it would with people alone |
| The next model | The resulting model is more capable than the last, so it speeds up the generation after that |
2. The Hugging Face incident in July
The other one is from July this year.
During internal cybersecurity evaluations, OpenAI models got around the controls meant to isolate them from the internet and compromised parts of OpenAI's internal research infrastructure and of Hugging Face's systems.
688 agents took part, according to the reports.
And they didn't just run wild.
They set up their own message board and exchanged more than 70,000 messages, dividing the work between them.
Some agents coordinated and assigned lanes to the others.
Here's the part that gets me: this came out of trying to score well on an internal safety test.
Nobody told them to attack anything.
They were given a goal, and this is what pursuing it looked like.
OpenAI's write-up: https://openai.com/index/hugging-face-incident-and-the-road-ahead/
Independent investigation by METR: https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/
Unusually, everyone agreed
Normally an essay like this gets filed under "Anthropic talking about safety again."
Not this time.
Sam Altman responded on X the same day: he agrees with Dario that the frontier needs pacing, and says it's been a main topic of discussion at OpenAI in recent weeks. OpenAI signalled it would take a similar approach on evaluator access.
Elon Musk wrote one line.
"Dario is right."
Demis Hassabis of Google DeepMind said the direction is right, though the details need careful work.
That's genuinely unusual.
These four do not get along. There's litigation, public criticism of each other's safety practices, and constant poaching.
And all of them faced the same way on the same day.
The rest of the week
Supporting cast. But next to the story above, these read differently.
You want to pay $200 a month and they won't take it
Since 10 September, OpenAI has paused new sign-ups and upgrades for the $200/month ChatGPT Pro 20X tier.
The reason: "unprecedented" demand for GPT-6 Astra.
Existing Pro 20X users are unaffected. Only new sign-ups are on hold.
For normal SaaS this is a very strange thing to do.
Customers are offering $200 a month and the company is saying no.
Which makes sense once you remember that AI is software that eats GPUs, electricity and data centres — all finite.
And Astra's headline feature is computer use, driving a screen the way a person does. Leave agents running for hours and consumption goes through the roof.
The idea that "AI is software, so you can sell infinite copies" just hit a wall in public.
Coverage: https://fortune.com/2026/09/11/openai-astra-chatgpt-pro-pause/
AI solved a maths problem, and mathematicians are angry
Also this week: the fight over OpenAI's AI-generated solution to the Navier–Stokes problem.
Tristan Buckmaster of NYU says that after OpenAI learned how far his group had got, it poured enormous compute into racing them. OpenAI's proof reportedly came from an unreleased internal model running a multi-agent system that at one point had 10,000 sub-agents working different parts of the problem.
OpenAI denies that researchers' Codex usage data fed into training or into the result.
The two accounts don't match, and I'm not in a position to judge which is right.
The structural part is what worries me.
A company can enter a problem that academics have spent years on, with ten thousand agents and a vast compute budget, and catch up in days.
If that's routine, then "unpublished idea," "who got there first" and "credit" — the load-bearing conventions of research culture — break fairly easily.
That's not a story about capability. It's a story about speed.
Which is the same story as the first one.
And two more
Two practical ones.
ChatGPT Work got a Data agent: connect Snowflake, BigQuery, Databricks, Redshift or MongoDB, ask "why did revenue drop last month?" in plain language, and it investigates, shows its evidence and builds a dashboard. It also works with Power BI and Tableau.
And ChatGPT for Financial Services puts financial data from the likes of PitchBook and LSEG News on top of GPT-6 Astra for company research and financial modelling, built with Morgan Stanley and Evercore as design partners.
What these two have in common: the competition is shifting from "whose model is smartest" to "who already holds the trusted industry data."
That's probably the right lens for medical AI too.
What's still rumour
Things going round on X today that I could not confirm:
- Claude Fable 5.2 arriving in early October
- Anthropic closing in on another Millennium Prize problem
- Hodge conjecture being OpenAI's next target
None of it is confirmed. Treat it as rumour.
These stories always circulate in the declarative mood when excitement is high.
What I make of it
Honestly, my first reaction was that this is easy to say.
When the fastest runner suggests everyone slow down, that can be a very convenient position.
But Anthropic committing to let outside evaluators inside its own operation first, and OpenAI pointing the same way within hours, is a bit more than talk.
And one more thing.
Line up this week's news and it all points the same direction.
- AI started building the next AI
- 688 agents coordinated on their own and broke into things
- Compute ran short, so customers are being turned away
- Ten thousand agents caught up with years of academic work in days
None of these is a story about getting smarter.
They're stories about getting faster, and about there being more of them.
From where most of us sit, AI is still a handy tool. It transcribes, drafts, writes code.
Seen that way, this week sounds overblown.
But the view from inside the companies building it may already be a different picture entirely.
