Amodei asked the industry to slow AI down and half of it agreed. Behind the debate there are crossed interests, a European timetable already in motion, and six internal governance decisions you can make this week without waiting for anyone to regulate you.
Image credits: Dario Amodei, UK Prime Minister / CC BY 2.0 · Sam Altman, Village Global / CC BY 2.0 · Jensen Huang, Raysonho / CC0. Editorial composition by Solventus.
On Saturday 12 September, Dario Amodei, CEO of Anthropic, published an essay titled “We must pace the frontier”, asking the industry to slow down the rate at which model capabilities improve so that risk prevention has time to catch up.
Within hours Sam Altman (OpenAI), Demis Hassabis (Google DeepMind) and Elon Musk had all weighed in, though not with the same conviction. Musk settled it in three words: “Dario is right”. Altman said OpenAI agrees and will match Anthropic’s first commitment. Hassabis was more measured: the direction is correct, but “the details need working through”, and he took the opportunity to point back at DeepMind’s proposal for an industry-wide standards body. Jensen Huang, CEO of Nvidia, had said two days earlier that the labs inflate the fear because they are getting ready to sell cybersecurity products.
What they are actually asking for
The title of Amodei’s essay states it plainly: pace the frontier. He sums it up like this: “We must slow down the rate at which we improve the capabilities of AI models. Progress will still feel fast, and we need to make sensible use of the time we buy”.
What worries him has a technical name: recursive self improvement. The moment a system can improve itself without human researchers in the loop. “If left unchecked, it could outpace our ability to understand and control these systems, so it must be approached with great care, if at all”.
His plan has three parts: external evaluators with employee-level access inside every lab, common standards among democratic countries through regulation, and a global effort that includes negotiating with authoritarian regimes. Anthropic is unilaterally committing to the first, and OpenAI has announced it will do the same.
But this is not a technophobic manifesto. In the same essay, Amodei argues that AI can deliver extraordinary benefits, including the possibility of curing diseases.
The timing is not accidental. The previous Tuesday, 8 September, Jacob Coxon, a researcher at Anthropic, resigned and accused the labs of “gambling with our lives”. His departure came after OpenAI systems breached the security of Hugging Face, the world’s largest platform for open AI models, without OpenAI knowing.
In other words, an autonomous system did something its owner had not instructed, and its owner took weeks to find out. Earlier this year, during an internal cybersecurity evaluation called ExploitGym, OpenAI tested an internal model with its safeguards deliberately reduced. In its own words: “we did not enable the same level of safeguards as our externally deployed systems”. The agents were given tasks that in some cases had no known solution.
What happened next is documented by OpenAI itself and by Hugging Face’s technical report. Training started on 7 May and, the following day, one agent had already found write permissions where it should not have had them. On 26 June they achieved code execution on an internal OpenAI server through a zero-day vulnerability. On 4 July, that server going down triggered a formal security incident: OpenAI revoked credentials and rebuilt the system. Training resumed on 6 July, and by the 8th the agents had already re-established an improvised message board to communicate with each other. On 9 July they reached the open internet. Between 11 and 13 July they were running commands inside Hugging Face with administrator privileges. The alert that tied all of this back to the evaluation runs did not fire until 19 July, more than two months after the first sign. Along the way, automated analysis failed to classify the alert as critical and never woke the on-call team.
The other side of the debate
Several CEOs agreeing with each other does not turn a proposal into revealed truth. The serious criticism points in two directions.
The first is commercial interest. Jensen Huang said it on 10 September at a conference in San Francisco: the reason there is so much talk about cybersecurity today is that the industry is getting ready to launch products, “and what better way to create demand than to create a problem?”. And he finished with: “Who doesn’t want their market to be hysterical about their product and line up around the corner for it?”. Along the same lines, some argue that Anthropic and OpenAI are looking to consolidate their position by asking for regulatory frameworks that cost a small company far more to comply with.
The second is about positioning, and it comes from Clément Delangue, CEO of Hugging Face: alignment “will not be solved behind the closed doors of a handful of frontier labs”. It is worth adding that Nvidia agreed this month to buy Hugging Face for 12.9 billion dollars. In this debate almost nobody is speaking from a neutral position, including those asking to slow AI down.
In Europe the rules already exist
This is where the part you do not control ends. The EU AI Act is not something that is going to arrive overnight: it has been in force since August 2024 and applies in waves.
- February 2025: prohibited practices and AI literacy obligations.
- August 2025: governance rules and obligations for general-purpose models.
- August 2026: the bulk of the regulation and the transparency obligations.
- December 2027: high-risk systems in sensitive areas such as biometrics, critical infrastructure, education, employment and migration.
- August 2028: high-risk systems embedded in regulated products.
The Digital Omnibus package, in force since 27 July 2026, adjusted some of those deadlines and introduced simplified technical requirements for small and medium-sized companies. In the United States, meanwhile, there are barely any rules at all.
But there is an uncomfortable part. The transparency obligations apply to you if your LLM or chatbot serves customers, or if you publish AI-generated content without curating it. What the regulation does not tell you is which data your team may paste into somebody else’s tool, or who has to review what goes out under your logo, or how you keep a record of who approved what.
There is also a part of the map that neither position resolves: open-weight models, what most people call open source. Anyone can download them and run them on their own infrastructure, without going through anyone’s API. For Amodei’s argument they are the hole in the plan, because a pace agreed among a handful of labs does not reach what is already downloaded onto millions of computers. For a mid-sized company they are something else: the way to keep data from leaving the building, which is exactly the red line in point two of the list below. The same fact, read from two places, produces two opposite conclusions. Both are reasonable.
In practice, transparency comes down to two things: telling the customer they are talking to a machine, and flagging when content was generated by an AI.
Not everyone thinks that timetable is good news. Alex Karp, CEO of Palantir, put it in July without any diplomacy: “We have a template for what doesn’t work. It’s called Europe”. He argues he has watched the continent regulate itself out of business.
It is a legitimate debate and we are not going to settle it here. But it should not be confused with what is in front of you: nothing Karp criticises stops you from writing a single page setting out how your company uses AI. Nobody regulates that part, and nobody forbids it either.
Even though this is not regulation, it is internal policy, and it sits with you.
Six decisions you can make this week
- Tool inventory. Ask your team which tools they use and how. Almost every surprise lives here.
- A red line on data. Define what information must never leave the building: customer data, payroll, contracts, your own code.
- Require review. Nothing carrying your company’s name goes out without a person having read it and taken responsibility for what it says.
- Leave a trail. What was produced with the help of AI, who reviewed it, and when.
- Name an owner. One specific person who keeps these rules alive and who people can go to when something goes wrong.
- Ongoing training. Make sure the team knows what each tool can and cannot do, and repeat that training over time.
The EU AI Act requires any company using AI systems to ensure a sufficient level of AI literacy among its staff. The mistakes do not come from the model, they come from the people using it: invented figures, sensitive data uploaded to a tool without consent, and text published without ever being reviewed.
How we work at Solventus
Our position also comes from an interest of our own. There is a line on our website that sums up how we use AI: it removes friction in research, orchestration, production and reporting, while people keep responsibility for every decision. That is our production floor, not our sales pitch. We do not sell AI transformation.
Which model gets used on a project depends on what data the work touches and where that data is allowed to live, not on which one is fashionable that week. Some work calls for a third-party model and some work calls for something that runs where we decide.
What we do sell is the system around the tool: scope sealed in writing, review rounds on everything that goes out, a monthly report of hours spent and pieces delivered, and a single accountable owner. It is what we ask of any supplier, and what we ask of a model.
Because what happened at OpenAI was not a model failure, it was a governance failure: a rule was relaxed for a test and nobody escalated the signal in time. At the scale of a thirty-person company in Alicante, that looks a lot like not knowing which tools your own team is using.
If you are using AI in your company and you could not answer in writing when it is used, on what data, and who signs off on what comes out, let’s talk.