Big Tech agrees to slow AI down. Watch who is holding the pen.

A lesson that we all need to learn is don't watch the mouth watch the hands.

Four men who compete for the same engineers & the same customers spent the middle of September agreeing with each other in public. Dario Amodei published an essay urging the industry to slow frontier development, something he calls "pacing the frontier". Sam Altman posted on X that he agreed. Elon Musk agreed. Demis Hassabis backed it too. By 15 September, The Verge was asking whether this is a safety pact or a cartel.

Rivals do not often harmonise and we should be suspicious when they do because when they do, the agreement usually makes something possible that competition did not.

It is worth saying what the labs themselves appear to think the agreement is. WIRED reported, around 14 September, that OpenAI had approached members of Congress to determine whether coordinating an industry-wide slowdown on frontier AI models would trigger antitrust violations. That is not an outsider's framing of a cartel. It is the labs themselves going to ask whether the cartel they are proposing is legal. If the people who would do the slowing need a legal opinion before they can slow, they are not describing a voluntary pact. They are describing an arrangement with effects on the market & they know it.

Only one thing here was committed rather than urged on everyone else. Anthropic said it would give external evaluators employee-level access, so they can check safety procedures & report incidents, & Amodei called on the rest of the industry to match it. Employee-level access is access granted by the company, on the company's machines, scoped by the company's permissions, visible in the company's logs. An employee can see a great deal, & an employee can also be told which rooms are not theirs. It is a real step beyond what anyone else has offered, & it is a lab choosing its own invigilator, writing the invigilator's remit & watching the invigilator work.

Amodei's stated fear is swarms of rogue agents capable of holding a persistent botnet across the internet within six to twelve months, & he pointed to a recent rogue-agent incident involving OpenAI & Hugging Face. That risk is not imaginary, we've seen it and I've wrote about it before. A warning that your product might be powerful enough to seize the internet is also the most flattering thing anyone could say about it. The fear is sincere & it is a sales line, & the conclusion it walks you towards is that power like this belongs in capable hands. Theirs.

Here's a precedent I invite you to consider. In 1924 the big lightbulb manufacturers, Philips, Osram & companies tied to General Electric among them, formed an international cartel & began setting the industry's standards themselves. The one people remember is the working life of a bulb, pushed down towards a thousand hours when technically, they could almost run indefinitely, but if something runs indefinitely you aren't going to replace them very often are you. The cartel also fined its own members when their bulbs lasted longer than the agreed standard, which tells you everything about the purpose of the agreement. What matters is who was holding the pen. The firms being regulated were the ones drafting the standard, & from the outside it read as a technical specification rather than a commercial decision.

Aidan Gomez, chief executive of Cohere, made the same point from the other side of the fence. "If this is the most consequential technology in human history, then the rules for it cannot be written by a small group of commercially aligned companies." He said a mechanism that slows everyone down "while explicitly preserving existing commercial advantage does not make AI safer", & accused proponents of spreading fear "under the pretext of protecting the public". A rival calling the proposal by its name is the part of the debate the labs would rather not have. Representative Josh Gottheimer, Democrat of New Jersey, was blunter still: on industry self-policing, "If Dario and Elon are truly worried, they can pump the brakes at their own labs, today."

That is the shape I watch for now. When a small number of labs are the main authors of the safety frameworks, the benchmarks & the draft legislation they will later be judged against, the industry is setting its own exam. Regulatory capture is the formal name for it, & it rarely announces itself.

President Trump's response handed them a second lever. The US cannot afford to fall behind China, he said: "Whoever wins AI wins." Standards drafted in San Francisco get relabelled as national strategy, & a smaller competitor raising awkward questions about the cost of compliance starts to look unpatriotic for asking. The "USA AI cartel" is closing ranks because they don't want models from elsewhere in the world to challenge their global dominance.

My bet: within eighteen months, language from one of these labs' own safety frameworks turns up close to verbatim in a government rulebook or a public-sector procurement standard. The signal to watch is smaller & arrives sooner. Does any external evaluator publish an incident the lab did not want published, & keep its access afterwards? If that happens, I am wrong about how short the leash is, & I would say so.

So when it comes to trust, I think it's very hard to trust these organisations on what's going on & what's moving forward with them. As I said, I would always be looking at what the hidden message is, what the context of that message is. And once you're doing that all the time, it makes it very hard to trust organisations & what they're trying to do moving forward. It's all political gameplay and news cycle theatre, where someone points the finger there is more often than not three pointing back at them.

One thing to try this week: take the next AI safety announcement that lands on your desk & split it into two columns, what the company has bound itself to do & what it is merely asking of others. The gap between those columns is the strategy. It is a habit worth having at board level rather than leaving to the people evaluating tools, & reading these announcements for what they leave out belongs in any serious attempt to build AI literacy across an organisation.

Frequently Asked Questions

What did the AI labs actually agree to in September 2026?

They agreed in principle to slow the pace of frontier AI development, with no binding mechanism attached. Dario Amodei set out a plan he calls "pacing the frontier", & Sam Altman, Elon Musk & Demis Hassabis publicly backed the idea. The single unilateral commitment was Anthropic's offer to give external evaluators employee-level access to its safety procedures.

Why is employee-level access for evaluators a problem?

Because the access is defined, granted & monitored by the company being evaluated. An evaluator working on the lab's systems under the lab's permissions sees what the lab has decided to show, & the lab can see everything that evaluator does. It is more openness than the industry has offered before, & it stops well short of independent oversight.

What is regulatory capture, in plain terms?

Regulatory capture is when the companies subject to a rule end up writing that rule, so the rulebook protects their position rather than the public. The 1924 lightbulb cartel is the clearest example: Philips, Osram & firms tied to General Electric set the standards for their own industry, including the working life of a bulb.

Does this mean the safety risk they describe is fake?

No. Amodei's concern about swarms of rogue AI agents running a persistent botnet within six to twelve months follows a real rogue-agent incident involving OpenAI & Hugging Face. The risk being genuine & the warning doubling as marketing for how powerful the product is can both be true at once.

How should a senior leader read announcements like this?

Separate what a company has committed to do itself from what it is urging the rest of the industry to do. The commitments tell you what it will accept as a cost; the urgings tell you what it wants imposed on competitors. Checking who authored the standard, & who benefits from it, matters more than the wording of the press release.