Why AI leaders warn about catastrophe while they keep building

Why build increasingly powerful AI while warning that it could cause catastrophe? The question becomes easier to examine when we separate a forecast about the technology from a claim about who should develop it, and from a proposal about who should regulate it. Those are three different arguments, even when one executive makes all of them.

A warning is not a measured probability

The Center for AI Safety’s extinction-risk statement includes Sam Altman, Dario Amodei and Demis Hassabis among its signatories. It calls for preventing AI-related extinction to receive attention comparable to other threats affecting society as a whole. The statement itself gives no numerical likelihood, deadline or experiment establishing that outcome. Signing it records a position on the importance of a risk; it does not turn that risk into a measured forecast.[1]

Amodei’s January 2026 essay makes a more detailed argument about possible future systems with exceptional capability, autonomy and scale. He distinguishes dangers from systems’ own behaviour from dangers caused by people using them. He also explicitly acknowledges uncertainty and argues that stronger interventions should depend on more concrete evidence. These are a company leader’s analysis and proposals, rather than experimental proof of a future catastrophe.[3]

In September, he said recent developments had convinced him that prevention also required slowing capability advances. He proposed embedded outside evaluators, coordination among frontier developers and international coordination. His stated position had become more cautious over the intervening months; the January essay should be read alongside that later proposal.[4]

Why a concerned developer might continue

OpenAI addressed this question in its May 2023 governance essay. Its authors argued that the technology could produce large benefits and that stopping development would be difficult as costs fell and more actors entered. Their response combined continued development with coordination, technical safety work and outside oversight. That is the company’s stated reasoning; whether its actions follow it is a further question.[2]

The July 2026 Pacing the Frontier statement gives the competition problem a specific form. It asks the US government to support international work on tools for controlling the pace of automated AI development. Its concern is that companies and countries may each resist slowing alone, even if more time would help everyone manage risk. The site lists employees and researchers across companies; individual signatures and personal comments should not be reported as binding commitments by every employer.[5]

The economic logic is understandable. If one laboratory delays a release while its competitors continue, it may lose customers, investment or influence without substantially reducing worldwide development. A shared rule could change that calculation. Equally, a shared rule designed by dominant firms could make entry harder for competitors. These are possible effects of institutional design, not evidence that any particular signatory is acting dishonestly.

Commercial interests are visible even when private motives are not

The FTC’s January 2025 study of Microsoft–OpenAI, Amazon–Anthropic and Google–Anthropic partnerships described equity and revenue-sharing rights, cloud-spending commitments and access to sensitive information. Staff identified potential effects on access to computing resources and the difficulty of switching suppliers. This provides evidence of overlapping commercial relationships. The study was not a finding about the sincerity of executives’ safety warnings, nor a comprehensive legal assessment of the partnerships.[6]

OpenAI’s 2023 proposal expressly exempted models below a significant capability threshold, including open-source projects, from its proposed licensing and audit regime. That detail matters when assessing claims about whom a safety rule would burden.[2]

A company can sincerely fear a dangerous outcome and benefit commercially from being seen as the developer best equipped to prevent it. Both possibilities deserve examination. The strongest reporting follows concrete decisions: which capabilities trigger restrictions, what evidence is disclosed, whether outside reviewers can disagree, and what the company gives up when a safety commitment becomes costly. Those decisions tell readers more than an unsupported theory about what a chief executive secretly believes.

Report an error or suggest a correction