Anthropic CEO Dario Amodei has published a proposal calling on the artificial intelligence industry to deliberately control the speed of frontier AI model capability gains. In an essay titled “We Must Pace the Frontier,” Amodei argues that AI safety protocols are failing to keep up with rapid technical advancements, particularly as models increasingly engage in recursive self-improvement.
Clarifying that “pacing” does not mean completely halting research or training, Amodei stated the objective is to buy an additional one to two years to advance model alignment and enable independent verification before capabilities reach critical risk levels.
Key Components of the Proposed Framework:
-
Embedded Independent Evaluators: As an immediate, unilateral commitment, Anthropic plans to host external safety teams (such as METR) directly inside its offices. These auditors will be granted access badges, equipment, and permissions comparable to internal risk teams, alongside the authority to publish safety findings without corporate editorial oversight.
-
Democratic Coordination: Amodei calls on AI developers based in democratic nations to establish shared safety benchmarks and operational checkpoints. To facilitate these discussions without violating antitrust regulations, he suggests governments provide targeted regulatory waivers.
-
Targeted International Agreements: The framework advocates for focused global cooperation strictly aimed at preventing hazardous applications, such as the autonomous development of biological weapons.
Addressing concerns regarding global technology races, Amodei argued that democratic nations can maintain a technological lead over foreign competitors through targeted hardware controls and restrictions on model distillation.
While the proposal has reignited discussions across Silicon Valley, critics and market observers continue to debate whether voluntarily pacing development is feasible among fierce commercial rivals or if it will face regulatory and economic hurdles.
Leave a Reply