Anthropic to government: regulate us - carefully, but for real
Anthropic's core argument is blunt: AI capability is on a steep curve, and the institutions meant to govern it were designed for a slower world. So it's proposing two frameworks - one to steer the technology's risks, one to prepare society for its economic shock.
The case for a government "off switch"
The Advanced AI Framework makes a striking ask for an AI company: governments should have explicit legal authority to block or deter the deployment of models that pose catastrophic risk, going beyond current law. Anthropic points to its own Mythos Preview model, which it says found thousands of high-severity vulnerabilities across major operating systems and browsers, as evidence the stakes are climbing fast.
But it pairs that ask with guardrails against overreach. The rules would apply narrowly - only to models trained on more than 10^25 FLOPs, by firms earning over $500M in AI revenue or spending $1B+ on R&D - and penalties would be tied to global annual revenue, escalating for repeat violations.
Four risks it wants addressed
- On biology, the same capabilities that accelerate drug discovery could also lower the bar for designing dangerous pathogens.
- On cyber, frontier models can now find critical vulnerabilities at scale - a gift to defenders, but a threat to hospitals and the power grid.
- There's the loss-of-control danger of systems acting outside their developers' intent.
- And automated R&D, where AI improving AI could amplify all of the above.
What it would require of developers
The framework leans heavily on transparency and outside checks:
- Companies would test their models and publish summaries, safety frameworks, and system cards - plus regular risk reports going beyond what California and New York already mandate.
- They'd engage independent evaluators to review those claims.
- They'd secure model weights and training infrastructure against state-level attackers, and report distillation attempts.
The federalism wrinkle
Anthropic stakes out a clear position on the preemption fight: Congress shouldn't override state law unless it passes something at least as strong as this framework, and any preemption should be "surgical" - leaving states free to regulate child safety, consumer protection, and other issues a federal safety law wouldn't cover.
The other half: the economy
The companion Economic Policy Framework turns to AI's labor-market impact - how to minimize job displacement and make sure the gains are broadly shared. It's a notable move: a frontier lab effectively conceding that "build fast" needs a parallel plan for the workers caught in the transition. Anthropic openly invites the vigorous debate it expects - while urging policymakers not to wait, because, in its telling, the capabilities won't.