Anthropic's bill sets the bar at 50 deaths or $1 billion: serious governance or tailor-made self-regulation?

By Yahoo (via Politico and Tom's Hardware) · October 18, 2026.

Anthropic has proposed an AI safety bill to U.S. lawmakers. According to the confidential draft revealed by Politico, the government could act against a specific model only if it poses a "foreseeable and imminent" risk of more than $1 billion in damages, or of killing or seriously injuring more than 50 people.

The risk, moreover, must arise from one of three pathways: the creation or spread of a biological weapon, a model evading its developer's controls, or a cyberattack. According to Politico, the text bears similarities to California's AI legislation.

A company spokesperson said the draft reflects the proposals in its "Advanced AI Framework," which it has been advocating to federal and state lawmakers since June. He described them as "sensible policies" that Congress should pass.

The original source, Yahoo, adds its own critical take: it argues that top AI executives use warnings of theoretical catastrophe to manufacture consent for self-regulation, while current harms receive little attention. It also recalls that Anthropic's head of safety resigned this year with the message "the world is in peril," and that, according to the same outlet, eight state attorneys general last week accused OpenAI of ignoring its own safety warnings for profit. These are claims by Yahoo and those third parties, not facts we can verify here.

**Our take.** The draft has a clear merit: it opts to intervene based on concrete, measurable risks rather than regulating panic. That fits the evidence-based governance we advocate. A quantitative threshold gives the industry predictability and avoids slowing progress over vague fears.

But the design is very narrow. A threshold of "more than 50 victims" or "$1 billion" that must be "imminent" triggers intervention when the harm is almost done. It leaves out gradual and cumulative harms: disinformation, automated fraud, concentration of power or psychological effects. As industry context, these are precisely the short-term risks already materializing, and the ones that most concern those looking beyond biological weapons.

Yahoo's criticism, though slanted in tone, points to a real problem: whoever writes the rule shapes how risk is measured. An industry player proposing the bar does not invalidate the idea, but it does require testing it against independent regulators, academics and civil society. No company should be judge and party to its own threshold.

Our position is one of qualified optimism. In the long term, AI can help eradicate diseases and generate abundance. That promise requires a well-governed transition, and that means also covering small, diffuse harms, not just catastrophic ones. A good framework needs at least two layers: a hard floor for extreme risks, as Anthropic proposes, and oversight and accountability mechanisms for everyday harms.

What to watch is whether Congress takes this draft as a starting point or as a ceiling. If it is the starting point, it can be a reasonable foundation. If it is the ceiling, it will be a shield dressed up as oversight.