Anthropic staff put extinction odds above 10% — the real news is the ask for enforceable pacing, not the number

Three researchers connected to Anthropic have said publicly that AI could kill off humanity within the decade, the Guardian reports, and one of them has resigned over it. Jacob Coxon, who previously worked at OpenAI, wrote on social media that "neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives," adding that the fear is genuine rather than positioning: "This is not a marketing stunt... many executives and senior researchers will couch their phrasing in the press to sound sensible — but I hear the same people express fear privately."

What makes this different from the usual doom cycle is who backed him up. Evan Hubinger, who describes himself as a lead in Anthropic's alignment division, wrote: "We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to." Samuel Marks, Anthropic's scalable oversight lead, posted a personal-capacity analysis saying developers believe their technology could cause human extinction or similarly bad outcomes, that it "could happen in the next few years," and — the detail worth underlining — that "in general, the more senior the employee, the more concerned they are." These are people still employed by the company, publicly grading their employer's plan as incomplete.

Anthropic's response to the Guardian defended the strategy without disputing the premise: transparency about "enormous benefits and unprecedented risks," pioneering work in mechanistic interpretability (the science of inspecting a model's internals to understand how it works), the first published Responsible Scaling Policy, and aggressive testing for dangerous capabilities in cybersecurity and biology. The statement ends with the sentence that actually matters: "the world would benefit from the industry adopting a lawful, verifiable way to work together to pace how we release powerful models." That is a company asking to be allowed to coordinate — a request that, in most other industries, runs straight into antitrust law.

Our thesis: the percentage is not the story, and treating it as one plays into both the hype and the backlash. A >10% credence is a considered guess, not a measurement — there is no instrument that produces it, and the honest read is that a senior alignment researcher thinks the problem is unsolved and the schedule is set by competition rather than by readiness. That is a governance fact, and it is independent of whether the number is 10% or 0.1%. The structural problem is visible without any forecast at all: firms that say the technology might end the species are also the ones that cannot unilaterally slow down without handing the lead to someone less careful. Voluntary safety policy does not survive that pressure. Verifiable, legally sanctioned coordination might.

There's a second trap to avoid, and it's the one we'd flag hardest. Existential framing has a way of vacuuming up all the political oxygen and leaving nothing for the harm that is already measurable. OpenAI president Greg Brockman has conceded that "we underestimated the real-world cyber capabilities of our AI models" — that is a concrete, dated admission about a capability shipping today, and it belongs in the same conversation as automated fraud, industrialized social engineering and labor displacement. A regulatory agenda that spends itself on superintelligence and ignores the agentic attack surface would be worse than useless; it would be the appearance of action.

And none of this changes the horizon. The reason to build these systems carefully is that the payoff is enormous — models that compress drug discovery, catch disease before symptoms, and eventually lift the material floor for everyone. You do not get there by pretending the risks are imaginary, and you do not get there by stopping. You get there by making the pace a decision someone can be held to, rather than an emergent property of a race. When a company's own alignment lead says there is no plan yet, the correct response is not panic and not dismissal. It's to ask what a plan would have to look like, and who would be empowered to check it.