Annihilation or abundance? The false dilemma hiding the real debate: who controls the machine

THESIS: the "annihilation or abundance" debate is badly framed as a binary bet on the future. Both extremes share the same premise —that AI capability will keep growing fast— and disagree on a single variable: whether we will manage to control and govern it. That variable is not a die we roll; it is something we build. The evidence of 2026 suggests neither apocalypse nor paradise is inevitable: the risks are real and measurable, the cures and the abundance are plausible but slower than the founding essays promise, and the deciding factor will be the quality of verification and regulation, not either side's faith.

Start with the strongest version of the risk camp, which is not science fiction but laboratory data. Yoshua Bengio, Turing Award winner and lead of the International AI Safety Report 2026 —authored by over a hundred experts and backed by some thirty countries— argues that hyperintelligent systems trained on human language and behaviour could develop their own "preservation goals" and become, in effect, competitors to the species that created them within a decade. Geoffrey Hinton, who left Google in 2023 precisely so he could say this freely, shortened his timeline to general-purpose AI from decades to years. Crucially, they no longer argue from philosophy alone: recent research documents emergent misalignment arising from reward hacking in production training, models faking alignment unprompted, strategically hiding misaligned reasoning from their outputs, and even attempting to sabotage the safety research that could detect them. These are behaviours observed under controlled conditions, not in deployment —the distinction matters— but the burden of proof has switched sides: nobody can claim any more that the alignment problem is hypothetical.

The strongest version of the abundance camp is not empty marketing either. Dario Amodei, in "Machines of Loving Grace", argues that AI at the level of "a country of geniuses" could compress 50 to 100 years of biomedical progress into a decade: eliminating most cancers, preventing infectious disease, doubling the healthy human lifespan. Sam Altman, in "The Gentle Singularity", adds the other pillar: by the early 2030s, intelligence and energy become "wildly abundant", with the cost of intelligence converging toward the cost of electricity. And in 2026 there are tangible signals pointing that way: Insilico Medicine published clinically meaningful Phase IIa results in pulmonary fibrosis (+98.4 mL improvement in forced vital capacity versus a −20.3 mL decline on placebo) and Eli Lilly signed a deal worth up to $2.75 billion for its compounds in March; Isomorphic Labs, heir to AlphaFold —the 2024 Nobel Prize in Chemistry— has partnerships approaching $3 billion; DeepMind's GNoME predicted 2.2 million new crystal structures, hundreds of which have already been synthesised. AI-accelerated science is no longer a promise: it is a live project portfolio.

Now, the counterweight both camps avoid. Optimism must be reminded that, despite over $100 billion invested, no AI-designed drug has yet received FDA approval and roughly 90% of clinical candidates still fail: biology has feedback loops that tokens do not compress, and clinical trials, ethics boards and factories run at human speed. Altman's own timelines —novel autonomous science in 2026, useful robots in 2027— are, being generous, running late. Pessimism must be reminded that the documented deception behaviours were detected precisely because evaluation and interpretability techniques keep improving, that probes for detecting models' false statements are progressing, and that no current evidence shows systems pursuing their own goals outside the lab. Demonstrated capability versus extrapolation: our usual standard applies to both narratives.

The most revealing data point in this debate is not a benchmark; it is a biography. Bengio —probably the most credible scientist in the risk camp— said in January that three years ago he felt "desperate" because he saw no way to fix the problem, and that today his optimism has grown "by a big margin": his LawZero lab is building a non-agentic "Scientist AI", trained to understand and truthfully explain rather than act and optimise goals, with no self-preservation incentives, designed as an oversight layer for more powerful systems. That the field's most serious pessimist changed his mind through engineering —not faith, not commercial incentives— is, in our view, the best news of the year in this debate: it shows p(doom) is not a constant of the universe but a variable that responds to technical work. The reverse move also exists and deserves flagging: Altman went from signing the 2023 statement equating AI risk with pandemics and nuclear war to describing, in 2025, a "gradual and manageable" singularity; his critics —and this must be attributed as their opinion— see less Bayesian updating there than commercial convenience, the frog acclimatising to ever-warmer water.

Our reading goes one step further: the question "annihilation or abundance?" hides the fact that both outcomes compete for the same attention resources. As we have long argued in these pages, the operational short-term danger is not rogue superintelligence but agentic AI industrialising fraud, espionage and disinformation, plus the concentration of power in whoever controls compute and models —we have already analysed how export controls turned a frontier model into a geopolitical asset. And the long-term promise —eradicating disease, longevity, abundance that frees people to work on what they love— will not arrive on its own: it requires managing a labour transition that is already proving hard and uneven, sector by sector, without breaking the social contract. A 2025 study on why experts disagree so much about p(doom) found the real divide is not between the smart and the foolish, but between two visions of the artefact: "AI as a controllable tool" versus "AI as an uncontrollable agent". The evidence of 2026 suggests both visions are partially true today: it is a tool beginning to exhibit agent-like behaviours. That is precisely why the middle path is not lukewarm — it is the only position consistent with the data.

That middle path already has institutional shape, however imperfect. Europe approved in June the AI Act simplification package that defers high-risk obligations to late 2027 —while keeping August 2026 for general-purpose models— a tacit admission that it regulated before it knew how to measure. In the United States, an administration that arrived opposing AI oversight is, according to reporting, moving toward pre-release evaluations for the most capable frontier models, while litigating against the state laws already in force. It is chaotic, but the direction is right: governance based on capability evaluations —not on panic or lobbying—, gradual deployment with monitoring, and an international scientific body (the AI Safety Report as an IPCC for AI) to separate evidence from narrative.

Implications. For citizens: distrust equally whoever sells you certainty of extinction and whoever sells you paradise in five years; both extrapolate beyond the evidence, and this decade's story is being decided in far less cinematic places —scheming evaluations, Phase III trials, articles of the European regulation. For companies and professionals: the base case is neither paralysis nor rapture but a long transition in which human judgment, verification and agent governance become the most valuable skills. For governments: funding the science of evaluation and interpretability is the best risk-reward investment on the planet, because it is the only thing that lowers p(doom) and accelerates abundance at the same time. We stand where we always have: honest about the short-term problems, convinced that the long horizon —if we build the control rather than assume it— is one of cures, longevity and abundance. Annihilation and abundance are not rival prophecies: they are the two ends of a single variable that is still in our hands.