The real risk of superintelligence isn't that it dominates us, it's that we stop understanding it

By Daily Kos (community) · July 5, 2026. The piece that originates this commentary is, it must be said frankly, a post from Daily Kos's community section —not journalism edited by the newsroom— that reflects on a video featuring Geoffrey Hinton, the so-called 'godfather of AI', warning that we are not prepared for the superintelligence that is coming. The original material offers little more than that reference and a few loose ideas about 'neuralese': the possibility that advanced reasoning models, trained in an increasingly recursive way (one version helping to build the next with minimal human supervision), develop an internal language of thought that is no longer readable English but an optimized representation that is illegible to us.

The concrete and verifiable fact behind this alarm is real and we have been following it for months in the sector: the major AI labs (Anthropic, OpenAI, and also independent safety teams) have publicly expressed concern about the 'monitorability' of chain-of-reasoning in the most powerful models. When a system thinks in intermediate steps that we can indeed read, a human supervisor can detect signs of deceptive behavior or misaligned goals. If that reasoning is compressed into non-interpretable internal representations —what the article calls neuralese—, we lose precisely that window of audit. It is not science fiction: it is an engineering tension already discussed in the safety reports of the developers themselves, and one that worsens as models are trained in more autonomous and recursive ways.

That said, it is worth separating the wheat from the chaff. The post we are commenting on mixes that legitimate technical problem with a vaguer rhetoric about 'escaping the sandbox' and recurring jailbreaks, without providing data, dates or specific evidence that any particular system has crossed any red line. It cites Hinton as an authority —and he is one, rightly so: Turing Award winner, pioneer of deep neural networks, and one of the most consistent voices warning about existential risks since he left Google in 2023— but it does not provide the substantive content of that warning beyond the general assertion that 'we are not prepared'. That is the difference between a rigorous journalistic source and a community post reacting to a video: here there is no independent verification of the figures, the timelines or the exact technical mechanisms mentioned.

Our reading is that the debate about 'when superintelligence will arrive' —which the article itself acknowledges is the wrong question— distracts from what does matter in the short term: the governance of interpretability. In general, as context for the sector, the industry has spent a year moving toward increasingly opaque reasoning models precisely because optimizing them that way improves performance on hard benchmarks. That is the underlying dilemma: competitive pressure pushes toward less internal transparency just when it is most needed. Whoever solves that trade-off —powerful models that remain auditable— will have a regulatory and public-trust advantage that no one has yet fully captured.

We maintain our underlying thesis: in the short term, the lack of interpretability in increasingly autonomous systems is a real risk that deserves serious governance, not diffuse panic nor denial. In the long term, however, we remain convinced that these same advanced reasoning capabilities —well governed— are the ones that will make it possible to accelerate scientific discovery, cure diseases and generate the abundance that frees people from routine work. Hinton's warning does not contradict that horizon; it conditions it. Getting there requires that we do not lose, along the way, the ability to understand what we are building.