Parrot or world model? Five years on, the right question is no longer whether AI ‘understands’

THE THESIS. ‘It just predicts the next word’ accurately describes how a language model is trained. It poorly describes what happens inside it. And neither tells us whether it ‘understands’. Five years on, our reading is that three questions need separating. First: does it work? Yes, and increasingly well. Second: does it generalize beyond its training data? Sometimes, unevenly, and with measurable failures. Third: does it understand in the philosophical sense? A fair question, but current science cannot settle it. Nor is it the one that decides whether AI changes our lives. The question that matters in practice is the second. That is where to look at data rather than slogans.

WHERE THE PARROT CAME FROM. The phrase comes from ‘On the Dangers of Stochastic Parrots’, presented at the FAccT conference in March 2021. Its authors were linguist Emily M. Bender, researcher Timnit Gebru, Angelina McMillan-Major and Margaret Mitchell. The core argument: a language model stitches together word sequences based on learned probabilities, with no reference to meaning. ‘Stochastic’ means random according to probabilities. The paper also had a corporate backstory: according to Gebru and Mitchell, their departures from Google were tied to it; Google gave a different account. One point is often forgotten. The paper was not mainly a philosophical treatise. It was about concrete risks: environmental cost, bias from unfiltered data, and the danger of people reading intent into fluent text. Those risks remain. What has aged worse is the metaphor as a technical description.

WHAT IS INSIDE. The strongest reply to the metaphor came from interpretability, the field that tries to read what a network computes internally. In 2022, Kenneth Li and colleagues trained a small model, Othello-GPT, only on lists of legal moves from the board game Othello. It never saw a board. Yet by probing its activations (its internal numerical values) they found a representation of the board state. Crucially, when they edited that representation, the model changed its moves to match the altered board. That is causal evidence, not mere correlation. In 2023, Wes Gurnee and Max Tegmark showed that Llama-2 encodes, almost linearly, the geographic location of places and the dates of historical events. In 2025, Anthropic published ‘circuit tracing’ work on Claude 3.5 Haiku. According to the company, the model chains intermediate steps (it activates ‘Dallas is in Texas’ before naming the capital of the state Dallas is in). It also picks the rhyming word before writing the line. Our reading: ‘it just predicts the next word’ is as true and as uninformative as saying a chess player ‘just moves pieces’. To predict the next word well in text about the world, it pays to build representations of the world. That is not human understanding. But it is not a parrot repeating either.

THE FAILURES THAT WON’T GO AWAY. The other side of the scale has data too. The ‘reversal curse’ (Berglund et al., 2023) showed that a model trained on ‘A is B’ does not, by itself, infer ‘B is A’. In their example, GPT-4 correctly named Tom Cruise’s mother 79% of the time. But it named that same woman’s famous son only 33% of the time. In 2024, Apple released GSM-Symbolic: the same grade-school math problems with names and numbers swapped. Performance dropped. When an irrelevant sentence was added, the fall reached up to 65% for some models, according to the authors. In 2025, Apple’s study ‘The Illusion of Thinking’ described reasoning models collapsing on puzzles such as the Tower of Hanoi as complexity rose. That study drew rebuttals: some critics attributed part of the failures to output-length limits and unsolvable puzzles. And Subbarao Kambhampati has spent years documenting that disguising the names in classic planning problems makes performance fall. Our reading: these failures do not prove ‘they cannot reason’. They prove their reasoning is uneven. It is very good where training data is dense and brittle where the surface of a problem shifts. That is the signature of a system that mixes general rules with statistical shortcuts.

CHOLLET’S ARGUMENT. François Chollet, creator of the Keras library, proposed a useful definition in 2019: intelligence is not what you can do, but how efficiently you acquire new skills from little experience. That idea produced ARC-AGI, a set of visual puzzles that are easy for anyone and designed to resist memorization. ARC’s recent history sums up the whole debate. Models with extended reasoning have climbed sharply on the earlier versions: independent trackers put the best systems above 80% on ARC-AGI-2, though those figures still await full official verification. The foundation answered with ARC-AGI-3, built on interactive, game-like environments. At launch, according to The Decoder and ARC Prize itself, frontier models scored below 1%, while untrained humans solve them. The foundation itself warns that its results should not be read as evidence of AGI, either way. Our reading: every time a ‘memorization-proof’ test is designed, models eventually improve on it. But the next frontier also appears. That is not AI failing. It is the scientific method working: measuring generalization rather than proclaiming it.

THE CHINESE ROOM AND WHAT FOLLOWS. That leaves the third question. In 1980, John Searle imagined a person locked in a room who manipulates Chinese symbols using a rulebook without understanding a word. In 1990, Stevan Harnad framed the ‘symbol grounding problem’: how does a symbol gain meaning if it only points to other symbols? These are serious objections. Multimodal models and agents acting in real environments partly erode them, but do not resolve them. Our position is pragmatic. Understanding is not an on-off switch. It is a gradient of measurable abilities: representing the world, transferring what was learned, and correcting errors. The short-term implications are concrete. Be wary of both ‘it’s just autocomplete’ (which leads to underestimating the impact on jobs and cybersecurity) and ‘it already reasons like a human’ (which leads to unsupervised delegation in tasks where a small change makes it fail). The useful rule: trust AI where its generalization has been measured, and supervise it where it has not. In the long run we are optimistic. Interpretability is turning the black box into something that can be audited. A system that learns world models, even imperfect ones, is exactly what is needed to accelerate science, from drug design to biology. Whether it understands ‘like us’ may never be provable. Whether it helps us understand more is starting to be.