What is latent reasoning? How AI can think without words

By Turing Post · July 27, 2026.

The editorial in this edition of FOD (Turing Post) raises a fundamental question about today's reasoning models: why does a machine have to think in language? According to the email, reasoning models now spend thousands of tokens solving hard problems in the form of text, even when no one ever reads that generated text. A growing group of researchers proposes replacing those token chains with continuous hidden states, capable of holding more information and of exploring several reasoning directions at once. The email notes that this shift could reduce inference costs and improve planning, but would also make the model's reasoning much harder to inspect.

The article opens by recalling how Chain-of-Thought (CoT) became popular: models learned to produce ever longer reasoning traces, reinforcement learning rewarded the chains that led to correct answers, and inference systems began reserving thousands of "thinking tokens" before returning an answer. The results were remarkable in mathematics, programming and planning, but the models became slower and considerably more expensive. The text sums it up with irony: "we taught machines to reason by writing essays that no one would ever read", and that expense became economically viable ($$).

At the same time, according to the email, the habit of reading those "essays" became widespread: evaluators, safety teams and an entire oversight practice were built on the idea that a model's reasoning is an inspectable document. The author qualifies that this was a side effect: Chain-of-Thought emerged simply as a way of giving a language model more compute, not as a deliberate window into its interior. The only workspace a token predictor has is the token sequence, so that "scratchpad" turned out to be written in English by accident of design, not by intention.

The section "Language was useful scaffolding" explains why language has been useful but limited scaffolding. Models generate one token at a time, and each token becomes part of the context for the next, which turns text into a convenient workspace: it allows a problem to be broken down into steps, intermediate results to be preserved, errors to be detected and attempts to be retried. But natural language, the email recalls, was designed for communication between humans, not for the internal computation of a neural network. Every word forces the model to make a discrete decision: a rich internal state, which could hold several possible interpretations, must collapse into a single token ("therefore", "seven", "left", "false"), and the reasoning that follows starts from that commitment already made. Language also adds overhead in the form of transitions, repetitions, grammar and filler phrases such as "let's analyze this carefully", which consume tokens without necessarily advancing the solution. The email compares this to a chess engine or to AlphaGo, which do not need to write a paragraph explaining every move they consider, whereas many of today's reasoning systems do exactly that in practice.

The email includes a comparison table between Chain-of-Thought and latent reasoning, with these differences: the reasoning medium goes from discrete language tokens to continuous hidden representations; computation goes from sequential textual steps to iterative updates inside the model; the possible paths go from normally committing to a single written route to being able to preserve several possibilities simultaneously; inference cost, which in CoT grows with the length of the reasoning tokens, may require fewer decoding steps in latent reasoning; training, which in CoT works naturally with token-level RL, in latent reasoning requires new objectives and training policies; and visibility, which in CoT is partially readable (though not always faithful), in latent reasoning is hard to interpret directly. The most important distinction, according to the text, is that latent reasoning separates computation from communication: the model can work in its native representational space and return to language only when it is ready to answer.

The section "From chains of words to continuous thought" reviews the work that made this idea legible. It mentions that there were earlier attempts —pause tokens, filler tokens, implicit Chain-of-Thought through distillation— but that **Coconut** (Chain of Continuous Thought), from Meta, was the clearest demonstration: instead of decoding a hidden state into a token, Coconut feeds that state directly back into the model. A token represents a single chosen direction; a continuous state can preserve a mixture of possible directions. The email indicates that on some planning tasks, Coconut showed behavior similar to a breadth-first search, considering several possibilities before committing to one.

Another work cited is **Soft Thinking**, which creates intermediate "concept tokens" by combining several token embeddings instead of selecting a single word. According to the email, the researchers reported modest accuracy gains of up to 2.48 points in pass@1, along with a reduction in token usage of up to 22.4%; the email itself clarifies that those two headline figures come from different models, something worth bearing in mind before citing them. The text adds a relevant nuance: the authors of Soft Thinking claim their outputs remain readable, because a probability-weighted mixture of embeddings can be projected back into text, something Coconut does not offer, since a hidden state is not a sentence that has simply had its punctuation removed. Hence the email concludes that latent reasoning is not a single thing: one branch "softens" the token stream and keeps something inspectable, while the other branch abandons the token stream entirely, and the two do not cost the same in terms of lost visibility.

The email also mentions, without going into detail, other projects that use recurrent loops, diffusion-style refinement, compressed reasoning states or small networks that repeatedly revise their answer, noting with humor that the terminology is already saturated (continuous thought, soft thought, latent chains, implicit reasoning, recurrent depth, abstract Chain-of-Thought), as if AI research had once again discovered "the same continent under twelve different flags". Even so, the author maintains that the underlying trend is coherent: reasoning is beginning to separate from text generation.

The section "Reinforcement learning was the wall" argues that latent reasoning has had a serious disadvantage: explicit reasoning fits naturally with reinforcement learning because the model produces tokens, each with an associated probability, which makes it possible to compare trajectories and update the policy with methods such as PPO or GRPO. Continuous hidden states, by contrast, do not offer that same clean interface: they are representations, and it is harder to assign probabilities to the transitions or to determine when the model should stop "thinking". The email indicates that the past few months have brought a concerted effort to solve this problem.

As sector context, it is worth recalling that Turing Post is a newsletter focused on AI research and trends that often devotes entire editions (like this one, part of its "FOD" series) to explaining a single technical concept in depth, in this case latent reasoning versus explicit Chain-of-Thought.