What is latent reasoning? How AI can think without words

By Turing Post · July 27, 2026.

The editorial in this issue of Turing Post raises a fundamental question about the future of reasoning models: why should a machine have to think in words? Reasoning models currently spend thousands of tokens solving hard problems in natural language, even when no one ever reads that intermediate text. A growing group of researchers proposes replacing those token chains with continuous hidden states, capable of carrying more information and exploring several reasoning directions at once. The shift could cut inference costs and improve planning, but it also makes the model's reasoning far harder to inspect.

The article opens by recalling how, over the past few years, it has become common to see AI models "talk to themselves": phrases such as "let's think step by step," "wait," "alternatively" or "let me reconsider" are the manifestation of Chain-of-Thought (CoT), a technique that worked so well the industry turned it into a scaling strategy. Models learned to produce ever longer reasoning traces, reinforcement learning rewarded the traces that led to correct answers, and inference systems began reserving thousands of hidden "thinking tokens" before returning an answer. The results were remarkable: models improved a great deal at mathematics, programming and planning. But they also became slower and considerably more expensive: machines were taught to reason by writing, in practice, essays no one would ever read, and that expense became economically bearable.

Along the way, the industry picked up a second habit: it began reading those essays. Evaluators read them, safety teams built monitors on top of them, and an entire practice grew up around the idea that a model's reasoning is a document that can be inspected. The piece insists it is worth remembering where that document came from: Chain-of-Thought was, originally, a way to give a language model more compute, and its readability was a side effect, because the only workspace a token predictor has is the token sequence itself. No one deliberately designed a window into the model; what was built was a scratchpad that happened to be written in English.

From there, the article explains why language was, at the time, a useful but imperfect scaffold. Language models generate one token at a time, and each token becomes part of the context that determines the next one. That makes text a convenient workspace: a model can break a problem into steps, keep intermediate results, spot errors and try again. But natural language was designed for communication between humans, not for the internal computation of a neural network. Every word forces the model into a discrete decision: a rich internal state, containing several possible interpretations, has to collapse into a single token ("therefore," "seven," "left," "false"), and the rest of the reasoning proceeds from that commitment already made. Language also carries overhead — transitions, repetitions, grammar, filler phrases such as "let's analyze carefully" — that consumes tokens without necessarily advancing the solution, on top of the need to "be polite" because, in theory, a human might read that output.

The text explicitly compares Chain-of-Thought with latent reasoning along several dimensions: the reasoning medium (discrete language tokens versus continuous hidden representations), the type of computation (one textual step after another versus iterative updates inside the model itself), the possible paths (usually a single written path versus the ability to preserve several alternatives at once), inference cost (which grows with the length of the textual reasoning versus a potentially smaller number of decoding steps), training (which works naturally with token-level reinforcement learning versus the need for new objectives and policies) and visibility (partially readable though not always faithful, versus something hard to interpret directly). The most important distinction, according to the article, is that latent reasoning separates computation from communication: the model can work in its own native representational space and return to language only when it is ready to answer.

To illustrate this idea, Turing Post reviews the work that made the concept of latent reasoning legible to the rest of the sector. It notes there were earlier attempts — pause tokens, filler tokens, implicit Chain-of-Thought through distillation — but singles out Coconut ("Chain of Continuous Thought"), from Meta, as the clearest demonstration. Instead of decoding a hidden state into a token, Coconut feeds that state directly back into the model. That seemingly small change alters the reasoning process: a token represents a single chosen direction, whereas a continuous state can preserve a mixture of possible directions. On some planning tasks, Coconut displayed behavior resembling a breadth-first search, weighing several possibilities before committing to one.

The article also describes Soft Thinking, a related approach that creates intermediate "concept tokens" by combining multiple token embeddings instead of selecting a single word. According to the email, researchers reported modest accuracy gains — up to 2.48 points on pass@1 — along with a reduction in token use of up to 22.4%, although those two figures come from different models, a caveat the newsletter itself flags when citing them. It also highlights a notable finding from that same work: Soft Thinking's outputs remain readable, since a probability-weighted mixture of embeddings can be projected back into text, something Coconut does not offer, because a hidden state is not simply a sentence without punctuation.

Hence the newsletter's conclusion that latent reasoning is not a single thing: one branch "softens" the token stream and keeps something observable, while the other abandons the token stream altogether. The two do not cost the same in terms of visibility, and that difference will determine how much inspectability is lost. The email also mentions, more briefly, other projects that use recurrent loops, diffusion-style refinement, compressed reasoning states or small networks that repeatedly revise their answer, noting wryly that the terminology is already saturated (continuous thought, soft thinking, latent chains, implicit reasoning, recurrent depth, abstract Chain-of-Thought), as if AI research had discovered the same continent under twelve different flags. Even so, the underlying trend is consistent: reasoning is starting to separate from text generation.

Another section of the email tackles what the newsletter calls the reinforcement learning "wall." Explicit reasoning fits naturally with reinforcement learning because the model produces tokens, each with an associated probability, which makes it possible to compare trajectories and update the policy using methods such as PPO or GRPO. Continuous hidden states, by contrast, do not offer that same clean interface: they are representations, and it is harder to assign probabilities to transitions or to determine when the model should stop "thinking." The text says the past few months have been a concerted effort to solve this problem.

Finally, the top of the newsletter includes the usual "This Week in Turing Post" section, with a note that the publication is on its summer schedule: the final episode of the series "The Org Age of AI" was published on Friday, and Sunday brings the "Library" section devoted to the most interesting computer-use AI agents, though the email does not elaborate on the content of either.