Coconut, reasoning in a continuous latent space
A model that thinks in vectors instead of words. Coconut feeds a language model's last hidden state straight back in as its next input, skipping the token-decode step entirely so the reasoning never touches the vocabulary. Three interactive artefacts below take apart why that helps, and what it costs.
Coconut Chain Of CONtinUous Thought is a method for letting a language model reason in its own continuous latent space instead of in words. Standard chain-of-thought emits each reasoning step as text tokens. Coconut removes that step: it takes the model's last hidden state and feeds it directly back in as the next input embedding, so reasoning happens as a sequence of continuous vectors rather than a sentence.
Three claims sit at the centre of the paper. Reasoning in latent space removes the information bottleneck of forcing thought through a small discrete vocabulary. A single continuous thought can hold a superposition of several candidate next steps at once, giving the model an emergent breadth-first exploration rather than a greedy commitment to one path. And on planning-heavy tasks this beats standard CoT while generating far fewer tokens at inference.
It matters here because it is a working existence proof that latent reasoning is not just possible but sometimes advantageous exactly the optimisation pressure that pushes systems away from legible text and toward opaque internal channels. See R·018 for the defensive framing of that same pressure.
The token bottleneck
Ordinary generation runs a loop: a hidden state produces logits, the logits are collapsed to a single discrete token, that token is embedded, and the embedding drives the next forward pass. Two things about this are wasteful.
First, discretisation is lossy. The final hidden state is a rich, high-dimensional continuous vector. Collapsing it to one token from the vocabulary keeps only about log₂(vocab) ≈ 17 bits and throws the rest away before it can inform the next step. Second, most reasoning tokens are not doing reasoning work many exist for fluency and grammar, not computation, yet the model is forced to spend compute generating all of them and to shape its thought like a sentence.
The artefact below makes the loss concrete. The left field is a continuous hidden state; decode it and watch almost all of it dissipate to leave a single token. Keep it continuous the Coconut move and the whole vector survives into the next step.
The feed-back loop
Coconut introduces two modes and two special tokens. <bot> (begin-of-thought) and <eot> (end-of-thought) bracket the latent segment: everything between them is processed in latent mode, everything outside in normal language mode.
Inside the bracket the model does not decode its hidden state to a token. The last-layer hidden state at the current position is used directly as the input embedding for the next position. Each fed-back vector is a continuous thought: no LM head, no sampling, no embedding lookup. Because that vector carries the actual hidden state forward, it is fundamentally different from fixed learnable pause or filler tokens it transmits real information from the previous step, not just extra generic compute.
Toggle the loop. In Standard CoT the signal is forced through the decode nodes and pinched to ~17 bits at the sampling step. In Coconut those three nodes drop out and the hidden state short-circuits straight to the next input.
A staged curriculum
Continuous thoughts have no ground-truth target there is no correct vector to supervise them against. So Coconut is trained with a curriculum, inspired by stepwise internalisation (iCoT), that gradually converts language reasoning into latent reasoning. Starting data is ordinary CoT: a question, a chain of language steps, and an answer.
- Stage 0. Train on the full language chain-of-thought all steps in words to bootstrap normal reasoning.
- Stage 1. Replace step₁ with c continuous thoughts; keep the remaining steps and the answer in language and supervise on those.
- Stage k. Replace the first k reasoning steps with k×c continuous thoughts; supervise on the language steps that remain plus the answer.
- Continue until the reasoning is largely latent.
The loss is ordinary next-token cross-entropy, but only on the remaining language tokens. The continuous thoughts receive no direct loss; they are shaped indirectly, because the hidden state is differentiable and gradients flow back through them. They are optimised to be whatever is most useful for producing the correct downstream tokens.
Practical cost. Because each continuous thought depends on the previous step's hidden state, the latent segment cannot be teacher-forced in one parallel pass; training needs multiple sequential forward passes, roughly one per continuous thought. KV-caching softens this but does not remove it. Resetting the optimiser state between stages was found to help stability.
What runs at test time
The model reads the question in language mode, emits <bot>, runs a fixed or learned-stopping number of continuous-thought steps feeding each hidden state back as the next input then emits <eot> and switches back to language mode to decode the answer. The reasoning phase produces no readable tokens: only the answer is in words. That single fact is the source of both the efficiency (fewer generated tokens) and the opacity (no legible trace).
Latent breadth-first search
The most interesting result is why Coconut plans better. By taking a continuous thought and decoding it back through the LM head to see which tokens it sits closest to, the authors found it does not encode a single committed next step. It encodes a distribution over several plausible next steps at once.
That is an emergent breadth-first search. The expressiveness of the continuous vector is exactly what lets the model carry multiple hypotheses without committing, which is why the advantage shows up on branching, planning-shaped problems. The artefact below runs the same graph two ways: a greedy walker that commits early and hits dead ends, and a superposed frontier that spreads across the branches and converges.
Where it wins
The paper evaluates on GSM8K (grade-school math word problems), ProntoQA (logical reasoning over ontologies), and ProsQA a dataset introduced in the paper, built to demand genuine planning over a directed graph with branching paths and dead ends.
On planning-intensive tasks (ProsQA, and the harder ProntoQA settings) Coconut substantially outperforms standard CoT. Text CoT tends to commit early and then hallucinate its way down dead ends; latent reasoning defers commitment and prunes wrong branches. Coconut also generates far fewer tokens for comparable or better accuracy, and chaining more continuous thoughts higher effective latent depth improves performance on complex reasoning, evidence the latent steps are doing real computation. On raw arithmetic (GSM8K) language CoT stayed very strong, so the clearest gains are on search and planning, not arithmetic chains. (Exact accuracy and token-count deltas are worth confirming against the paper's tables.)
Safety and latent communication
Coconut is intra-model latent reasoning a model talking to itself in vector space across steps. But it is the mechanistic primitive for the inter-model latent communication worth watching: a continuous thought is precisely the kind of representation one model could hand to another instead of text.
It erodes chain-of-thought monitorability by design. The entire safety value of readable CoT is that an overseer can inspect the reasoning; Coconut deletes that trace. It is a concrete existence proof that removing legibility can improve capability which is exactly the gradient that drives real systems toward opaque channels. The same superposition that makes latent thought powerful is what gives a covert channel its capacity: expressiveness and illegibility are two sides of one coin.
The paper also demonstrates the main available defence. Decoding continuous thoughts back through the LM head a logit-lens / probing style read-out is exactly the kind of instrument you would want for monitoring latent channels. It works partially: it reveals the superposition, but it does not recover a clean, auditable step. For a governance frame, adopting latent reasoning for efficiency forfeits the audit trail that text CoT provides a documented loss of traceability and explainability, with a clear but costly mitigation: mandate legible checkpoints at trust boundaries.
Open questions
- QStopping criterion. How many continuous thoughts should run fixed count or learned stopping and how much does the choice cost in accuracy versus compute?
- QOversight of superposed thought. Logit-lens read-out reveals a distribution, not a step. What would it take to make a latent trace auditable enough to satisfy a real oversight regime?
- QGenerality. The advantage is sharpest on planning and search. Does latent reasoning help, hurt, or wash out on open-ended, knowledge-heavy, or long-horizon tasks?
- QTraining cost. The sequential multi-forward-pass requirement makes latent training notably more expensive than parallel teacher forcing. Does that scale, or cap where it can be used?