Tele-Lens — How Far Ahead Do LLMs Actually Plan in Chain-of-Thought?

Paper. Liyan Xu, Mo Yu, Fandong Meng, Jie Zhou. How Far Ahead Do LLMs Plan? Uncovering the Latent Horizon in Chain-of-Thought Reasoning. ICML 2026. WeChat AI, Tencent. [arXiv] · [code]

0. The Picture in One Paragraph

A persistent question in CoT interpretability: do LLMs actually plan globally in their hidden states before verbalizing the reasoning steps, or are they making purely local token-by-token transitions? Prior evidence suggests both directions — hidden states seem to encode future information, yet removing explicit CoT steps destroys compositional reasoning. Tele-Lens settles this question empirically with a probing framework that tracks three signals simultaneously across the full CoT trajectory: predicted next tokens, predicted final answer, and predicted total reasoning length. The answer is myopic: for compositional tasks (multi-step algorithmic reasoning), the model’s hidden states carry near-chance final-answer predictions (0.49–0.51) throughout most of the trajectory, spiking to 0.94–0.99 only in the final one or two steps before completion. LLMs do not pre-plan; they compute step by step. This insight has two practical payoffs: a sparse set of “pivot” positions (identified by Tele-Lens signals near CoT completion) can represent the full trajectory’s uncertainty with +6% accuracy improvement, and automatic CoT bypass achieves a 16.2% bypass rate with negligible performance loss by detecting when hidden states already carry high early confidence.

1. The Open Question — Do LLMs Plan Ahead in Their Hidden States?

The empirical evidence before this paper pulled in two directions:

Evidence suggesting latent planning: “Future Lens” (Pal et al., ACL 2023) showed that intermediate Transformer layer activations encode information about tokens and even final answers before those outputs are verbalized. This implies some degree of latent forward planning.

Evidence against global planning: For tasks requiring compositional computation — multi-step algorithms, logical deduction chains — removing explicit CoT steps causes catastrophic performance drops. If the answer were truly pre-planned in latent space, removing intermediate verbalizations should matter less.

Tele-Lens asks: at what point in the CoT trajectory do hidden states reliably encode the final answer? If the model is planning globally, that signal should appear early. If it is computing locally, the signal should appear only near the end.

2. Tele-Lens — A Probing Framework for Teleological Information

2.1 Conceptual lineage

Tele-Lens generalizes two prior probing tools:

• Logit Lens (Nostalgebraist, 2020): bridges each Transformer layer’s hidden state to the LM head, showing how token predictions evolve through depth.
• Future Lens (Pal et al., 2023): extends this to predict subsequent tokens from a single hidden state.

Tele-Lens adds a third dimension — reasoning length prediction — and applies all three probes simultaneously across the temporal dimension (CoT trajectory position) rather than just depth.

2.2 Three probing dimensions

For each hidden state at each step in a CoT trajectory, Tele-Lens trains probes to predict:

2.3 Architecture

A bottleneck low-rank adapter with added nonlinearity is trained on top of each hidden state to produce the three probing outputs. The adapter has far fewer parameters than the LLM itself, reducing overfitting risk. One adapter per task family is trained on held-in data.

2.4 Two model settings

• In-Domain LLM: Qwen2.5-7B-Instruct fine-tuned via GRPO on the target tasks. Chosen because this model lacks a native “thinking” mode, enabling a clean bootstrap of CoT.
• Off-the-shelf LLM: Publicly available models without task-specific training.

2.5 Task suite

12 datasets across three categories:

3. The Core Empirical Finding — Myopic Planning

3.1 Final-answer probability trajectory

For compositional tasks (the hardest and most diagnostic category), the final-answer probability extracted from hidden states follows a characteristic profile:

Most of CoT trajectory:   ~0.49–0.51 (near chance)
Final 1–2 steps:          0.94–0.99 (decisive spike)

The model has essentially no reliable internal prediction of the final answer until the very last steps of the chain, regardless of how long the chain is. This directly contradicts a “latent global planning” hypothesis.

Concrete examples:

• Parity task (is the number of 1s in a binary string odd or even?): The model cannot predict parity until it has counted every bit. The final-answer hidden state signal only appears after the last digit is processed.
• Cycle task (does a graph contain a cycle?): The answer crystallizes only after the model has verbally traced a complete path or detected a cycle step-by-step.

3.2 Reasoning length prediction — no global clock

Early hidden states show no reliable internal estimate of total reasoning length — with an important caveat:

For Parity and Subsum, early states appear to correlate with true reasoning length. But this turns out to be a confound: in those tasks, reasoning length is proportional to input length (a surface shortcut). For Cycle tasks, reasoning length scales with path depth — which is not proportional to input length — and here early states completely fail to predict reasoning length.

This exposes the Parity/Subsum correlation as shallow pattern matching, confirming the absence of genuine global planning.

3.3 Simple tasks — coarse gist, not plan

For semantics/knowledge tasks, early hidden states do carry some predictive signal about the final answer. But the paper characterizes this as pattern-matching gist — recognizing a coarse category from surface features — not step-by-step planning. The signal is imprecise and degrades on unfamiliar inputs.

3.4 What this means for CoT

The findings land squarely on one side of a foundational debate:

CoT = scratchpad computation, not “decoding a latent plan.”

For compositional reasoning tasks, the intermediate steps are not verbalizations of a pre-existing internal plan — they are the computation. Removing them removes the reasoning itself.

4. Practical Applications

4.1 Pivot-based uncertainty estimation

The finding that useful hidden-state signal concentrates in the final 1–2 steps of CoT suggests that a sparse set of “pivot positions” (positions where Tele-Lens signals are most informative) can stand in for full-trajectory uncertainty estimation.

Result: +6% accuracy improvement compared to naive full-trajectory uncertainty estimation.

Comparison: positions selected by standard LM entropy spread across the whole trajectory; positions selected by Tele-Lens cluster near CoT completion — confirming that Tele-Lens adds signal beyond vanilla entropy.

4.2 Automatic CoT bypass

When Tele-Lens signals show high early confidence — the model’s hidden states already encode the final answer clearly from the start — CoT may be unnecessary overhead.

Result: 16.2% bypass rate (proportion of inputs where full CoT is skipped entirely) with negligible performance loss.

This provides an inference-time efficiency win without requiring a separately labeled router.

5. Why It Matters

Three reasons:

1. It resolves the latent-planning debate empirically. The question “are LLMs pre-computing in latent space?” has significant implications for understanding what CoT is doing and how to improve it. Tele-Lens gives a definitive answer for compositional tasks: no. The scratchpad is the computation.
2. It reframes the value of CoT. If CoT were just verbalizing a latent plan, you could in principle extract the plan directly and skip the verbalizations. Tele-Lens shows this is not possible: each CoT step genuinely updates the model’s internal state in a way that previous steps did not accomplish. This validates the investment in long CoT for hard tasks.
3. The uncertainty estimation application has practical value. A method that can reliably identify the handful of positions in a CoT trajectory that carry the most uncertainty information — using learned probes, not heuristics — enables cheaper and better calibration of model confidence, with direct deployment relevance.

6. Limitations Worth Knowing

• Training requirement. Tele-Lens requires a trained probing adapter, which needs task-specific data. It is heavier than zero-shot interpretability approaches.
• In-domain focus. Key quantitative results are on the GRPO- trained Qwen2.5-7B-Instruct model on the curated 12-task suite. Direct generalization to frontier closed-source models (GPT-4, Claude, Gemini) is not confirmed.
• Task scope. The 12 tasks skew toward controlled algorithmic and knowledge-QA settings. Whether the myopic pattern holds for open-ended long-horizon tasks (multi-document synthesis, large codebase reasoning) remains open.
• Correlational, not causal. Probing establishes that hidden states do not encode future planning signals early — but does not establish why, or whether interventions could force global planning.
• Per-domain adapter calibration. The adapter needs domain- specific calibration, limiting plug-and-play deployability across arbitrary tasks.

7. The Takeaway for a First Reader

If you remember three things:

1. LLMs are myopic planners. For compositional tasks, hidden states carry near-chance final-answer predictions (0.49–0.51) throughout CoT, spiking to 0.94–0.99 only in the final 1–2 steps. There is no evidence of global latent planning — CoT is scratchpad computation, and each step genuinely advances the reasoning in a way hidden states alone could not.
2. Tele-Lens probes three signals simultaneously across the CoT trajectory: subsequent tokens, final answer, and reasoning length — using a bottleneck low-rank adapter trained per task family. The reasoning-length result exposes a surface confound in simpler tasks, strengthening the myopia conclusion.
3. Two practical payoffs: a sparse set of pivot positions near CoT completion can represent full-trajectory uncertainty with +6% accuracy improvement, and automatic CoT bypass achieves 16.2% bypass rate with negligible performance loss by detecting early high-confidence hidden states.

References

• Xu, L., Yu, M., Meng, F., & Zhou, J. (2026). How Far Ahead Do LLMs Plan? Uncovering the Latent Horizon in Chain-of-Thought Reasoning. ICML 2026. arXiv:2602.02103.
• Code & data: https://github.com/lxucs/tele-lens
• Pal, A., et al. (2023). Future Lens: Anticipating Subsequent Tokens from a Single Hidden State. ACL 2023.
• Related on this site: MCTD review — inference-time search as an alternative to relying on a single CoT trace.