- llm
- nlp
- reasoning
- rl
- rlhf
- ssm
- theory
- information-theory
- representation-learning
- optimization
- rag
- alignment
- paper-review
- survey
- notes
•
•
•
•
•
•
•
•
•
•
•
•
•
•
-
Reasoning Cache — Continual Improvement Over Long Horizons via Short-Horizon RL
Paper review of Reasoning Cache / RC (Wu, Qu et al., ICLR 2026, CMU) — an iterative decode-and-summarize algorithm that lets LLMs trained on short token budgets extrapolate to reasoning horizons more than 10× longer at test time. A summary replay buffer trains the model to condition effectively on intermediate states. RCT-4B (trained at 16K tokens) reaches ~70% on HMMT-Nov-2025 and surpasses Qwen3-30B-A3B on IMO-AnswerBench at 256K-token test budget.
-
Context-Folding — Scaling Long-Horizon LLM Agents via Branch-and-Fold
Paper review of Context-Folding (Sun, Lu et al., ICML 2026, ByteDance Seed / CMU) — a framework that lets LLM agents actively manage their own working context by branching into sub-trajectories and folding them away once complete. FoldGRPO adds token-level process rewards (Unfolded Token Penalty + Out-of-Scope Penalty) to teach agents when and how to fold. Result: 10× smaller active context, +20% on BrowseComp-Plus, +8.8% on SWE-Bench Verified vs. 327K-token ReAct baselines.
-
T3S — Training-Trajectory-Aware Token Selection for Continual Reasoning Distillation
Paper review of T3S (Shen, Hu et al., ICML 2026) — a token-level fix for "Imitation Shock," the failure mode where continual reasoning distillation causes all metrics to collapse simultaneously despite decreasing loss. T3S masks already-learned Imitation-Anchor Tokens from the AR loss using a trajectory-aware confidence-shift signal, enabling Qwen3-8B to exceed its DeepSeek-R1 teacher on AIME24/25 with under 1,000 training examples.
-
BG-MCTS — Budget-Guided Tree Search for Fixed Token Budgets in LLM Reasoning
Paper review of BG-MCTS (Miyamoto, Oba, Okazaki, ICML 2026, Institute of Science Tokyo) — a training-free MCTS variant that adapts exploration strategy in real time based on how much of the token budget remains. A budget ratio ρ governs exploration annealing, completion bias, and widen annealing, producing a wide-to-deep schedule that outperforms beam search by 2× on MATH500 Level-5 hard problems at a 10K-token budget.
-
Blend-ASC — Optimal Self-Consistency via Power-Law Sample Efficiency
Paper review of Blend-ASC (Feng, Alonso et al., ICML 2026, Yale / Huawei Noah's Ark Lab) — a theoretically grounded adaptive self-consistency method that achieves 6.8× sample efficiency over vanilla SC by deriving the first comprehensive power-law error decay analysis and interpolating between ASC and PPR-1v1 based on sampling progress. Evaluated on MMLU, MATH, and GPQA-Diamond across five LLM families.