DoubtLess: a small model talks a big reasoner out of re-checking

Date: 2026-10-15 · Source: HuggingFace Daily Papers (14 Oct) · Paper: arXiv 2610.18862 · Code Raw: raw/huggingface/2026-10-14-doubtless-training-free-contrastive-decoding-for-efficient-r.md

TL;DR

Reasoning models over-spend tokens, mostly by re-checking answers they already have. Existing fixes either need training per model, add a fixed steering vector, or use blunt heuristics (stop early, ban the word "wait") that cost accuracy. DoubtLess is training-free. A much smaller reasoning model reads the big model's chain of thought (CoT) so far twice: once prompted to be brief, once prompted to be verbose. The difference between those two logit vectors, times a strength, is added to the big model's logits at every CoT step. The small model supplies a "brevity direction" that adapts to the current context. On seven math, science and coding benchmarks it shortens Qwen3-32B's CoT by 31% at unchanged accuracy, beating five inference-time baselines; it cuts thinking tokens per turn by 20% in a self-debugging coding agent; it transfers to two other model families; and it stacks on top of models already trained for brevity. Most savings come after the correct answer first appears in the CoT.

flowchart LR
  C["CoT so far<br/><small>big model's prefix</small>"] --> B["Small LM, brief<br/><small>prompted to be short</small>"]
  C --> V["Small LM, verbose<br/><small>prompted to be long</small>"]
  B --> D["Logit difference<br/><small>times strength</small>"]
  V --> D
  C --> L["Big LM logits<br/><small>Qwen3-32B</small>"]
  L --> M["Steered next token<br/><small>less re-checking</small>"]
  D --> M
  M --> O["31% shorter CoT<br/><small>same accuracy</small>"]
  classDef input fill:#d0ebff,stroke:#1971c2,color:#1b1b1b,stroke-width:2px
  classDef core fill:#e5dbff,stroke:#6741d9,color:#1b1b1b,stroke-width:2px
  classDef loop fill:#fff3bf,stroke:#f08c00,color:#1b1b1b,stroke-width:2px
  classDef exit fill:#d3f9d8,stroke:#2f9e44,color:#1b1b1b,stroke-width:2px
  class C input
  class B,V,D loop
  class L core
  class M,O exit

Key points

• Contrastive decoding, repurposed. Classic contrastive decoding subtracts a weak model's logits to boost what the strong model knows that the weak one does not. DoubtLess contrasts the same small model under two prompts, isolating a style direction (brief minus verbose) rather than a knowledge direction.
• Cost. Two extra small-model forward passes per CoT token. With a small model several times smaller than the target, the 31% shorter CoT is a net saving; the paper does not report wall-clock net of the overhead in the abstract.
• Where the tokens go. Savings concentrate after the answer first appears, i.e. the method mostly cuts verification loops. That matches the 10-14 "Thinking Inertia" finding that models keep thinking when told not to.

Relation to prior wiki pages

• Same mechanism family as LoopCD (10-03), which contrasts a looped model's own loop depths. Two contrastive-decoding steering papers in two weeks; both are training-free.
• Complements the 10-06 shorter-CoT faithfulness result: length pressure lowered faithfulness mostly through inconsistency, but monitorability survived. DoubtLess cuts length at decode time, not by training, so it is a cleaner test of whether shorter traces stay monitorable.
• Contrasts with HyperThink (10-14), which replaces the trace with a query-specific weight update. HyperThink removes the trace; DoubtLess trims it.

Research angle

• Does the brevity direction also shorten traces that matter for safety monitoring? The method suppresses re-checking, which is exactly where a model might reveal doubt.
• The steering model could be a decision model reading "is the answer already in the CoT?" A stop signal is cheaper than per-token steering.

Concept page: Test-time compute allocation