ChainFlow and Speculative Listening
TL;DR. Two papers widen speculative decoding (a cheap drafter proposes tokens that the big model checks in one parallel pass).
ChainFlow (arXiv 2611.08669, Kurate cs.AI #7) is the first flow-matching drafter conditioned on the target's hidden states. Autoregressive drafters (EAGLE-3) pay per drafted token. Diffusion drafters (DFlash) draft in parallel but predict positions independently. ChainFlow treats drafting as a short transport from the target's hidden state, needing one or two flow steps rather than many denoising steps from noise. A VAE compresses the hidden states so drafter cost barely grows with target size; light feedback heads restore inter-token dependencies; tree drafting raises acceptance. In vLLM on Qwen3.5 at matched drafter size and data: +39% over EAGLE-3 and +11% over DFlash, with the gap widening when drafters are adapted to new languages sequentially.
Speculative Listening (arXiv 2611.06982, HuggingFace 11-07) applies speculation to input in full-duplex speech models that alternate fixed listen and speak chunks. Listen chunks leave the GPU idle. If you assume the next audio chunk is silence and decode ahead, the output is unchanged in 95.3% of chunks, and 88% of changes begin at the first token. So it drafts under silence plus a second branch from the runner-up first token, pools drafts across chunks, and verifies when audio arrives. Training-free and lossless: speak chunks 2.6-2.8x faster, tool-call chunks up to 23.4x, response-start delay down 73% to 74-241 ms (human turn gap is ~200 ms).
flowchart LR
H["Target hidden state<br/><small>last verified token</small>"] --> V["VAE encoder<br/><small>compress, size-independent</small>"]
V --> F["Flow steps<br/><small>1-2 transport steps</small>"]
F --> FB["Feedback heads<br/><small>restore token links</small>"]
FB --> T["Draft tree<br/><small>candidate branches</small>"]
T --> Ve{"Target verify<br/><small>one parallel pass</small>"}
Ve -->|accepted| O["Output<br/><small>+39% vs EAGLE-3</small>"]
Ve -->|next step| H
classDef input fill:#d0ebff,stroke:#1971c2,color:#1b1b1b,stroke-width:2px
classDef core fill:#e5dbff,stroke:#6741d9,color:#1b1b1b,stroke-width:2px
classDef loop fill:#fff3bf,stroke:#f08c00,color:#1b1b1b,stroke-width:2px
classDef exit fill:#d3f9d8,stroke:#2f9e44,color:#1b1b1b,stroke-width:2px
class H input
class V,F,FB,T core
class Ve loop
class O exit
linkStyle 6 stroke:#2f9e44,stroke-width:2px
linkStyle 7 stroke:#f08c00,stroke-width:2px
Relation to prior wiki
- Extends the 11-06 thread (D3D and SpecOPD): D3D, a diffusion drafter that ignores target hidden states, beat DFlash 5.00x vs 4.03x on Qwen3-8B. ChainFlow goes the other way and leans harder on target hidden states. Both beat DFlash. Whether hidden-state conditioning or independence from it wins at scale is now an open, falsifiable question.
- Confirms the SpecPilot result (2611.02501) that released drafters sit far below their attainable speedup.
- Speculative Listening is the first wiki example of speculating on future input rather than future output. It is an idle-compute scheduling idea as much as a decoding one.
Gaps
ChainFlow reports matched-size comparisons in vLLM but not batch-size sweeps against D3D. Speculative Listening is measured on four models and its own TickTock-Bench; noisy real-world audio (not silence) may lower the 95.3% stability.
Concept page: speculative decoding