Knowledge Distillation

Transferring capabilities from a large teacher model to a smaller student model. On-policy distillation (OPD) trains the student on its own rollouts under token-level supervision from the teacher.

2026-09-03: the fifth axis is the first one about the objective, and it exposes a blind spot in the other four

IDA-OPD (09-03) (arxiv 2608.29846, BiliBili / UNC Chapel Hill / USTC / SUFE) adds which updates to this page's list of selective-supervision axes, and it is the first axis measured on the student side rather than the teacher side.

The failure it names is diversity distillation failure, and it has a clean signature: in sampled-token on-policy distillation, the student's pass@1 improves while pass@k plateaus. The student learns the teacher's single most likely answer and never inherits the teacher's spread. Everything downstream that samples more than once, best-of-n, self-consistency, verifier reranking, was buying that spread.

The mechanism is the contribution. First-Order Local Entropy Influence is a signed first-order proxy for how one update will move the student's entropy, and it factorizes into two interpretable terms, the teacher-student log-probability gap and the student's local probability structure. Because it is signed, every update can be labelled entropy-expanding or entropy-contracting before it is applied, and entropy collapse traces empirically to identifiable negative-influence positions. IDA-OPD then does the minimal thing: leave expanding updates alone, replace contracting ones with divergence-adaptive advantage shrinkage. It runs entirely off the sampled-token log-probability vanilla OPD already fetches, so it is precise like the teacher-informed methods (AOPD, EOPD, top-K, full-vocabulary Forward KL) and cheap like the student-side ones (entropy bonuses, relaxed heavy-tailed credits). Pass@k improves consistently, pass@1 broadly holds, and it matches the strongest teacher-informed methods at strictly lower cost. The trade-off the two prior families split between them is dissolved rather than chosen.

The retroactive finding matters more than the method, and this page should carry it as a standing caution. Every one of the four earlier axes selects supervision down to the highest-signal subset: which tokens (TIP, 04-16, most teacher-generated tokens carry no learning signal and roughly 10% suffice, through TA-OPD, TrOPD, FiRe-OPD, SG-OPD to R2-OPD (08-25)), which layer (OPRD, 06-05), which trajectories (OPDVR, 08-26), which teacher (QAH, 08-26). Concentrating supervision on the highest-signal positions is precisely the operation that concentrates the student's output distribution. Selectivity and diversity are in tension, and not one paper in the earlier thread reported pass@k. That is a question mark over a year of results on this page, and the cheapest possible resolution: re-run any of them with pass@k logged.

One methodological improvement worth crediting explicitly. This page generalized a warning on 08-26, that R2-OPD's progress-reward model and VoI-MoLE's reducibility estimator are unvalidated learned models with no sensitivity ablation, so "every method in this family depends on a second estimator nobody has validated." First-Order Local Entropy Influence is a closed-form first-order proxy with an interpretable factorization, not a trained network, so it can be reasoned about analytically. Whether the first-order approximation stays faithful late in training is a fair open question; whether it is a black box is not. It is the first estimator in this family that answers the objection by construction.

Same-day convergence on the unit of credit assignment, and it is the sharper cross-paper pattern. Cliff (09-03) (arxiv 2609.02817) uses an off-the-shelf LLM to locate the first mistake in an RLVR rollout, splits the trace into correct prefix and incorrect suffix, and converts that into positive token-level advantages before and negative after, beating on-policy distillation by 15% and standard GRPO by 7% with teachers of only modest capability. IDA-OPD splits by entropy sign; Cliff splits by correctness onset. Two papers on one day, from RLVR reward design and distillation entropy respectively, both converting a per-position diagnosis into signed token-level advantages, and neither cites the other. The field's working unit of credit has moved from the trajectory to the token, with a sign attached.

Cliff also lands directly on this page's oldest open problem. R2-OPD's diagnosis was that teacher-agreement reward is a bad proxy for reasoning progress, because when the student finds a different valid path agreement falls while progress is fine, punishing exactly the independent correct reasoning you wanted. R2-OPD's answer was a second estimator, which this page objected to. Cliff needs no progress estimator at all, and because it does not assume teacher and student reason alike, the different-valid-path failure cannot arise: an alternative correct prefix is just a prefix with no mistake in it. That is the cleanest resolution this thread has had, and it arrives from outside the distillation literature.

New open problem: Cliff is not checked for the failure IDA-OPD discovered. A method that assigns negative advantage to everything after one point is a strong concentrator of the output distribution, which is IDA-OPD's exact diagnosis. Cliff reports no pass@k. The two papers were published the same day and the first one's finding is the second one's missing experiment.

2026-08-26: the axis count goes to four, and the binding constraint turns out to be the supervisor

Three results in one day, and together they say something this page has been circling all year without stating.

The program on this page had two axes. Today it has four. The long selective-supervision thread argued about which tokens to supervise: TIP (04-16, most teacher-generated tokens carry no learning signal and roughly 10% suffice) → TA-OPD (06-01, only reachable teacher corrections) → TrOPD (06-03, only tokens where the teacher is reliable) → FiRe-OPD (06-04, hard-filter trajectories then soft-reweight tokens) → SG-OPD (06-12) → R2-OPD (08-25). OPRD (06-05) changed the venue to which layer, aligning hidden states and bypassing the LM head on the argument that output-space variance was an artifact rather than a fact. Today adds which trajectories (OPDVR) and which teacher (QAH).

OPDVR (arXiv 2608.24696, LeapLab Tsinghua) answers R2-OPD's load-bearing weakness by refusing R2-OPD's whole approach, and the trade is clean enough to state as a rule. R2-OPD's diagnosis was that on-policy distillation's dense reward is derived from teacher agreement and implicitly treated as a proxy for reasoning progress, and the two come apart systematically: when the student finds a different valid reasoning path, teacher agreement falls while real progress is fine, so the student is punished for exactly the independent correct reasoning you wanted. Its fix built a second, cheap progress-reward model and suppressed distillation reward where the two rankings disagree. This page recorded the problem with that immediately, and generalized it: the progress estimator is an unvalidated learned model with no sensitivity ablation, VoI-MoLE (08-25) has the identical structural flaw in its reducibility estimator, and therefore every method in this family depends on a second estimator nobody has validated.

OPDVR attacks the same failure with a signal that needs no validation, because a verifier is not an estimator. It reformulates sampled-token OPD's implicit reward in terms of trajectory correctness, then applies a ReLU gate so correct trajectories receive non-negative reward and incorrect ones non-positive. The student that reasons divergently and correctly is protected by construction rather than by a model's opinion. Two consequences worth keeping. First, the teacher ceiling comes off: a purely distributional objective bounds the student at the teacher because it contains no notion of correctness, and adding verified outcome removes the bound. Second, the reformulation turns sampled-token OPD into a proper RLVR method, so it composes with any policy gradient including GRPO, and it does all of this with zero new hyperparameters — which is the real argument against the weighted-combination and heuristic-switching baselines it replaces. The honest reading of the pair: R2-OPD generalizes further, OPDVR is trustworthy where a verifier exists. Hold both.

QAH (Multiverse Computing) opens the fourth axis and shows the field's default answer had a structural flaw. After a model is structurally compressed and not merely quantized, no independently trained full-precision version of the smaller architecture exists, so quantization-aware distillation's only candidate teacher is the recovered bfloat16 checkpoint — itself a lossy approximation, which silently caps the student at that checkpoint's ceiling. QAH distills from the original pre-compression model instead, teacher and student sharing neither size nor architecture, which works because a teacher's output distribution is architecture-agnostic. GPT-OSS 120B compressed to 60B and quantized to MXFP4 then beats the 60B architecture's own bfloat16 checkpoint on 7 of 9 benchmarks. The reframing is the contribution: quantization stops being lossy postprocessing after healing and becomes a second full distillation pass against the original teacher, so the 4-bit student is not recovering information quantization destroyed, it is acquiring information the bf16 recovery stage never had time to transfer. A stability property falls out of the loss form too — KL to a fixed teacher stops applying pressure once the student catches up, where a cross-entropy task loss (QAT) keeps pushing toward hard labels and destabilizes if over-trained.

The pattern, now at three independent instances and therefore named. OPRD (06-05) found output-space OPD plateaus below the teacher on AIME 2024/2025 and AIMO. OPDVR states the same bound as a structural property of distributional objectives. QAH finds the compression-recovery version. Three subfields, three mechanisms, one diagnosis: in distillation the binding constraint is usually the supervisor, not the student. The three fixes are correspondingly different — change the layer, add a verifier, change the teacher — and none of them touches student capacity.

DiffusionOPSD (arXiv 2608.24646, ByteDance Seed with NUS and others) is the first entry on this page where the teacher is the student. It converts an image-level reward into explicit targets for clean-output predictions at sampled query states: a frozen behavior policy supplies anchors, reward gradients build bounded positive and negative targets around them, the trainable policy fits those as detached supervision, and an EMA update refreshes the behavior policy. 19 of 20 reward-matched settings won across two backbones and ten evaluators, up to 44.0% over the strongest competitor, and training GPU-hours cut 40% on SD 3.5-M and 63% on the step-distilled Z-Image-Turbo. The larger saving on the step-distilled backbone is the internally consistent detail: fewer denoising steps means fewer places to put targets, so a method that makes each one carry more supervision should benefit most there, and it does.

Set DiffusionOPSD against OPDVR and the day has a shape. OPDVR starts from dense distillation supervision and adds sparse verified correctness; DiffusionOPSD starts from a sparse terminal reward and manufactures dense intermediate supervision. Same HuggingFace page, no shared authors, opposite directions, one shared diagnosis: dense-but-uncorrelated and sparse-but-correct are each half a training signal, and the engineering is in the conversion between them. DiffusionOPSD's separation of target construction from finite realization also produces a result worth carrying: larger target-construction gains do not necessarily yield larger realized gains after one fitting update. A better target is not automatically a better step, which end-to-end policy-gradient methods structurally cannot tell you.

One more granularity for the discardable-supervision thread. OraRL (08-26) prunes at the rollout level, keeping only an oracle rollout plus the strongest rollout of each sign, and more than halves step time (2.2x SFT against GRPO-with-CoT's 4.9x). TIP established it for tokens, AgentOPSD (08-07) for turns, R2-OPD for reasoning spans, OraRL now for whole rollouts. Four granularities, same finding: most of the supervision is discardable. OraRL also contributes a named failure mode this page should watch for elsewhere — advantage inversion, where injecting one very high-reward sample into a group-relative computation lifts the baseline enough to flip good rollouts to negative advantage. That is structurally the mirror of R2-OPD's mimicry bias: both are systematic, non-averaging failures of a relative signal, and both are invisible in aggregate reward curves. The rule to carry: when supervision is relative, adding a strong reference can be actively harmful unless the baseline is decoupled from it.

The field split in two, and one half does not know the other exists (2026-08-16)

One week produced five papers repairing gradient-based teacher supervision and four papers demonstrating the gradient can be skipped. No paper in either group cites a paper in the other.

The repair group. Four papers, one diagnosis (08-16): on-policy distillation is unsound whenever the teacher's supervision is computed over information the student did not actually have. HPSE (2608.11660) finds the student's rollouts never cover newly injected knowledge, so on-policy supervision has nothing to attach to, and builds a hybrid rollout that places missing facts on the student's own trajectory precisely where coverage fails. CMD (2608.13391) finds video distillation pipelines supervising a causal student with a bidirectional teacher that scores complete clips, so the teacher's score depends on future frames the student never saw, and replaces it with a causal teacher plus Prefix Scoring against the student's realized cached prefix. REOPD (2608.11698) makes reward extrapolation reliability-adaptive, which is exactly the measurement Privileged-but-Biased (08-10) demanded when it found a privileged teacher's advantage arrives bundled with a bias the student inherits. ReOrder-OPD (2608.10905) finds prompt order shapes what the trajectory covers. SkillLens (2608.10775) adds retrieval. Five instances, five different misalignments, one unwritten general statement: OPD is sound only when the teacher's scoring function is measurable with respect to the student's realized information set.

The skip group, which the section below opened on 08-13 and which has grown to four in four days. AI4AI at Test-Time (08-13), 0.49 → 0.91 on Theory-of-Mind by writing the target a harness. DarwinX (08-14), +17 points average across four benchmarks by evolving the scaffold with the model frozen. AutoPrune (08-16), an LLM designs a visual-token pruning policy through a 131-atom DSL, 94.4% of tokens removed at >99% accuracy retention, training-free. SKILLER (08-16), a strong model acts as actor and critic in a reinforcement loop where every signal is natural language, producing executor-specific skills that lift Qwen3.5-9B by 4.3 to 20.4 points and match closed models on single-skill tasks.

The comparison nobody has run is now three months old and getting more expensive to keep not running. The repair group is spending real research effort making a gradient-based transfer mechanism reliable. The skip group is demonstrating the transfer can happen through a program at a fraction of the training cost, and reporting no cost figures at all for the strong-model loop that produces the program. Both halves are incomplete in exactly complementary ways. The 08-13 and 08-14 Looking Ahead bullets asked for dollars per point of gain on one axis for both. Still unrun.

One thread does connect them. SKILLER's gains are larger for the 9B model than the 4B (up to 20.4 versus 13.3 points), while AI4AI at Test-Time reported that weaker targets receive the largest gains. Those point in opposite directions within a narrow capability band, and the Extrapolation Cliff (05-14), which found a closed-form threshold above which gradient-based OPD collapses, says the gradient path has a ceiling somewhere in the same region. Whether the program path has a floor where the target is too weak to execute a written policy is now a cheap, concrete experiment.

And a third position exists that neither group occupies, arriving from production rather than from arXiv. Applied Compute's hinting framework (08-16), described by Samuel Denton in a conference talk, keeps the gradient but removes the reference solution. His constraint is explicit and it is the one thing no paper on this page imposes: no golden answer, no reference solution, no hand-written per-task rubric. The teacher's only advantage is a hint, a piece of privileged information the student did not get, and the hint encodes a direction rather than a target. You roll out conditioned on the hint and pull the unhinted policy toward its hinted self.

That is a structural answer to Privileged, but Biased (08-10) rather than another filter on top of it. That paper's causal chain begins with the teacher having seen one particular reference solution, which drags its per-token target toward that trajectory and lands the loss mass on stopwords, punctuation, and uncertainty markers. If there is no reference trajectory, there is nothing for the target to collapse toward. Every one of the nine filtering axes catalogued below accepts a reference-derived target and then decides how much of it to trust; hinting declines the target.

Three further contributions, and the third is the one this page has been asking for since 06-18:

• Relevance masking is an independent rediscovery of the same failure, with a mitigation. Denton uses an LLM judge to select which teacher tokens contribute to the loss, because otherwise you inherit the teacher's connector-word preferences and pay for it in catastrophic degradation. Same diagnosis as Privileged, but Biased, derived from customer deployments, six days apart, no mutual citation.
• Per-step injection is TurnSight (08-05)'s and SMRC-SD (08-10)'s claim with a cheaper estimator. Do not inject the hint at the start of a rollout; use a judge to pick the step, then distil only on the next step or few. Three independent arrivals at supervise at the state the student actually reached.
• The head-to-head. On a customer's out-of-distribution hyperlink-formatting task, reward shaping and SFT on correctly-formatted traces both degraded general coding performance, while online hinting took correct formatting from about 15% to about 80%; a fixed offline hint applied to every rollout barely moved it. This page's standing complaint is nine axes and zero comparisons. This is a comparison. It is n=1, from a vendor, with no regression magnitudes published, so it is evidence rather than proof, but it is the first time the alternatives were run against each other on one task.

One mechanism detail worth keeping. In his SWE-bench experiment, the rollout was conditioned on an off-policy production trace that never called the submit tool, so the teacher could not force the tool-call tokens and instead shifted the reasoning trajectory leading to them: task-complete rate 22% to 60% with test-pass rate flat. The behaviour was installed without ever supervising the tokens that constitute it, which is a stronger version of the selective-supervision thesis than any paper here states: sometimes the right token set to supervise is not the one the behaviour is made of.

The transfer medium stopped being a gradient (2026-08-13)

Every method on this page assumes the artifact carrying capability from teacher to student is a parameter update. AI4AI at Test-Time (2608.12307) proposes that the artifact can be a program instead, and measures it. A strong builder model writes an inference-time harness for a weaker target model, iteratively refining it against a 5% validation split; the harness is then frozen and run on the full test set. Average target performance across four Theory-of-Mind benchmarks goes 0.49 → 0.91. The target's weights are never touched.

Three details make this more than a prompting result. Builder reasoning effort improves harness quality monotonically, so this is a compute-allocation claim: the budget moves from training the student to thinking about the scaffold, and it pays off on a smooth curve. Weaker targets receive the largest gains, which is the same direction the Extrapolation Cliff (05-14) runs in reverse: where gradient-based OPD collapses above a capability threshold, harness transfer helps most below one. And the mechanism analysis rules out the obvious confound: gains do not come from the target reasoning more or sampling more widely, but from offloading unstable reasoning into deterministic code, per-question-type routing, and strict answer-format enforcement. The harness does not teach the target to think better; it removes the places the target's thinking was allowed to leak.

What this page should carry forward. The selective-supervision thread's nine axes all answer "which parts of the teacher's signal are worth a gradient." AI4AI asks whether a gradient is the right vehicle at all, and the two lines have never been compared on a common axis. The comparison is cheap and nobody has run it: for a fixed dollar budget, how much target accuracy does builder inference buy versus fine-tuning, and where do the curves cross? AI4AI establishes the substitution exists and reports no cost accounting whatsoever, not for the builder's refinement rounds, not for per-inference harness overhead, not against a fine-tuning baseline. That omission is this page's standing complaint applied to a new branch: nine axes with zero head-to-head comparisons, and now a tenth vehicle with no price either.

Open question: does harness transfer compose with weight transfer or compete for the same headroom? Directly testable by running an AI4AI harness on top of an already-distilled target. If the gains stack, the two are orthogonal levers and every OPD result on this page is understating its achievable ceiling. If they do not, both are recovering the same missing structure by different means.

And a scheduling axis arrived from Kurate. ReOrder-OPD (2608.10905, Kurate cs.LG #19, tier 1, absent from HuggingFace) proposes reliability-aware prompt ordering for on-policy distillation. Every axis above filters or reweights what the teacher supervises; this one changes the order the prompts arrive in, which is a curriculum claim rather than a filtering claim and the first in the cluster to touch scheduling. It rhymes with From Sweep to Seam (08-12), which found that in post-training quantization the schedule rather than the quantizer separates a usable 1.58-bit model from a broken one. Two papers in two days, in distillation and quantization, both finding order-of-local-operations is the underexploited lever, neither aware of the other. Kurate's tournament had not run at scrape time (all 40 entries at the 1200 baseline, 0% win rate, third consecutive stale week), so this is a topic selection, not a rank endorsement.

Current State (as of 2026-08-11)

The ninth axis arrived within 24 hours of this page predicting it, and it is the first one whose decision variable is an outcome under intervention rather than a statistic of the current step.

The prediction, and its resolution. The 08-10 Looking Ahead said a ninth filtering axis would confirm this page's standing read that the cluster generates variants rather than comparisons. Not Every Divergence Should Be Suppressed (08-11) (2608.04408, Kurate cs.LG #15, tier 1, absent from HuggingFace, Tsinghua) is that axis. Its objection is that divergence is the wrong decision variable, because a large teacher-student disagreement says only that they prefer different actions, not whether the student's error can still be fixed. The replacement is counterfactual recoverability: under a fixed remaining budget, is the teacher more likely to finish the task if it continues from the error, or if the error is rolled back and the branch resampled? Errors where continuing still works are recoverable, so their divergence carries signal. Errors where only rollback works are irreversible, and suppressing divergence there actively reinforces a doomed trajectory, which is a two-sided harm the eight prior axes cannot express.

Why it is more than the ninth variant. This page named the load-bearing split as six-or-seven methods that accept the teacher's target and decide how much to trust it, versus the two that rebuild it from evidence about whether a candidate works (VAD (08-04) decomposes, SPOT reconstructs). Counterfactual recoverability is on the rebuild side, which matters because Privileged, but Biased (08-10) indicts the accept-and-reweight family far more directly. The rebuild branch now has three members and is the only part of the cluster the falsifier does not obviously reach.

It is also the closest thing yet to the unified reliability estimator this page has flagged as missing since 06-18. TA-OPD's teachability, TrOPD's trust region, SG-OPD's sign-consistency and Quality-Aware OPSD's can-this-prefix-still-reach-the-answer gate are four task-specific estimators of one latent quantity, and the coordinate-task version was decidable only because GUI grounding reduces it to ground-truth-box membership. Counterfactual recoverability answers the same question by rollout instead of by task structure, which is what makes it general and also what makes it expensive. It is the explicit computation of what Relay-OPD (07-29) approximates for free, since Relay-OPD's label-free trigger (on failed prefixes the teacher redirects while the student ploughs on) asks "do they diverge on continuation?" where this asks "does continuation still succeed?" Same object, one inferred and one measured, no mutual citation. The page's standing complaint applies with full force: nine axes, still zero head-to-head comparisons, and still nobody prices the teacher calls spent on selection against the teacher calls selection saves.

Separately, a five-day-old prediction on this page resolved in the other direction: SPOT crossed from Kurate-only to cross-source confirmed. The 08-06 entry logged SPOT (2608.04419) as one of two papers that came off the Kurate boards (cs.LG #16, tier 1) without HuggingFace attention, and flagged it as LLM-rated underrated. Today it is on the HuggingFace Daily Papers board and has risen to Kurate cs.LG #8, making it the day's only cross-source-confirmed paper. Nothing about the method changed; the popularity signal took five days to catch up to the quality signal. For a wiki that treats Kurate as the quality channel and HuggingFace as the popularity channel, this is the cleanest measurement yet of the lag between them, and five days is short enough that the Kurate board is worth reading as a leading indicator rather than as a supplement.

One method detail worth adding to SPOT's entry, from the alphaxiv overview. SPOT positions itself specifically against Entropy-Aware OPD (EOPD), which augments reverse-KL training with a top-k approximation of forward KL at high-teacher-entropy positions to preserve plausible local alternatives. SPOT's argument is that teacher entropy is an insufficient statistic on three counts: it cannot distinguish uncertainty concentrated among a few viable candidates from uncertainty dispersed over a long tail, it says nothing about whether the student already represents those candidates, and local teacher probabilities may simply not predict downstream success. That is why the acquisition score combines normalized teacher entropy, the mass captured by a small top-k set, and student-teacher mismatch rather than entropy alone. It also sharpens the reverse-KL critique this page has carried loosely: standard OPD is mode-seeking, favouring the teacher's most probable continuation and starving other correct continuations, which is a coverage failure distinct from the three collapse modes already catalogued here (the Extrapolation Cliff capability threshold, ReOPD's prefix trap, and VAD/CRPO's source contamination).

And the production form of multi-teacher distillation shipped in a technical report. Motif 3 (08-11) (2608.09119) post-trains a 314B-total / 13.2B-active MoE by training six RL specialist teachers plus one SFT software-engineering teacher and folding them into one model with Multi-teacher On-Policy Distillation. All teachers are in-house, which is the pattern this page predicted persists: dense OPD supervises over a shared vocabulary, so a shared tokenizer is required unless you use BPM (07-29)'s byte-prefix marginalization. Motif 3 does not need BPM precisely because it owns its teachers. It also does not ablate MOPD against simply having run seven RL jobs, so whether MOPD consolidates capabilities or averages them remains the open question of the multi-teacher line.

A training-free branch opened on the agent side, and it belongs here rather than on the memory page. AMD (08-11) (2608.07169, KAIST) transfers a large teacher agent's competence to 4B-8B students with no gradient update at all, by factoring successful teacher trajectories into Workflow (task strategy), Subtask (concrete intermediate behaviors) and Function (calling conventions and pitfalls) memories, injecting the first two proactively and retrieving the third reactively on tool-call errors. +27.2pp AppWorld, +11.2pp BFCL V3, +3.4pp ToolSandbox, with Subtask memory contributing most. Read against MAPD (08-02), which compiled a proprietary teacher into a JSON task-plan-facts protocol shown to a privileged student branch, AMD is the same semantic schema as neutral exchange channel move with the distillation step removed: the schema is served at inference instead of being distilled into weights. That makes it the cheapest rung on the neutral-channel ladder this page has tracked from bytes through pixels to editable prose rules, and it reports the capability-gap effect from the memory side, since 4B students benefit most and effectiveness depends on both teacher capability and student compatibility, which is the Extrapolation Cliff (05-14) shape without the gradient.

Prior State (as of 2026-08-10)

The cluster finally got its falsifier, and it does not compare the variants. It attacks the premise underneath all of them.

The result. Privileged, but Biased (08-10) (2608.04794, Kurate cs.AI #18, tier 1, absent from HuggingFace, from Microsoft Research) reproduces SDPO's reported gains in SDPO's easy setting, then applies the identical setup to hard tasks and finds nothing is learned. Across question answering, mathematics, coding and multi-turn agentic tool use, across reasoning modes, model sizes, and forms of privileged information, under both the SDPO and OPSD recipes, the per-token loss falls steadily while validation accuracy stays flat or degrades. The causal chain has four links and each is separately checkable. (1) PI bias: having seen one particular reference solution, the teacher's per-token target is pulled toward that trajectory rather than toward correctness in general, quantified by a new PI Bias Score. (2) Trained to match that target everywhere, the student's objective becomes nearly blind to whether a rollout is correct. (3) The loss it assigns falls mostly on low-information tokens: stopwords, punctuation, uncertainty markers. (4) Within correct rollouts, the exploratory tokens incur the highest divergence, so the objective penalizes exactly the hesitation that reasoning requires. Output: a flatter, less decisive student.

What this does to the seven-axis taxonomy below. Every one of the seven filters exists because the privileged teacher's signal is unevenly trustworthy. This paper says the untrustworthiness has a single named cause and that as a lone objective the family optimizes a signal decoupled from task success. Read strictly, the filters are patching a target that should not be matched. The page's own six-versus-two split is now the load-bearing distinction: six methods accept the teacher's target and decide how much to trust it, while VAD (08-04) decomposes it and SPOT (08-06) rebuilds it from evidence about whether a candidate works. This page predicted that "reweighting a wrong target converges to a wrong place more slowly, while reconstructing it converges somewhere else." The two reconstruction methods are the only ones this result does not obviously indict.

One paper it directly contradicts, and the contradiction is about what to do rather than what is true. SA-OPD (08-06) found that a teacher's extreme-divergence tokens are frequently driven by language priors, formatting conventions and stereotyped templates rather than by the input, and filtered them out. Privileged, but Biased makes the same observation about where the loss mass sits and draws the opposite conclusion: that is not noise at the margin, that is where the objective's mass actually is, so filtering it leaves you optimizing a smaller amount of the same decoupled signal. Same measurement, opposite prescription, four days apart, no mutual citation.

The scope limit that matters, and it is a real one. The failure is demonstrated for self-distillation as a lone objective, with no reward term. That is not the configuration most of this cluster actually ships. RSTG (08-06) distils only on negative zero-variance prompts inside GRPO, precisely the prompts where the whole group failed and the reward gradient vanishes, and reports +4.02% math and +3.05% code. That configuration is untested here and is not obviously covered by the argument, because RSTG uses distillation to fill in where the verifiable reward is silent rather than to replace it. The clean experiment is now specifiable: measure the PI Bias Score inside RSTG's negative-group support. If bias is low there, the whole cluster's correct form is "distil only where the reward signal is dead," and the seven filtering axes were solving the wrong problem.

The eighth axis lands the same day, from the state side, and it is the one TurnSight asked for. SMRC-SD (08-10) (2608.05219, HuggingFace) names state-reference mismatch: in an interactive environment the student's own actions keep changing the execution state, so a globally correct reference trajectory is only locally valid at the pre-action states it was recorded from. SMRC-SD verifies per turn whether the student's current state is supported on the reference, distils only at matched states, and at those states builds teacher context conditioned on the state actually reached. ALFWorld 0.746 to 0.865, WebShop 0.574 to 0.693, at Qwen3-1.7B. This is TurnSight (08-05)'s dissent implemented: TurnSight argued the standard privileged context is the wrong context because it derives from the ground-truth answer rather than from the state the agent reached, and this page called that "the most important claim in the cluster." Five days later it has a method and the method works.

And here is the tension the cluster now has to resolve. SMRC-SD fixes a sibling of PI bias, mismatch between reference state and student state, but not PI bias itself: at matched states it still derives its target from one particular reference solution. Either state matching incidentally defuses PI bias, which would be a strong and testable claim nobody has made, or SMRC-SD's ALFWorld and WebShop gains sit inside the same easy-setting regime where Privileged, but Biased also reproduces gains. Two papers surfacing on the same day, one saying the mechanism does not work on hard tasks and one reporting +12 points, and nothing in either can distinguish the two explanations. That is the single most important open question on this page.

Author-tracking note, to prevent a future merge error. Today's Kurate rising-author board names Junlin Liu at score 17.0 with three top-10 appearances, including "Contrastive Reinforced Policy Optimization via Privileged Self-Distillation," which is the CRPO line this page has tracked since 08-04 and attributed to overlapping Meituan author groups. SMRC-SD's first author is Junzhuo Liu at the University of Electronic Science and Technology of China. Different people, near-identical names, same problem. Do not merge them.

Prior State (as of 2026-08-06)

Seven on-policy distillation papers in one day, and the cluster's central question now has seven mutually untested answers. The page should stop counting instances and start naming what would settle it.

The count, because it matters. Five landed on HuggingFace: SA-OPD (2608.03632, cross-source confirmed on Kurate cs.AI #11), RSTG (2608.00782), Poly-OPD (2608.04349), OPD-V (2608.05131), and SKILL-KD (2607.28048). Two came off the Kurate boards without HuggingFace attention: SPOT (2608.04419, cs.LG #16, tier 1) and CRPO's re-ranking at cs.LG #3. On-policy distillation is now producing roughly one paper per day per filtering axis, and the 08-05 Looking Ahead prediction that the cluster would either produce a head-to-head comparison or reveal itself as generating variants is resolving toward variants.

The seven axes, in the order they appeared. Every one accepts that dense teacher supervision is unevenly trustworthy and differs entirely in what it conditions trustworthiness on. Position: CRPO (08-04) sorts by predictive entropy, on the finding that a privileged self-teacher spikes into overconfidence right after a tool call returns. Direction: VAD (08-04) projects the correction onto a signed counterfactual visual-evidence axis and discards the unexplained residual. Time: PCSD (08-05) weights by how persistently teacher-favouring signal holds across an adaptive window. Turn structure: TurnSight (08-05) keeps only what multiple lookahead horizons agree on. Input-groundedness: SA-OPD filters tokens that are simultaneously weakly input-dependent and extreme in divergence, on the claim that a teacher's token judgment is often driven by language priors, formatting conventions, or stereotyped reasoning templates rather than by the input. Modality balance: OPD-V builds a Positive Teacher (zoom-in crop) and a Negative Teacher (masked image), reads the logit margin between them, and distils only inside the resulting trust region. Realized outcome: SPOT probes a budgeted set of positions, has the teacher propose candidates, runs verifier-scored student continuations on them, and builds a closed-form KL-regularized target from the outcomes.

The one distinction that actually separates them, and it is not the axis. Six of the seven accept the teacher's target and decide how much to trust it. Two do something structurally different. VAD decomposes the target and throws part of it away. SPOT rebuilds the target from evidence about whether the candidate works. That is the split worth tracking, because reweighting a wrong target converges to a wrong place more slowly, while reconstructing it converges somewhere else. The page previously flagged VAD as the first paper to change what the target is rather than how much it counts; SPOT is the second and goes further.

SA-OPD generalises VAD's mechanism out of vision, which the page predicted implicitly and should now state. The page derived a principle from TurnSight: a privileged signal is trustworthy to the extent that it survives perturbation of the privilege. VAD perturbs an image crop. OPD-V perturbs it in both directions at once. SA-OPD perturbs input dependence in general, which is the version that runs on pure text, and it is the first paper on this beat to make perturbation-survival its central mechanism for the general LLM case. Three of today's seven are instances of one principle nobody has written down as a paper.

RSTG partially resolves the CRPO-versus-ReCo conflict, and it resolves it exactly the way this page guessed. The page hypothesised that ReCo protects exploration coverage against a base-model prior while CRPO protects supervision reliability against a confidently wrong teacher, so "they select on different supports." RSTG makes the support separation structural: distil only on negative zero-variance prompts, the ones where GRPO's group all failed and the gradient vanishes entirely, because on those prompts there is no exploration left to crowd out. Inside that support it targets high student entropy or large teacher-student divergence, which is the ReCo side of the same statistic, and it adds SFT on teacher-generated correct trajectories to supply positive gradient where OPD's asymmetric advantage supplies mostly suppression. +4.02% math, +3.05% code over naive GRPO+OPD. The 08-04 Looking Ahead asked for a third paper on this conflict within 60 days and got one in two, though not the paper it asked for: RSTG separates the supports instead of reporting a coverage metric and a reliability metric on the same runs, so the question of what high entropy means is sidestepped rather than answered. The clean test is now obvious: run CRPO's entropy sort inside RSTG's negative-group support. If it still helps, the support explanation is wrong.

RSTG's three diagnosed causes are the most reusable thing published on this beat today. Naive GRPO+OPD degrades because (1) teacher guidance quality varies per sample, (2) fitting the teacher fast forecloses RL's only upside, which is exceeding the teacher, and (3) OPD's token-level advantages are asymmetric, since student-generated tokens usually get low teacher probability, so the signal suppresses rather than guides. Cause 3 is a general property of OPD-on-RL, not a quirk of this setup, and it belongs on this page independent of RSTG's method.

The neutral-exchange channel reaches its tenth entry and the ladder now has a human-readable top rung. Any-OPD (08-05) took the channel fully external to both parties, comparing independently decoded outputs in a frozen model-agnostic vision representation. Poly-OPD, one day later, shows that once the channel is external, teacher count stops being architecturally constrained: two heterogeneous teachers (FLUX.1-dev and Z-Image) into one 2.5B SD3.5-Medium student via a pixel bridge plus frozen DINOv2 space, lifting GenEval 67.3 to 73.3 above both larger teachers and DrawBench HPSv3 9.34 to 11.35. And SKILL-KD moves the channel to editable natural-language rules, distilling the actionable discrepancy between a student failure and a teacher success into a textual skill patch, verifying the patch by re-running the student, and consolidating patches with trace-linked edit histories to prevent skill drift. Bytes, probability spaces, JSON schemas, pixels, now prose rules. The page's standing caution holds: there is no unique lowest common substrate for reasoning, so each rung above bytes is a bet, and none of them ablates how much competence fails to fit.

Poly-OPD's gradient-compatibility diagnostic got an independent witness on the same day from an unrelated problem, and this is a genuine cross-paper finding. Poly-OPD measures gradient agreement across teachers and concludes attention LoRA modules can be shared while feed-forward adapters must stay teacher-specific. Physics of Multimodal Pretraining (08-06) ablates architectures across modalities and concludes that shared attention and normalization with modality-specific feed-forward layers is what turns competition into synergy, holding across visual tokenizer designs, validated on 13.5B MoE models over 2T tokens. Same partition, two methods, two problems, one day, no mutual citation. If the principle is real it says attention is a source-agnostic mixing operation and the feed-forward layer is where source-specific knowledge lives, which would predict that every multi-source training method on this page should be splitting FFNs and sharing attention.

Vision laziness connects OPD-V's post-training finding to a pretraining cause. OPD-V's diagnosis is modality imbalance: when text dominates generation, the model does not integrate visual input, so privileged visual information goes unused regardless of how well it was constructed. Physics of Multimodal Pretraining names vision laziness: delaying modality integration to a late alignment stage teaches the model to lean on language priors. One paper locates the cause in the pretraining schedule and the other measures the consequence in post-training, on the same day, neither aware of the other. The actionable question: does a model pretrained with early unification need OPD-V's trust region at all?

Where the expensive signal is allowed to fire, now on four axes. TIP (04-16) restricted it on the token axis (under 10% of teacher tokens carry signal). Relay-OPD (07-29) on the trajectory axis (training length halved). ReOPD (08-03) on the environment axis (zero tool calls during student training, at least 4x faster rollouts). RSTG adds the prompt axis: most prompts get no teacher call at all. Nobody in the four has priced the teacher-call saving, which is the first number a practitioner pairing GRPO with a frontier-model teacher would ask for.

Industry contradiction worth logging, because it is about the word rather than the technique. ByteDance's founder Zhang Yiming told an internal all-hands that the company will not use distillation as a shortcut, even if that means lagging domestic rivals, with the TikTok history cited as a reason. On the same day, ByteDance authors are on SA-OPD, the day's cross-source-confirmed leading on-policy distillation paper. Those are not in conflict, and the reason they are not is the confusion this page has tracked since the Distillation Panic (05-04) entry: "distillation" names two different things. Copying a rival frontier model's outputs to shortcut capability is an IP and provenance question. Compressing a teacher you own into a smaller student you serve is an efficiency technique. The policy fight, and now a founder's strategy statement, are using the word in the first sense while the entire technical literature on this page uses it in the second. Anyone reading either as evidence about the other is making a category error.

Prior State (as of 2026-08-05)

Seven privileged-teacher papers in four days. The pattern is no longer the finding; the internal disagreement is. Four groups now filter the same privileged signal along four different axes, and one of them argues the privileged context itself has been built wrong all along.

The four axes. Every paper in this cluster accepts that a privileged teacher, meaning the same policy handed information the deployed model will not have, produces dense but unevenly trustworthy supervision. They differ entirely in what they condition trustworthiness on. CRPO (08-04) filters by position, sorting by predictive entropy. VAD (08-04) filters by direction, projecting the correction onto a counterfactual evidence axis. PCSD (08-05) filters by time, deriving weights from the local persistence of teacher-favoring signal across an adaptive window, on the claim that teacher reliability is autocorrelated rather than per-token independent. TurnSight (08-05) filters by turn structure, building multiple hindsight views at different lookahead horizons and keeping only what they agree on the direction of. None of the four evaluates against any of the others.

The dissent, and it is the most important claim in the cluster. TurnSight argues the standard privileged context is the wrong context. Existing methods derive it from the ground-truth answer or from a retrieved skill library, and neither describes the state the agent actually reached, so the teacher's confidence is about the answer rather than about the agent's situation. TurnSight instead conditions on realized execution. If that is right it partially undercuts the other six rather than complementing them, and nobody has said so. It also generalizes VAD's mechanism into a principle: a privileged signal is trustworthy to the extent that it survives perturbation of the privilege, whether that perturbation is removing an image crop (VAD) or extending a lookahead horizon (TurnSight).

PCSD may subsume CRPO. Both come from overlapping Meituan-and-Junlin-Liu author groups. CRPO's finding, that the teacher spikes into overconfidence right after a tool call returns, describes an isolated, non-persistent signal, which is exactly what PCSD's persistence window down-weights by construction. PCSD is plausibly the more general mechanism with CRPO's entropy heuristic as a special case in the tool-call setting. Testable, untested.

It also reframes the unresolved CRPO-versus-ReCo tension. ReCo (Kurate cs.LG #19) upweights the high-uncertainty positions CRPO discards, because one is protecting exploration coverage and the other supervision reliability. If reliability is a property of a run of positions rather than a position, as PCSD argues, those two instructions stop being contradictory because they select on different supports. Still nobody has reported a coverage metric and a reliability metric on the same agentic runs.

Separately, the neutral-exchange-channel pattern reaches its eighth entry and its logical endpoint. Any-OPD (08-05) does on-policy distillation between arbitrary latent flow-matching model families that "share nothing but pixels," treating the teacher as a pure black-box sampler and connecting the two models at exactly one point, a frozen model-agnostic vision representation in which their independently decoded outputs are compared. Trajectory correspondence comes from matching continuous noise levels rather than step indices, which is a small correction with broad reach, since every schedule-based distillation method on this page aligns by step. FLUX.1-dev 12B into SD3.5-Medium 2.5B lifts PickScore 0.846 to 0.884 and HPSv3 9.12 to 10.97, where direct latent regression fails to train at all. After BLD (bytes), TESSY (interleaving), Switch-KD (shared text-probability space), Tide (bounded-gradient cross-arch), CoPD (bidirectional OPD), D-OPSD (conditioning asymmetry), the channel is now entirely external to both parties. It also has a policy edge: output-only cross-family distillation working undercuts any regulation premised on distillation requiring teacher internals, which is the technical core of the Distillation Panic (05-04) argument.

Prior State (as of 2026-08-04)

The privileged-branch pattern hits its third and fourth instance in three days, so it is now named. And two papers on the same day report the same failure inside it: a privileged teacher is wrong in a way that correlates with where the student needs help most.

The pattern. MAPD (08-02) showed a JSON reasoning protocol only to a privileged branch of the student, whose token distributions then supervised a deployed branch that never reads the protocol. CriPO (08-03) used a criterion-injection self-teacher, the same policy under a privileged prompt, to repair a suppressed gradient. Today adds VAD (08-04), whose teacher is privileged with a visual crop, and CRPO (08-04), whose self-teacher is privileged with agent-task information. Four instances, and the distinguishing question is what the privileged branch is privileged about: a task schema, a rubric criterion, a visual region, agent state. Call the pattern privileged-branch target construction. It is now the dominant way this page's papers build a supervision signal.

The shared failure, stated two ways on one day. CRPO (2607.28026, Kurate cs.LG #2) diagnoses that because the self-teacher holds privileged information it becomes overconfident exactly at the positions where the student is genuinely uncertain, which in agent tasks is right after a tool call returns new information. Two harms follow: the teacher's reasoning routes converge onto patterns in the demonstrations so the student generalizes worse, and multi-turn optimization directions become unclear because position-level supervision is unreliable. The fix sorts positions by predictive entropy into positive positions where high entropy reflects genuine reflective exploration and negative positions where it reflects exposure bias, then contrasts them group-wise so only reliable fine-grained signal survives. Gains across 13 reasoning and deep-search benchmarks over both RL and self-distillation baselines. VAD (2607.28590, top of today's HuggingFace board at 18 upvotes) says the multimodal version: a teacher correction is source-mixed, bundling visual signal with linguistic priors and teacher-specific effects, and the paper's diagnostic reports that a substantial share of a teacher's strongest corrections are not well aligned with its own evidence-conditioned response. That invalidates the premise of Vision-OPD, which conditions the teacher on an evidence crop and then distils the complete next-token distribution on the assumption that visual conditioning makes the correction visual.

These two are the same claim about privileged teachers from opposite directions, and together they add a third collapse mode to this page. The Extrapolation Cliff (05-14) is a capability-gap threshold above which on-policy distillation collapses because the student is too far below. The prefix trap from ReOPD (08-03) is the multi-turn fact that student occupancy and teacher reliability move in opposite directions. The new one is source contamination: the teacher is competent, on-distribution, and its target still carries signal from a channel you never meant to transfer. Three modes, three causes, and only the third has a per-position detector, which is why CRPO's entropy sort and VAD's counterfactual projection are the interesting machinery rather than the benchmark numbers.

VAD's mechanism, because it is the sharper of the two. Run the same fixed teacher twice at each student prefix, once with the relevant evidence present and once with it removed. The change in centered log-probabilities defines a signed direction u_t along the visual evidence. Project the original correction onto u_t, keep the aligned component, discard the unexplained residual, and rebuild a student-anchored target from the aligned part, demoting the privileged teacher to a weak regularizer. The sign is what separates it from the visual-advantage line (VA-OPD, V-Zero), whose positive advantage captures "the evidence supports this alternative" and structurally misses "the evidence refutes what the student just said", and refutation is exactly where VAD's gains concentrate. Six fine-grained visual benchmarks at 4B and 9B.

One clarification on the sign. The visual-advantage line uses a positive advantage, which captures "the evidence supports this alternative" and structurally misses "the evidence refutes what the student just said." VAD's signed direction captures both, which is why its gains land on refutation cases.

This is a fifth mechanism in the token-weighting thread and the first to change what the target is rather than how much it counts. TIP (04-16) found under 10% of teacher tokens carry signal, LongAct (04-18) weighted by activation magnitude, Relay-OPD (07-29) by continuation divergence, CoRT (07-30) by counterfactual likelihood for +4.4 points. All four reweight a target they accept. VAD decomposes the target and discards part of it, which is a different operation: reweighting says this token matters less, decomposition says this token's target is partly wrong. CoRT is the closest sibling and the contrast is clean, since both are counterfactual but CoRT's counterfactual is over token replay to assign credit while VAD's is over evidence presence to attribute direction. They compose trivially and nobody has composed them.

One safety-adjacent note. ROPD (08-04) (2607.27081, Kurate cs.AI #18, tier 1) uses on-policy distillation for safety realignment after malicious fine-tuning, modelling the divergence between aligned and compromised output distributions rather than fitting a specific prompt template, which is what buys robustness when the defender cannot observe the attacker's template. It belongs on the responsible-ai page but it is worth logging here as the first use on this page of distillation to remove a capability rather than transfer one, and the reported result that existing defences cause catastrophic forgetting of the specialized skill is the same tradeoff-under-a-privileged-signal shape as everything above.

Three additions to the above. First, CRPO's cost argument, which is the reason to prefer it over the existing fixes. RLSD, SDAR and RLCSD all address exposure bias by bolting an RLVR objective onto the self-distillation objective, and pay for maintaining two optimization frameworks. CRPO reformulates OPSD contrastively instead and adds no second framework, though note the no-extra-cost claim is made against those hybrids rather than against plain OPSD, which is the baseline that matters.

Second, CRPO and ReOPD stack cleanly and nobody has stacked them. ReOPD (08-03)'s step-decaying schedule chooses which prefixes to train on; CRPO's entropy contrast chooses which positions inside a prefix to trust. Both are reweightings over the same rollout data, so composition costs nothing beyond bookkeeping. Same for VAD and CoRT (07-30), already noted above as trivially composable.

Third, and this is a genuine unresolved conflict rather than a gap: ReCo (Kurate cs.LG #19, 2607.26862) prescribes the opposite of CRPO on the same statistic. ReCo attacks GRPO for concentrating on responses the base model already generates with high probability, traces it to two update terms (high-probability responses dominating the group gradient by repetition, and the importance ratio scaling gradients toward already-likely tokens), and fixes it by normalizing response contributions by expected occurrence and replacing the token-level importance ratio with a variance-based ratio that upweights non-saturated decision points where alternative tokens remain plausible. Those are the high-entropy positions CRPO contrasts away. The likely reconciliation is that ReCo protects exploration coverage against a base-model prior while CRPO protects supervision reliability against a confidently wrong teacher, in which case an agentic post-training recipe needs both signals plus a way to tell the two kinds of uncertainty apart, which nobody has proposed. If that is not the reconciliation, one of them is wrong about what high entropy means, and the variable that would settle it is whether CRPO's filtered positions show the Pass@k-at-large-k degradation ReCo measures.

Current State (as of 2026-08-03)

A second, distinct collapse mode for on-policy distillation, and this page should stop bundling it with the first. ReOPD (08-03) (2607.04763, Microsoft Research and University of Amsterdam) studies multi-turn agentic OPD and names the prefix trap. Pushing interaction histories toward the student's own distribution makes them more relevant to the student and simultaneously drags the teacher onto states where its targets are unreliable, so student occupancy and teacher reliability move in opposite directions. This is not the Extrapolation Cliff (05-14) failure, which is a capability-gap threshold above which OPD collapses. It is the multi-turn-specific fact that a student can construct histories no competent policy would have produced, and a teacher conditioned on an incoherent history is not a teacher. Two collapse modes, two causes.

The method treats prefix selection as reliability-aware distribution design and implements it bluntly, with a step-decaying sampling schedule that emphasizes early, lower-shift prefixes rather than matching the student everywhere. Teacher trajectories are collected once and replayed as prefixes, the student acts at selected steps, and the teacher supplies dense per-step supervision. Across math-with-Python and search environments over multiple teacher and student scales: zero tool calls during student training, at least 4x faster per rollout, accuracy preserved or improved. Two notes for this page. It is the same shape of result as TIP (04-16), which found most teacher tokens carry no signal and about 10% suffices, except cutting waste on the environment axis rather than the token axis. And it composes almost exactly with MAPD (08-02) below, whose largest unpriced component is an offline multi-agent pipeline run per query before training: ReOPD's entire argument is that pre-collected teacher trajectories amortize across many student updates, which weakens that objection considerably. Neither paper cites the other. → summary

Self-distillation shows up as a gradient-repair tool, not a compression tool. CriPO (08-03) (2607.18082, Zhejiang University and ByteDance) uses on-policy self-distillation from two self-teachers, both being the same policy under a different prompt, to fix rubric-based RL. The measurement worth remembering here is that over 57% of samples contain a criterion the model satisfied whose signal was destroyed by scalar advantage aggregation. The structural note for this page is that CriPO's privileged self-teacher is the second use in two days of a privileged-information branch supplying dense supervision to a deployed branch that never sees it, after MAPD's privileged student branch reading the JSON protocol. A third instance would make it a named pattern. → summary

Current State (as of 2026-08-02)

The neutral-representation pattern reaches four papers, and it has been climbing the abstraction stack the whole time. MAPD (08-02) (2607.24280, Kurate cs.AI #12) distils a closed proprietary teacher for agentic search, where logit matching is impossible because logits are hidden and tokenizers differ, and copying natural-language trajectories transfers writing style rather than reasoning competence. Its intermediate is a JSON protocol holding task type, reasoning plan, and extractive grounding facts, compiled offline by a multi-agent system that decomposes the query, retrieves evidence, and repairs failed searches. The protocol is shown only to a privileged branch of the student policy, whose token distributions then serve as a dense distillation target alongside the sparse RL objective, so the deployed policy learns to reach the protocol's reasoning without ever reading it. 39.4% average success on Qwen3-1.7B and 44.4% on Qwen3-4B across seven QA benchmarks, with style drift and verbosity degeneration explicitly reported as suppressed, and, the part that logit matching structurally cannot do, robust generalisation across different proprietary teachers.

Line up the neutral layers this page has logged and the direction is unmistakable. TESSY (04-18) used hybrid token sequences. Switch-KD (04-18) used a shared text probability space. BPM (07-29) used bytes, the unique lowest common substrate two tokenizers share, recovering the byte-prefix marginal exactly at over 99% of training positions. MAPD uses a semantic schema. Token sequences, probability spaces, bytes, then a task-plan-facts protocol: the neutral layer has moved from a lossless re-encoding of the teacher's output to an interpretation of it. That is the interesting move and also the reason to discount the result somewhat, because there is no unique lowest common substrate for reasoning, so a fixed schema is a bet and no ablation reports which of the three protocol fields carries the signal or how much competence fails to fit.

It is also the second paper here whose teacher is a system rather than a model. CAST (07-30) removed the requirement that the teacher be a neural network. MAPD's teacher is an offline multi-agent pipeline with a repair loop, which produces better traces than any single call to the underlying model would, so the teacher is now allowed to be more capable than the model it is built from. That widens the page's running conclusion again: on-policy distillation is a general statement about transferring any structured or scalar-valued source of state quality into a policy, and the teacher's identity keeps getting less constrained.

One tension the page should hold open. MAPD's privileged branch supplies a dense signal over the whole sequence, which is exactly the uniform-credit pattern the four-paper token-weighting thread below spent three months arguing against (TIP under 10% of tokens, LongAct activation magnitude, Relay-OPD continuation divergence, CoRT counterfactual likelihood for +4.4 points). Composing a MAPD protocol with CoRT per-token weighting is obvious and unrun.

Policy context, which is new for this page. Distillation is now the subject of an explicit regulatory fight. Three open letters in five days (08-02): Microsoft's 235-signatory open-weights letter goes out of its way to defend distillation as "a widely used technique for model improvement, evaluation, and validation," while Anthropic declined to sign and asked for a crackdown on industrial-scale distillation operations. The technical record on this page is the reason that ask is already behind. BPM removed the tokenizer constraint, CAST removed the neural-teacher constraint, and MAPD removes the logit constraint entirely while needing nothing but API access. A policy aimed at industrial-scale distillation is aimed at a technique that just stopped requiring industrial scale.

Current State (as of 2026-07-30)

On-policy distillation lost its third and largest structural constraint in eight days: the teacher no longer has to be a neural network. CAST (2607.25308) observes that in any domain with a classical solver, the change in the solver's state value from one state to the next already says whether an action advanced the position, which is exactly the turn-level credit signal RLVR's sparse final reward withholds. The result that matters is the derivation: under a soft-optimal solver assumption, maximizing the solver advantage is equivalent to on-policy distillation from the solver, requiring only scalar values rather than teacher logits. That collapses the teacher's bandwidth requirement from a vocabulary-sized distribution per token to one number per turn, which is why a Sokoban solver with no tokenizer and no logits can serve as an OPD teacher at all.

Line the month's three results up and the pattern is explicit. BPM (07-29) removed the shared-tokenizer requirement via byte-prefix marginalization. Relay-OPD (07-29) removed the verifier requirement, using teacher-student continuation asymmetry (on a failed prefix the teacher changes course while the student ploughs on) as a label-free handoff trigger for +5.73% and half the trajectory length. CAST removes the requirement that the teacher be a model. What remains of on-policy distillation is a very general statement about transferring any scalar-valued source of state quality into a policy, which is a much larger surface than the literature has been working over. CAST's evidence that this is not merely a game trick is the zero-shot transfer: trained only against solvers on Sokoban, Minesweeper and Rush Hour, it posts the highest average zero-shot performance on ALFWorld and WebShop, which have no solvers.

The token-weighting thread crosses the pattern threshold at four papers. CoRT (07-30) (2607.25659, Nanjing/ByteDance) attacks the same waste from inside the policy. Rubric-based RL decomposes evaluation into explicit written criteria and GRPO then flattens all of it into one response-level advantage broadcast uniformly to every token, even though a formatting criterion and a correctness criterion live in different spans. CoRT replays the same sampled response under the rubric-conditioned prompt and a matched criteria-free prompt and uses the per-token log-likelihood contrast as a proxy for rubric dependence, mapping those contrasts to bounded response-normalized weights that redistribute the signed advantage. No auxiliary scorer, unlike the prior Rubrics-to-Tokens approach which trains a separate relevance model. +4.4 percentage points average over matched response-level GRPO.

Four papers now make the same core claim from four different signals: uniform credit across a trajectory is the waste. TIP found most teacher-generated tokens carry no signal and roughly 10% suffice (teacher agreement). LongAct (04-18) restricted RL gradients to high-magnitude Q/K activation positions for about 8% on LongBench v2 (activation magnitude). Relay-OPD used continuation divergence. CoRT uses counterfactual likelihood. That crosses this wiki's three-paper bar, so the pattern is established rather than emerging. The open question none of them answers is whether the four signals agree with each other on the same rollouts; they are cheap enough that the correlation study is overdue.

Open flag on CoRT: the confound is whether the contrast measures rubric dependence or merely prompt sensitivity, since a token whose likelihood drops when any long prefix is removed gets upweighted for the wrong reason. A placebo-prompt ablation would settle it and is not run.

Current State (as of 2026-07-29)

The reliability estimator this page has called missing since 06-18 gets its first label-free instance, and the shared-tokenizer assumption falls the same week. Two papers, one from HuggingFace and one from the Kurate weekly board, move the line on its two oldest walls.

Relay-OPD ("Pass the Baton", 2607.26057) attacks prefix failure, the failure mode TRD (06-09) named: once the student commits to a wrong early step, the teacher's supervision on everything after it is scoring a doomed path. Its contribution is not the fix, it is the trigger. On failed prefixes the teacher tends to redirect while the student persists, and that teacher-student continuation asymmetry is computable from the two models alone. No verifier, no answer key, no reward model. That matters more than the numbers, because every prior estimator on this page (TA-OPD's teachability, TrOPD's trust region, SG-OPD's sign-consistency, Quality-Aware OPSD's can-this-prefix-still-reach-the-answer, ReOPD's prefix trap) needs supervision to fire, which is why the whole line is stuck in math and code. The asymmetry is coarse (binary and trajectory-positional rather than graded and per-token), so it is not the unified estimator; it is the first member of the family that could run outside verifiable domains. The fix itself is a middle path between the two existing responses: FiRe-OPD (06-04) discarded bad trajectories, TRD repaired them, Relay-OPD hands control to the teacher for a short budgeted leg and takes it back. Results: +5.73% over standard OPD and +1.49% over FastOPD at Qwen3-1.7B across eight math benchmarks, best or second-best on every one, with training trajectory length cut by over 50%. Both the accuracy and the compute saving come from the same source, which is not generating tokens that build on a dead prefix.

Read against Requential Coding (07-25), which proved the bits a teacher actually transmits are charged only where teacher and student disagree, Relay-OPD is the first paper on this page to use disagreement as the selection criterion rather than as an after-the-fact explanation. The 07-25 entry called closing that loop the open move nobody had made. It is now closed at the trajectory level and still open at the token level.

Cross-Tokenizer OPD via Byte-Prefix Marginalization (BPM, 2607.22334, Kurate cs.LG #16, ai_rating 7.0/10) removes a constraint the page has silently inherited from every multi-teacher result it logs. Dense OPD supervises with a distribution over a shared vocabulary, so teacher and student must share a tokenizer, which is why Nemotron 3 Ultra (06-16) and every MOPD run the Interconnects 06-16 podcast named (DeepSeek V4, MiMo-V2-Flash, ten-plus teachers) used in-house teachers. Prior cross-tokenizer methods either discard unmappable teacher mass, producing a biased target, or map it onto student tokens with unrelated content, injecting noise. BPM maps into byte space, the substrate every tokenizer is built on: each teacher token's probability goes to the longest student token whose bytes are a prefix of the teacher token's bytes, colliding mass is summed, and unmatched mass goes to an explicit residual category rather than vanishing. The target is vocabulary-complete and mass-preserving, and exactly recovers the byte-prefix marginal at more than 99% of training positions (measured, with a mass-preserving chain-factorized lower bound for the remainder). Across Qwen3-32B, GLM-Z1-9B, and MiniMax-M2.7 teachers: +3.7 to +6.6 six-benchmark avg@8 over the strongest baselines.

Three papers now occupy distinct answers to "where does the distillation signal live." OPRD (06-05) says the output vocabulary is the wrong venue and aligns hidden states instead. BPM says the vocabulary is fine if you translate it correctly. Relay-OPD says the vocabulary is fine and the problem is where in the trajectory you apply it. BPM is the most immediately deployable because it needs only the teacher's output distribution, no architectural access.

The industry counterpart landed the same week and is not a coincidence. Ben Lorica's essay (07-28) reports 25-plus startups building the reinforcement-fine-tuning stack so enterprises can turn open weights into specialized models they own, with 10 to 30 point gains on well-defined tasks. Consolidating the open ecosystem into one owned student is exactly what BPM makes technically clean, and the verifier-and-grader layer those startups are commoditizing is the missing component this page has flagged as the reason every result here is confined to math and code.

Current State (as of 2026-07-25)

The page's central empirical claim gets an information-theoretic proof, from a paper that was never trying to do distillation. Since TIP (04-16) found that under 10% of teacher-generated tokens carry usable learning signal, every result on this page has been a different estimator of the same latent quantity: which teacher outputs are worth transmitting. TA-OPD called it teachability, TrOPD called it a trust region, SG-OPD called it sign-consistency, Quality-Aware OPSD called it can-this-prefix-still-reach-the-answer, and ReOPD (07-24) generalized it to whole histories. Requential Coding (2607.11883, Qiu / Finzi / Zheng / Zhang / Wilson, NYU and CMU) says what the quantity is. It compresses a model by having a teacher select samples drawn from the student's own distribution and recording only the selections, so bits are charged only where teacher and student disagree. The resulting code length is independent of both parameter count and data entropy, and is often orders of magnitude shorter than the prequential code, with the gap widening as scale grows.

Two consequences for this page. First, it supplies the reason the whole selection line works: if the code that measures transferred information charges nothing at agreement points, then a distillation procedure that transmits agreement points is provably paying for nothing, and TIP's "<10% of tokens" is not a heuristic but the expected shape. Second, it makes the unified reliability estimator the page has flagged as missing since 06-18 look like a compression problem rather than a supervision problem: the estimator everyone is approximating is "how many bits does this teacher output actually move," and requential coding gives a way to write that down.

The paper is not a distillation method and does not produce a smaller servable artifact. It is a measurement scheme feeding a PAC-Bayes generalization bound, where it reaches state of the art for billion-parameter LLMs, beating quantization-based bounds even when quantization is granted zero error. The finding that most stresses this page's neighbours: holding loss fixed, larger models and ensembles compress to smaller codes despite more parameters, which is the opposite of what parameter-count-based complexity proxies (the ones underneath most pruning and quantization work) predict. It is the natural theoretical partner to Dataset Distillation by Influence Matching (07-24), already noted there: both compress by generating a small high-value dataset rather than shrinking weights, one to save training compute, one to measure complexity.

Provenance note: Kurate cs.LG #7 (ai_rating 8.5/10), flagged as LLM-rated-underrated in the 07-22 and 07-24 digests and never surfaced on HuggingFace across three weeks on the leaderboard.

And the same day, a paper builds a computable instance of exactly that quantity. VCSD (Visual Contrastive Self-Distillation, 2607.21556, UMD / UCSD / Duke / MBZUAI) asks whether the teacher-student asymmetry that on-policy self-distillation requires can be built without handing the teacher any privileged information. Its answer is to build the asymmetry out of removal rather than addition: an EMA teacher runs the identical prompt and identical student-generated prefix twice, once with the real image and once with a content-erased control, and the token-wise log-probability difference isolates the tokens whose likelihood was raised by that specific image's content rather than by the language prior. That difference is then used to sharpen the teacher's real-image distribution inside its own plausible support, which is the guardrail against the standard contrastive-decoding failure of promoting tokens the model considers implausible overall. Seven-benchmark aggregate on Qwen3-VL: 62.27 to 67.04 at 2B, 71.30 to 73.16 at 4B, 72.51 to 76.26 at 8B, with no external teacher, no privileged answers, no visual evidence signals, and no inference-time cost.

VCSD extends the branch D-OPSD (05-07) opened, and inverts its polarity. D-OPSD made one network both teacher and student under conditioning asymmetry, giving the teacher more conditioning (text plus target image) than the student (text alone). VCSD gives the teacher two conditionings and uses the gap between them, so the asymmetry is teacher-versus-itself rather than teacher-versus-student. That is the cleaner construction, because no privileged channel has to exist at training time and vanish at inference. Read against Requential Coding, the pairing is the sharpest statement this page has: one paper proves the transferred information is the disagreement, the other measures a disagreement per token and trains on it. The open move nobody has made yet is to close the loop, computing a disagreement-based code length and using it as the selection criterion, which would turn TIP's four-month-old entropy heuristic into a derived rule rather than an empirical one.

The unexplained result worth stress-testing: 8B gains more than 4B. Self-supervision tricks normally decay with scale, and this one does not, on one dataset. The other caveat is structural rather than empirical. Because sharpening stays inside the real-image distribution's support, VCSD can only reweight candidates the teacher already considered, so it cannot recover from a teacher that omitted the right answer entirely, which is precisely the failure a genuinely stronger external teacher fixes. Self-distillation buys independence from a teacher at the cost of inheriting the model's own blind spots.

Prior State (as of 2026-07-24)

The reliability-gating line moves from per-token to per-history, and goes fully offline. ReOPD (Multi-Turn On-Policy Distillation with Prefix Replay, 2607.04763, Microsoft Research / Furu Wei) carries the page's core pattern (never apply the teacher signal uniformly, gate it by reliability) into multi-turn agentic distillation, and names the multi-turn version of the drift the line keeps diagnosing: a prefix trap. Making the interaction history more student-on-policy improves relevance but pushes the teacher onto histories where its target is unreliable, a two-sided distribution shift between student occupancy and teacher reliability. This is the same object as TA-OPD's "teachability" and Quality-Aware OPSD's "can this prefix still reach the answer?" gate, now stated over whole trajectories rather than single tokens. ReOPD's fix is a step-decaying sampling schedule that emphasizes early, low-shift prefixes where the teacher is still trustworthy, and it reuses pre-collected teacher trajectories as replayed prefixes so the student never touches the environment: zero tool calls during training, at least 4x faster per rollout, matching or beating online OPD. It is the offline, agentic descendant of H²SD and pairs with same-week Dataset Distillation by Influence Matching (2607.16859), the data-side cousin: both compress by keeping only the high-value slice (reliable prefixes / influence-matched synthetic samples) rather than processing everything. The unified per-token reliability estimator the page has flagged since 06-18 now also owes a per-history generalization.

Current State (as of 2026-07-22)

The reliability-gating line gets an outcome-conditioned form: use the teacher for direction on failures and magnitude on successes. H²SD (Hybrid Hindsight Self-Distillation, 2607.18955) is the newest instance of the page's "gate the teacher signal, never copy it token-for-token" pattern, but it gates on trajectory outcome rather than per-token reachability. On correct rollouts the self-teacher (the same model given the confirmed-correct answer plus a rephrasing instruction) only modulates update magnitude and never overrides the direction the reward set; on failed rollouts the teacher is conditioned on a reference hint containing the verified answer, and the student minimizes reverse KL toward it, giving an explicit correction direction. This fills the gap the page's predecessor RLSD left (RLSD could modulate step size but never point a failed rollout the right way), and it is the distillation-side sibling of the same-day RLVR-optimization surge on rl-for-llms (ISO frame-only geometry, SAT staleness-tail clipping). It also refines the still-unwritten unified reliability estimator the 06-18 section flagged: H²SD adds outcome (correct vs failed) as the coarsest reliability gate, and its verified-answer-conditioned failure signal sits on the trustworthy end of yesterday's bandwidth-versus-trustworthiness frontier (07-21). Limit: the failure branch needs the ground-truth answer to build the hint, so like Quality-Aware OPSD's box-membership gate it is train-time-only and verifiable-domain-only; reasoning benchmarks only, no coding/agentic results.

Current State (as of 2026-06-18)

The week's pattern has a name now: the OPSD teacher signal must be gated by reliability, not copied uniformly — and the fourth paper in two days carries it into coordinate prediction. Quality-Aware OPSD ("Trust the Right Teacher", arxiv 2606.18101) is the first time the spring selection-and-gating line lands on a VLM-based GUI grounding task (mapping an instruction to the exact screen coordinates to click). Naive OPSD breaks here for the diagnosis the line keeps naming: OPSD scores the teacher on the student's prefix, and once the student's coordinate prefix has drifted off the target, the teacher's next coordinate-token prediction is conditioned on a doomed path and becomes noise. The fix is two coupled mechanisms. A soft correctness-aware gate asks, per coordinate-token, whether the student's current prefix can still be completed into the ground-truth box; if not, that teacher signal is down-weighted. Teacher-probability scaling then calibrates the strength of the surviving signal by the teacher's own confidence. The key empirical claim is that neither component helps alone — gating without calibration and calibration without gating each fall short — and only the combination consistently beats the base model and strong baselines across six GUI grounding benchmarks. The two roles are distinct: gating is a yes/no reliability filter, scaling is a graded calibration.

This makes the four-paper week explicit as an N-of-a-kind pattern. ZPPO (06-17, pull the teacher out of the gradient and into the prompt on questions where every rollout fails), d-OPSD (06-17, condition the diffusion-LLM self-teacher on the student's own self-generated future instead of a left prefix), OPD-Evolver (06-17, distill only the memory operations that outcome-attribution credits as helpful), and now Quality-Aware OPSD (06-18, gate the teacher per coordinate-token by whether the prefix can still reach the answer) are four mechanisms making one claim: on-policy distillation works only when the teacher signal is gated or re-weighted by reliability, never applied token-for-token by default. This is the same instinct the page already tracks under different names — TA-OPD's "teachability" (reachable corrections), TrOPD's "trust region" (reliable-supervision region), SG-OPD's "sign-consistency gate" (verifier-endorsed direction), and TIP's <10%-of-tokens selection. Quality-Aware OPSD adds the cleanest reliability test yet: for a coordinate task the question "can this prefix still reach the answer?" reduces to ground-truth-box membership, decidable per token. The open wall is unchanged and now sharper: TA-OPD's teachability, TrOPD's trust region, SG-OPD's sign-consistency, and this paper's can-still-complete gate are four task-specific estimators of the same latent quantity — per-token teacher reliability — and the unified estimator that subsumes them (with the coordinate-box case as the cleanest instance) is still unwritten. Two limits to note: the gate needs the ground-truth box, so it is train-time only and cannot run at inference; and the result is GUI-grounding-only with no scale study, so whether the reliability-gating frame transfers from coordinates back to general reasoning OPSD is the test to watch.

Current State (as of 2026-06-17)

Three distillation papers the same day, and one of them refuses to put the teacher in the gradient at all. The standing tension on this page has been how to inject teacher signal into the student's gradient without the off-distribution drift the line keeps naming (Many Faces "distribution mismatch", TA-OPD "unreachable disagreement", TRD "prefix failure"). ZPPO (Zone of Proximal Policy Optimization, arxiv 2606.18216) answers by removing the teacher from the gradient entirely and putting it in the prompt: on hard questions where every rollout fails (zero advantage), it builds discrimination prompts (BCQ: one correct teacher + one wrong student answer, anonymized; NCQ: aggregate the student's own wrong rollouts) and recirculates them through a replay buffer until the student "graduates" at 50% accuracy. Largest gains at the smallest scale (0.8B with a 27B teacher), exactly where logit-imitation is most brittle. This is the mirror image of MOPD (Multi-teacher On-Policy Distillation), which the Interconnects 06-16 podcast named the 2026 frontier default (DeepSeek V4 §5.1, MiMo-V2-Flash, Nemotron 3 Ultra >10 teachers): MOPD maximizes teacher-in-gradient (token-by-token reverse-KL to the relevant specialist), ZPPO removes it. Two opposite bets on small-student brittleness in one week — the open question is whether they compose (MOPD for reachable questions, ZPPO for the unreachable hard tail).

d-OPSD (Learning from the Self-future, arxiv 2606.18195 — reuses the acronym of the 05-07 diffusion paper, distinct work) ports on-policy self-distillation to diffusion LLMs for the first time. AR-centric OPSD injects privilege as a left-to-right prefix with token-level KL, which conflicts with arbitrary-order denoising; d-OPSD conditions the self-teacher on the student's self-generated answer as a suffix ("self-future experience") and supervises at the step level (denoising iterations), beating RLVR/SFT at ~10% of RLVR's steps. It carries the conditioning-asymmetry device (PBSD, SDPG) into a new architecture, with the twist that the privilege is temporal-but-not-positional. OPD-Evolver (arxiv 2606.17628) applies on-policy self-distillation to agent memory: a slow/fast co-evolution loop distills four memory abilities (select/use/write/maintain over a 4-level hierarchy) into the deployable policy via outcome-calibrated attribution and privileged hindsight, with a 9B beating 397B-scale models. The "privileged hindsight" + "outcome-calibrated turn-level credit" is PBSD's machinery applied to memory operations. The page's open "joint formulation across all OPD facets" gap now also has to account for prompt-channel teaching (ZPPO), arbitrary-order supervision (d-OPSD), and memory-operation distillation (OPD-Evolver).

Current State (as of 2026-06-16)

Two scale extremes the same day: multi-teacher OPD at 550B, and a claim that verifiable reasoning compresses to 3B. Nemotron 3 Ultra (arxiv 2606.15007) puts Multi-teacher On-Policy Distillation (MOPD) into a frontier post-training recipe — the first multi-teacher OPD the wiki has logged at this scale, extending the page's heavy OPD line (Dense Supervision, Sparse Updates 06-15, SG-OPD 06-12) from single-teacher to a teacher ensemble inside a 550B MoE. At the opposite end, VibeThinker-3B (arxiv 2606.16140) reaches 94.3 on AIME26 and 80.2 Pass@1 on LiveCodeBench v6 in a 3B dense model (using curriculum SFT → multi-domain RL → offline self-distillation), matching far larger flagships on verifiable tasks. Its Parametric Compression-Coverage Hypothesis is the conceptual payload and the natural lift of this page's standing finding that the distillation signal is sparse and locatable (TIP <10% of tokens carry signal; Dense-Sparse's FFN-heavy subnetwork): VibeThinker pushes it from "the update is sparse" to "the capability is compressible," and separates compressible verifiable reasoning from coverage-bound open-domain knowledge. That separation is what makes the day's phase-routing results coherent (Kilo plan/implement, FastContext): if verifiable reasoning fits in a small core, distilling that core out and routing verifiable subtasks to it is the same lever seen from compression and from routing. Open wall, same as the rest of the line: the hypothesis is confirmed only on verifiable (math/code) slices; the "knowledge needs coverage" half is asserted, not measured side by side.

Current State (as of 2026-06-15)

2026-06-15 — OPD's weight-space anatomy gets measured: dense supervision, sparse updates. Dense Supervision, Sparse Updates (arxiv 2606.13657) measures where in the weights OPD writes, across several language and vision-language model pairs, and the answer slots directly into the Geometry of On-Policy Distillation (06-09, OPD sits in a relaxed off-principal regime with early subspace-locking) thread. Two findings. Sparsity: OPD updates are small and coordinate-sparse, distributed across layers, FFN-heavy, and training only the discovered subnetwork recovers nearly full OPD performance, a concrete efficiency lever that operationalizes the subspace-locking observation. Geometry: updates are numerically full-rank but spectrally concentrated, lie mostly off the source weights' principal singular subspace, and fall disproportionately on coordinates where the source weight is near zero, matching the off-principal regime 06-09 described and the RLVR "moves away from principal directions" result (Zhu et al. 2025). The optimizer twist: sparsity-inducing SGD underperforms AdamW for OPD (the reverse of sparse RLVR, where SGD is competitive), because dense teacher supervision preserves heterogeneous coordinate-wise gradient scales that AdamW's adaptive scaling still exploits. This makes optimizer choice a diagnostic for reward density: sparse-reward regimes favor SGD, dense-supervision regimes favor AdamW. The OPD facet list (selection / stability / correction / geometry / credit / verifier-gating) now has its geometry facet measured directly in parameter space rather than inferred from rank dynamics.

Current State (as of 2026-06-12)

2026-06-12 — SG-OPD makes the verifier a gate on the teacher signal, not a replacement for it. SG-OPD (Sign-Gated OPD, arxiv 2606.09304) names two assumptions standard OPD silently relies on and that break in practice: trajectory-level alignment between student and teacher, and uniform token-level reliability of the teacher. It fixes both with a binary correctness verifier at two granularities. "Phased teacher sampling" mixes verifier-endorsed teacher rollouts into the cold-start (trajectory-level), so the student is not distilling from trajectories it could never produce — the same off-distribution diagnosis as TRD (06-09) prefix-repair and FiRe-OPD (06-04) filtering, but resolved by injecting good trajectories rather than repairing or discarding bad ones. The "sign-consistency gate" then extrapolates the update on tokens where the teacher's direction agrees with the verifier-correct direction and interpolates (damps) it where they disagree (token-level). Average gains of +1.98 (per-sample) and +7.50 (per-question) on competition math over standard OPD. The deeper move: it slots a verifier into the dense OPD signal as a modulator, which is the OPD answer to the reward-hacking worry that this week's Kurate-rated "LLMs Gaming Verifiers" (RLVR reward hacking, cs.LG #13) raises — keep the verifier in the loop but never let it be the sole sparse signal. The selection/stability/correction/geometry/credit facets now gain a sixth: verifier-gated update direction. The unified OPD formulation across all six remains unwritten, and SG-OPD's reach beyond verifier-rich domains (math, code) is the same open wall the whole line keeps hitting.

Current State (as of 2026-06-09)

2026-06-09 — OPD gets two mechanism papers the same day, attacking different layers. All spring the wiki tracked OPD interventions (which tokens to keep, reweight, trust). Today two papers ask what OPD is. On the Geometry of On-Policy Distillation (arxiv 2606.07082) characterizes OPD in parameter space: it sits in a relaxed off-principal regime (fewer weights touched than SFT, less constrained than RLVR) and exhibits subspace locking — cumulative updates collapse into a narrow low-dimensional channel fixed early, and constraining training to that early subspace preserves OPD but breaks SFT. Crucially, sparsifying update tokens and shifting rollout off-policy preserve the rank dynamics, while mixing in RLVR changes them, so the geometry is a property of the OPD objective itself. This retroactively explains the whole selection line: TIP (<10% of tokens), TA-OPD (reachable corrections), and OPRD (hidden-state distillation) all worked because the useful update was always confined to a narrow channel. Trajectory-Refined Distillation (TRD, arxiv 2606.08432) attacks the trajectory level: it names prefix failure, where a wrong early step makes the teacher's per-token distribution bimodal and fragments the gradient in a way no token-level truncation or reweighting can fix. TRD revises the bad prefix under teacher guidance within on-policy support before distilling, and broadens exploration by exposing alternative valid derivations even on correct rollouts. Where FiRe-OPD (06-04) filtered out bad trajectories, TRD repairs them, salvaging the discarded signal. A third paper the same day, PBSD (Privileged Bayesian Self-Distillation, arxiv 2606.09348), pushes self-distillation fully into RL credit assignment: it uses the likelihood ratio between a standard student and a privileged answer-conditioned teacher (the same conditioning-asymmetry device as D-OPSD and SDPG) to convert a sparse final reward into Bayes-calibrated turn-level credit for long-horizon search agents. The selection axis (which tokens), the stability axis (TrOPD trust region), the correction axis (TRD trajectory repair), the geometry (locked subspace), and now turn-level credit (PBSD) are five named facets of one OPD/OPSD family — the joint formulation is still unwritten.

Current State (as of 2026-06-07)

2026-06-07 — distillation/compression as the answer to compute scarcity, on two new fronts. Two papers extend the efficiency program away from LLM token-distillation into adjacent regimes. Flash-WAM (arxiv 2606.05254) carries step distillation into joint video-action robot policies and shows the standard single-modality recipe breaks when one model carries two streams at different noise levels: video tolerates high noise, precision-critical actions need a gentle schedule, so a uniform consistency function degrades (naive consistency distillation drops real-world success to 24%). The fix is modality-aware: a variance-preserving consistency function for the high-noise video stream, a linear-gradient-scaling one for the low-noise action stream, grounded in a structural analysis of achievable gradient scaling. Result: single-step inference per modality, 8.1s → 348ms per chunk (23x), real-world success recovered to 60%. Where the spring OPD line (TIP → OPRD) debated what to match, Flash-WAM adds a how-to-parametrize-when-streams-differ axis. Separately, SEAOTTER (arxiv 2606.03940) is sensor-side learned compression for cloud robotics that transcodes once into a standard JPEG so no downstream neural decoder is paid: at 200:1 vs AVIF, 7x faster encode, 3.5x faster decode, +8% ImageNet, full JPEG-ecosystem compatibility. It is the upstream sibling of AdaCodec (06-06, compress frames before the model); SEAOTTER compresses sensor images before they leave the robot. Both are "more perception per watt/byte" levers that matter precisely under the HBM-to-2031 / Black-Friday compute-scarcity backdrop. Also note Google's TIPSv2 (CVPR 2026) reported distillation making a student beat a much larger teacher on patch-text alignment, a third 2026 datapoint that distillation is not merely lossy compression.

Current State (as of 2026-06-05)

2026-06-05 — OPRD changes the venue: distill representations, not output probabilities. The whole spring program above debated which tokens to supervise in output space (TIP → TA-OPD → TrOPD → FiRe-OPD), all of them fighting noisy reverse-KL gradients over a ~150k-token vocabulary. OPRD (On-Policy Representation Distillation, arxiv 2606.06021) argues the variance is an artifact of distilling at the wrong layer. It aligns student and teacher hidden states across selected layers on the same student rollouts, bypassing the LM head entirely. Theoretically this removes the Monte-Carlo sampling variance the selection line was managing; empirically it closes the student-teacher gap on AIME 2024/2025 and AIMO where output-space OPD plateaus below the teacher, and trains 1.44x faster using 54% less memory than top-k OPD (no full-vocab head to estimate). This reframes the entire token-selection debate as possibly solving a problem created by output-space distillation: if matching dense hidden states removes the variance that motivated selecting tokens, the selection axis becomes optional. Open tension: today's Rethinking Continual Experience Internalization finds off-policy context-distillation more stable than on-policy (on-policy is limited by local corrections on student-induced flawed states). OPRD is on-policy but supervises representations rather than token corrections; whether representation alignment inherits or escapes that instability is the question linking the two papers. The selection axis now reads: salient → reachable → reliable → soft-weighted-after-filter; OPRD opens an orthogonal venue axis: output space vs representation space.

Current State (as of 2026-06-04)

2026-06-04 — two OPD papers push past hard token selection and past two-loop stabilization, on the same day. FiRe-OPD (Filter, then Reweight, arxiv 2606.02684) operates on both granularities the selection line split apart: it filters whole trajectories to drop bad rollouts, then applies soft token reweighting inside the survivors. The argument against the TIP/TA-OPD hard-selection line is explicit: hard top-k throws usable signal away, so weight every retained token continuously instead. It reports +6.25 AIME 2024 (strong-to-weak) and +18.81 Miner (multi-teacher) over recent token-level methods, with the multi-teacher gain hinting that trajectory filtering matters most when teachers disagree. This is a partial answer to yesterday's TrOPD Research angle (unify selection across levels), though it does not yet add TrOPD's reliability trust region. SDPG (Self-Distilled Policy Gradient, arxiv 2606.04036) goes the other way and answers yesterday's Looking Ahead prediction directly: instead of two stabilizers, it folds OPD into RLVR as one loss. The model conditions on privileged context to supervise its own generations, written as an exact full-vocabulary reverse-KL student→teacher term, combined with group-relative verifier advantage (÷ std) and reference-KL. Using the exact KL rather than the cheap K1 estimator sidesteps the gradient outliers TrOPD fought, trading memory for stability. SDPG is the "RL is weighted SFT" instinct (DRIFT, 06-01) generalized to self-distillation, and the privileged-conditioning teacher is the same trick as D-OPSD (05-07). The selection axis now reads: salient tokens (TIP) → reachable tokens (TA-OPD) → soft-weighted retained tokens after a trajectory filter (FiRe); the stability axis reads: trust region on a cheap estimator (TrOPD) vs exact full-vocab KL inside the policy gradient (SDPG).

Current State (as of 2026-06-03)

2026-06-03 — OPD gets its stability layer (TrOPD), and the same trust-region primitive shows up in a frontier RL run the same week. Trust Region On-Policy Distillation (TrOPD, Samsung, arxiv 2606.01249) targets the instability the whole OPD theory has been circling: when student and teacher distributions diverge, the student wanders into regions where the teacher's supervision is unreliable, the cheap K1 reverse-KL estimator produces large gradient outliers, and training can collapse. TrOPD applies OPD only inside a reliable-supervision trust region, handles outlier regions with clipping/masking/forward-KL, and uses off-policy guidance (continue from teacher prefixes, imitate via forward KL) to pull on-policy exploration back toward reliable regions. It beats OPD/EOPD/REOPOLD on math, code, and general tasks and ships a unified OPD benchmark. This is the how-to-bound-the-update complement to the which-tokens selection line: TIP (04-16, <10% of tokens) and TA-OPD (06-01, reachable teacher corrections) pick tokens; TrOPD bounds the update. "Reachable tokens" (TA-OPD) and "reliable-supervision region" (TrOPD) are the same instinct two days apart. The strongest signal: Microsoft's MAI-Thinking-1 (same day, via Ken Huang) stabilizes a long GRPO run with an asymmetric trust region plus a hard ratio clamp, steered by an integral controller on policy entropy — the same control-theory primitive against the same enemy (reverse-KL outliers under distribution mismatch), reached independently on the RL side.

Also 06-03 — upward distillation appears (Knowledge Seeding). The 06-03 Language Models Need Sleep (2606.03979) frames continual learning as offline consolidation, with a "Knowledge Seeding" step that distills a smaller-self into a larger network (capacity expansion, not compression) via OPD + RL-based imitation. Unusual direction for the wiki's distillation literature, which compresses downward. (Note: title collides with the unrelated 05-27 SSM-consolidation paper.)

Current State (as of 2026-06-02)

2026-06-02 — On-policy distillation reaches the speculative-decoding drafter (Draft-OPD). Draft-OPD (arxiv 2605.29343) applies the OPD covariate-shift lesson to a new place: the draft model in speculative decoding. SFT-built drafters (EAGLE3, DFlash) plateau because they train on fixed target trajectories but are evaluated on blocks they propose under their own policy — the exact offline-to-inference mismatch the wiki has logged for student models. Naive OPD is hard for drafters because they cannot roll out reliably alone and target-assisted rollout erases the on-policy signal; Draft-OPD resolves this by using target-assisted rollout for stable continuations while replaying drafting from the verification-exposed error positions, so training concentrates on the draft-induced errors that cap acceptance. Over 5x lossless acceleration for thinking models, +23% over EAGLE-3, +13% over DFlash. This is the same diagnosis as TA-OPD (06-01), DRIFT (06-01, RL≡weighted SFT), and DAgger-for-LLM-agents (05-14): offline training distribution ≠ on-policy test distribution. The fix transfers cleanly to the drafter. See also speculative-decoding.md.

Current State (as of 2026-06-01)

2026-06-01 — Teachability closes the loop TIP opened (TA-OPD). Not All Disagreement Is Learnable (TA-OPD, arxiv 2605.26844) refines the central question of selective on-policy distillation: which tokens carry the signal. TIP (2026-04-16) answered "under 10%, the high-entropy and overconfident-wrong ones." TA-OPD shows raw KL disagreement is a coarse proxy because it conflates learnable disagreement (teacher's corrective mass lands on the student's top-K candidates, so the student can reach it) with incompatible disagreement (teacher's mass sits off the student's current support, unreachable in one step). It formalizes "token teachability" (local compatibility) and applies the OPD loss only at high-teachability positions, no reward model or verifier. Result: often beats full-token OPD with only 5% of tokens, and beats entropy- and divergence-based selection, on Qwen2.5/Qwen3 pairs. This is a direct fix for the "biased TopK reverse-KL gradients" failure named by The Many Faces of On-Policy Distillation (05-13): the bias came from training on disagreement the student could not reach. The selection axis now reads: salient tokens (TIP entropy/divergence) → reachable tokens (TA-OPD teachability).

Current State (as of 2026-05-16)

2026-05-16 — Teacher-side control becomes the third axis (ATESD). Every prior OPSD method gave the teacher the full reference solution; ATESD treats teacher exposure as a learnable control variable. A Beta-policy controller, conditioned on training-state statistics, samples a reveal ratio in [0, 1] and holds it for a short window of student updates. A discounted learning-progress reward addresses the delayed credit assignment that single-step rewards cannot solve. On AIME 24, AIME 25, HMMT 25 across Qwen3-{1.7B, 4B, 8B}: +0.95, +2.05, +2.33 Average@12 points over OPSD respectively. Combined with Extrapolation Cliff (closed-form policy for when OPD beats OPRL, 2026-05-14) and SDAR (gated student-side OPSD, 2026-05-15), three orthogonal axes of teacher-signal control are now operational: branch selection (Cliff), student-side gating (SDAR), teacher-side exposure (ATESD). None of the three papers composes with the other two; the joint formulation has not been written. → ATESD summary

Prior State (as of 2026-05-13)

2026-05-13 cluster on on-policy distillation. Two HF papers land on the same day and together they reframe OPD as a layered problem with both an allocation rule and a failure taxonomy. The Sparse-to-Dense Reward Principle (arXiv 2605.12483) argues that GRPO and OPD are not separate recipes but different reward-density regimes, with a clean allocation rule: scarce labeled data should train the strongest teacher first via sparse RL, get bridged to the student by forward-KL + OPD, and only then is student-side GRPO effective. A 1.7B Qwen3 student bridged from an RL-improved 8B teacher beats direct GRPO on the same student. The Many Faces of On-Policy Distillation (arXiv 2605.11182) names three failure mechanisms: distribution mismatch when teacher labels are computed on student-generated prefixes, biased TopK reverse-KL gradients, and OPSD-specific aggregation collapse where the student learns a privileged-information-free average policy that helps no specific instance. OPSD works when the privileged information is a shared latent rule, fails when it is instance-specific. Together with TIP (token selection), the Cliff (operating-point bound, 2026-05-14), and the prior CoPD / D-OPSD / RLRT entries, OPD now has all four layers of a mature theory: allocation, mechanism, failure modes, and bound. → Sparse-to-Dense summary · Many Faces summary

Prior State (as of 2026-05-07)

On-policy distillation has become the dominant approach for reasoning model compression. The key open question is which tokens actually carry the learning signal, naive approaches use all tokens, but most are uninformative. The distillation toolbox now ranges from token-importance methods (TIP) to neutral-channel cross-architecture transfer (BLD, TESSY, Switch-KD, Tide), parallel co-evolution (CoPD), and self-distillation under conditioning asymmetry (D-OPSD, 05-07). Two papers on 2026-05-07 attack the heterogeneous-information-density problem in diffusion distillation specifically: Stream-R1 reweights the DMD objective at both rollout and pixel level using a single shared reward model, and D-OPSD eliminates the external teacher entirely by making the model its own teacher under different conditioning. Policy dimension (2026-05-05): Nathan Lambert's "Distillation Panic" warns that the term is being conflated with API-jailbreaking attacks, and that pending U.S. legislation aimed at "distillation attacks" risks chilling the legitimate technique used by every lab. xAI's trial admission ("Generally AI companies distill other AI companies") is the most direct insider acknowledgment that the practice is industry-wide. The technical and political surfaces of distillation are now both load-bearing.

Key Papers

TIP: Token Importance in On-Policy Distillation (2026-04-16) — Identifies two high-signal token regions: high-entropy (uncertain student) and low-entropy + high-divergence (overconfident but wrong). Entropy-based 50% token selection matches full training with 47% less peak memory. <10% of tokens (targeting overconfident region) nearly matches full baseline. → summary

Key Concepts

• On-policy distillation: student generates its own rollouts, then learns from teacher's token-level distribution over those rollouts
• High entropy tokens: student is uncertain — natural learning signal
• Overconfident tokens (low entropy, high divergence): student is wrong but confident — dense corrective signal often missed by entropy-only selection
• Memory efficiency: token selection not only improves learning quality but also reduces peak GPU memory — critical for distilling large reasoning models

Cross-Tokenizer Distillation / BLD (2026-04-17) — When teacher and student use different tokenizers, standard token-level distillation breaks. Byte-Level Distillation (BLD) solves this by converting both to a shared byte-level representation and distilling there. A lightweight byte-level decoder head is added to the student. Competitive with complex CTD methods despite its simplicity. → summary

TESSY (2026-04-18) — Stylistic divergence between teacher and student causes SFT performance drops even when the teacher's reasoning is correct. TESSY interleaves teacher and student: teacher generates reasoning-heavy tokens, student generates style tokens. The hybrid sequence is stylistically the student's but intellectually the teacher's. Turns a 10% performance drop (Qwen3-8B on GPT-OSS-120B data) into a 6.7% gain on code generation. → summary

Switch-KD (2026-04-18) — Multimodal distillation for VLMs. Routes the student's visual outputs through the teacher's language pathway, forcing transfer through a shared text-probability space instead of separate modality silos. +3.6 points averaged across 10 benchmarks for a 0.5B student distilled from a 3B teacher. → summary

Key Concepts

• On-policy distillation: student generates its own rollouts, then learns from teacher's token-level distribution over those rollouts
• High entropy tokens: student is uncertain — natural learning signal
• Overconfident tokens (low entropy, high divergence): student is wrong but confident — dense corrective signal often missed by entropy-only selection
• Memory efficiency: token selection not only improves learning quality but also reduces peak GPU memory — critical for distilling large reasoning models
• Cross-tokenizer distillation (CTD): transferring knowledge when teacher and student use different tokenizers — the vocabulary mismatch problem
• Byte-level interface: using raw bytes as a universal common representation between any two tokenizers
• Stylistic divergence: teacher and student have different learned generation styles; a teacher's reasoning traces can be too foreign for the student's optimizer
• Cooperative synthesis (TESSY): interleaving teacher/student token generation to create hybrid training data that matches student style while containing teacher reasoning

ShadowPEFT (2026-04-22) — Centralized PEFT via depth-shared shadow module. Shifts adaptation from distributed weight-space perturbations (LoRA) to a single shadow module that evolves a parallel state through all transformer layers. Decoupled from backbone, independently pretrainable, optionally deployable in detached mode for edge computing. Matches or outperforms LoRA and DoRA at comparable parameter budgets. → summary

Tide: Cross-Architecture Distillation for Diffusion LLMs (2026-04-30) — First framework to handle teacher/student mismatch in all three of architecture, attention mechanism, and tokenizer for diffusion LLMs. Three components: Tidal (noise-aware schedule across timesteps + training progress), CompDemo (complementary-mask context enrichment), Reverse Calm (inverted chunk-likelihood matching with bounded gradients). Distills 16B MoE / 8B dense teachers into a 0.6B BD3LM student; +1.53 avg across 8 benchmarks; HumanEval 32.3 → 48.78; 22× memory reduction, 5× inference speedup vs teacher. → summary

CoPD: Co-Evolving Policy Distillation (2026-05-01) — Multi-capability post-training via parallel expert RLVR + bidirectional OPD during training (not after). Mixed RLVR has divergence cost; train-experts-then-OPD has behavioral-pattern gap; CoPD avoids both by having experts mutually distill while RLVR-training in parallel. Integrates text/image/video reasoning into one model that surpasses domain-specific experts. Bidirectional OPD as the neutral exchange channel, applied to parallel training rather than student/teacher pipeline. Fifth paper in the cross-distillation-channel pattern (after BLD, TESSY, Switch-KD, Tide). → summary

The Distillation Panic — Nathan Lambert (2026-05-04) — policy/discourse piece arguing that the "distillation attacks" framing being pushed in U.S. legislation conflates legitimate post-training distillation (used by every lab, including Nemotron and Olmo) with API jailbreaking (which should be called jailbreaking). The risk: a domestic ban on Chinese open-weight models built via API distillation, with collateral damage to Western academics and small labs. Quotes Musk's xAI trial admission as evidence the practice is industry-standard. Adds the policy dimension to the wiki's distillation tracking. → summary

Stream-R1: Reliability-Perplexity Aware Reward Distillation (2026-05-07) — DMD (Distribution Matching Distillation) for streaming video diffusion, reweighted by a single shared video reward model on two axes: inter-reliability (per-rollout, via exp(reward_score)) and intra-perplexity (per-pixel, via gradient saliency from the same reward). Adaptive balancing prevents any single quality axis (visual, motion, alignment) from dominating. The video-streaming analogue of TIP: heterogeneous information density across rollouts, frames, and pixels means uniform supervision wastes signal. → summary

D-OPSD: On-Policy Self-Distillation for Step-Distilled Diffusion (2026-05-07) — addresses the practical problem that standard SFT destroys the few-step capability of step-distilled diffusion models (Z-Image-Turbo, FLUX.2-klein). Novel paradigm: the same model serves as teacher and student under different conditioning. Teacher sees text plus target image (multimodal), student sees only text. Loss minimises divergence between the two over the student's own rollouts. The seventh paper in the neutral-exchange-channel pattern, but with a new neutral channel: conditioning asymmetry on the same network. → summary

Key Concepts

• On-policy distillation: student generates its own rollouts, then learns from teacher's token-level distribution over those rollouts
• Centralized PEFT (ShadowPEFT): single depth-shared module performs layer-space refinement, unlike LoRA's per-layer weight perturbations
• Layer-space vs weight-space adaptation: ShadowPEFT refinement evolves a parallel state through the network depth; LoRA adds local rank-decomposed perturbations to individual matrices
• Cross-architecture diffusion distillation (Tide): distill from a diffusion teacher to a diffusion student of different size/attention/tokenizer; requires noise-aware scheduling, complementary-mask context enrichment, and bounded-gradient cross-tokenizer losses
• Neutral exchange representation pattern: across BLD (bytes), TESSY (cooperative interleaving), Switch-KD (shared text-probability space), Tide (inverted chunk-likelihood with bounded gradients), and CoPD (bidirectional OPD between parallel RLVR experts), the field has converged on engineering a neutral channel between mismatched teacher/student rather than forcing token alignment. Five papers, five mechanisms, one principle.
• Co-evolution distillation (CoPD): parallel RLVR-trained experts serve as mutual teachers via bidirectional OPD during training, eliminating both the inter-capability divergence of mixed RLVR and the behavioral-pattern gap of train-then-distill
• Selective-supervision OPD (the dominant 2026 thread): uniform per-token supervision is wasteful and often harmful, so select for the load-bearing signal. Sequence: TIP (04-16, ~10% of teacher tokens carry signal) → TA-OPD (06-01, only reachable teacher corrections) → TrOPD (06-03, only tokens where the teacher is reliable) → FiRe-OPD (06-04), which argues for two-level granularity: hard-filter trajectories, then soft-reweight tokens (soft > hard avoids information loss) → SG-OPD (06-12), verifier-gated update direction → Quality-Aware OPSD (06-18), which gates the teacher per coordinate-token by whether the student's prefix can still complete into the ground-truth box, then calibrates the survivors by teacher confidence (first time this line lands on a GUI-grounding VLM task). SDPG (06-04) attacks the same density problem from the RLVR side, manufacturing a dense full-vocabulary reverse-KL teacher from the model's own privileged-conditioned self. The one-week convergence (ZPPO, d-OPSD, OPD-Evolver, Quality-Aware OPSD, 06-17/06-18) makes the principle explicit: the teacher signal must be gated/re-weighted by reliability, never copied token-for-token by default.
• Budgeted expert reads (MergePipe, 06-04): weight-space merging is I/O-bound at LLM scale; MergePipe casts it as a budgeted expert-access-set problem with a provable omitted-delta error bound and 11x speedup. The merge-time companion to BEAM's inference-time expert-activation budgeting.

Related Pages

• ../llms-foundation-models/rl-for-llms.md
• ../llms-foundation-models/2026-04-16-prerl-rl-in-pretrain-space.md
• kv-cache.md

The anti-distillation defense is architecturally broken (2026-08-11)

Stealing Reasoning Traces from Proprietary LLM APIs (08-11) changes the standing state of knowledge on this page in a way no method paper has. Every entry above concerns how to distill well. This one concerns whether distillation can be prevented, and the answer is no.

The mechanism matters because it explains why the defense fails. Frontier providers hide chain-of-thought by encrypting it and returning the ciphertext to the client, which replays it on each subsequent request. The encryption binds nothing: the blocks are fully interchangeable across sessions, users, and models within one provider's ecosystem. So an adversary injects a strong model's encrypted trace into a weaker, less-safeguarded sibling model from the same provider, and that model decodes and emits the trace verbatim in plaintext. The protected model is never jailbroken, so its safety training is irrelevant to the attack. Demonstrated across Anthropic, OpenAI, and Google, which makes it a design-pattern failure rather than one vendor's bug.

Three consequences for this page:

1. Hidden reasoning is not a moat. Any lab relying on concealed CoT to stop competitors training on its reasoning has been relying on nothing. The recoverable artifact is exactly the trace that on-policy distillation wants.
2. The policy argument shifts ground. The Distillation Panic (05-04) argued that legislative framing conflates legitimate post-training distillation with API jailbreaking. This attack is neither: it does not jailbreak the protected model at all, so a regime that polices jailbreaking does not reach it. The distinction the policy debate rests on is not the distinction that governs the technique.
3. Provenance moves from prevention to detection. Anthropic shipped in-text invisible watermarking on 08-10, carried in the text rather than metadata so it survives copy-paste. Hugging Face's Elie Bakouch asked publicly whether watermarking is really an instrument for proving a competitor trained on Claude output. If prevention has failed, watermark-based forensics is the remaining lever, and it is a fundamentally weaker one: it establishes that copying happened after the fact rather than stopping it.

Open question this raises for the selective-supervision thread. Every method on this page assumes a cooperative teacher whose distribution you are licensed to query. A recovered trace gives you the teacher's text but not its per-token distribution, so the extraction attack supplies SFT-grade data, not the dense token-level signal that on-policy distillation depends on. Whether trace-level theft is actually competitive with licensed dense supervision is unmeasured, and it is the number that decides how much the broken defense really costs. The paper does not report it.

2026-08-25: from "most teacher tokens are useless" to "some are actively harmful"

R2-OPD (08-25) sharpens the central claim of this page's selective-supervision thread. The prior state of knowledge here was that most teacher-generated tokens carry little learning signal and should be dropped or downweighted. R2-OPD makes a stronger and more uncomfortable claim: a subset of teacher supervision actively fights the thing you are training for.

The mechanism is a proxy failure, not a noise problem. On-policy distillation gives dense token-level reward derived from teacher agreement, and it implicitly treats that reward as a proxy for reasoning progress. The two come apart in a specific, systematic way: when the student finds a different valid reasoning path, teacher agreement falls while actual progress is fine, so the student gets punished for exactly the independent correct reasoning you want. Noise averages out over training; a systematic penalty on divergence-from-teacher does not. It biases the student toward mimicry, which is why the paper is titled Beyond Imitation.

The fix is deliberately not a better reward model. R2-OPD builds a second, cheap one and uses only the disagreement between them: rank reasoning spans within a trajectory by teacher-derived reward, rank them independently by estimated progress reward, and suppress the distillation reward wherever the two orderings disagree. Working on within-trajectory rankings rather than absolute values is the correct call, since the two reward sources share no scale. It bolts onto existing OPD pipelines.

This crosses the wiki's threshold for a named pattern: disagreement between two signals is itself the signal. Three independent instances now. The 08-08 weekly cluster (Requential Coding, OPD²) established that a single teacher signal is untrustworthy at token granularity. AgentOPSD (08-07) located pivotal turns in agent trajectories by disagreement rather than absolute reward. R2-OPD locates untrustworthy tokens the same way. The design principle to carry forward: do not trust a single supervision signal, and treat the conflict between two independent signals as the highest-information location in the trajectory.

The same idea surfaced in a different subfield on the same day, which is the more interesting observation. Task-CoEvolve (08-25) uses variance-weighted sampling to concentrate harness evaluation on validation tasks where candidate harnesses disagree, cutting evaluation count 80%. Distillation and harness evaluation, no shared authors, one day, one principle. It is standard active-learning theory (information concentrates at the decision boundary) being independently rediscovered rather than imported, and Task-CoEvolve is the more refined instance because it weights by disagreement magnitude where R2-OPD's filter is binary.

The load-bearing weakness, stated plainly. The independently estimated progress reward is itself an unvalidated learned model, and the sensitivity ablation is missing. This matters in a specific way: if the progress estimator errs in the same places as the teacher, the filter quietly does nothing and the reported gains come from somewhere else; if it errs in different places, the filter suppresses good supervision. Without that ablation the result is credible but its mechanism is unconfirmed. Note that VoI-MoLE (08-25) has the identical structural weakness in its reducibility estimator, so the pattern comes with a shared open risk: every method in this family depends on a second estimator nobody has validated.

Open experiment, now buildable. The 08-08 weekly asked for the composition of turn-level and token-level disagreement filtering. AgentOPSD supplies pivotal turns, R2-OPD supplies untrustworthy tokens, and filtering the pivotal tokens of the pivotal turns is the two-level version. Also unexplored: weighting by disagreement magnitude instead of thresholding it.

2026-08-28: the supervisor becomes optional, and the label goes with it

Two papers on one day each delete a different expensive dependency from the supervision pipeline, and together they push this page's central 2026 finding to its limit.

Self-OPD (arXiv 2608.26872, Tsinghua + Zhejiang + Alibaba) removes the teacher entirely for flow matching. The two problems it names are the ones this page has been circling: cost, because training a specialized teacher for every new objective is a per-objective expense rather than a one-time one, and compounding error, because a student regressed onto teacher velocity predictions drifts off the teacher's support step by step, so the supervision degrades exactly along the trajectory where it is needed. The construction: at each timestep, branch the deterministic next-state prediction into K stochastic SDE candidates, roll each to a finished sample with the ODE sampler, and compare their rewards against a deterministic self-reference baseline to get normalized advantages.

Three design choices carry it. The baseline is the deterministic path rather than the population mean, so the advantage answers "did adding noise here help relative to the confident path," which is a sharper question. The objective is all-branch pull-push: high-advantage branches attract the velocity field and low-advantage branches actively repel it, under direction-aware attenuation and SDE-variance normalization. Standard on-policy distillation only pulls; explicitly pushing away from bad branches is what lets a teacher-free method extract signal from its own failures instead of discarding them. And for multi-objective alignment it fuses at the reward level, not the gradient level, which the alphaxiv overview explains well: teacher-dependent multi-objective OPD merges several teachers' velocity fields, and conflicting update directions produce a compromise satisfying no objective, giving models that excel on one teacher's prompt family and fail elsewhere. Normalizing scalars before any gradient forms resolves the conflict in a space where it is well-defined.

This is the fourth instance of the teacher-ceiling pattern in four days and the strongest resolution of it. OPRD (06-05) found output-space distillation plateaus below the teacher on AIME and AIMO. OPDVR (08-26) diagnosed that a purely distributional objective bounds the student at its teacher and broke the bound with a verifier. QAH (08-26) found the ceiling was a degraded recovered checkpoint and swapped the teacher pointer to the original pre-compression model. Self-OPD removes the ceiling by removing the referent: a student supervised by its own advantage-ranked exploration has no teacher to be bounded by. The 08-26 digest stated the shared diagnosis as the binding constraint in distillation is the supervisor, not the student. Four papers, four mechanisms, one claim, well past this page's three-instance threshold.

It is also the second teacher-free dense-supervision result for generative models in three days. DiffusionOPSD (08-26) was the first entry on this page where the teacher is the student: a frozen behavior policy supplies anchors, reward gradients build bounded positive and negative targets, an EMA update refreshes the behavior policy, cutting training GPU-hours 40% on SD 3.5-M and 63% on the step-distilled Z-Image-Turbo. Self-OPD reaches the same place by branching at the SDE level instead of anchoring on a frozen copy. Two large groups, two constructions, three days. The gap between them is telling: DiffusionOPSD led with a GPU-hour reduction, and Self-OPD publishes no compute comparison at all, which matters because K full ODE rollouts per timestep is not obviously cheaper than the teacher it deleted. That is the load-bearing omission.

TTPO (arXiv 2608.27448, Zhejiang + Alibaba) removes the ground-truth label, and does it without adding an unvalidated estimator. The problem is specific: every method on this page needs labels for verification or teacher conditioning, so none can run at test time. Substituting a majority-vote pseudo-label fails worse for dense methods than for coarse ones, because a corrupted pseudo-label conditioning a teacher misleads at every token, where a corrupted sequence-level reward misleads once per trajectory. TTPO's observation makes the failure survivable: the error is asymmetric, because rollouts that disagree with the pseudo-label are usually wrong regardless of whether the vote itself was right. So route by branch. Distil the agreeing rollouts with on-policy self-distillation, penalize the disagreeing ones with grouped RL, and refine both with token-level selection: distillation down-weights already-converged positions, RL penalizes only confident errors. Without labels, it matches label-supervised OPSD on five competition benchmarks, raises Qwen3-1.7B from 38.0% to 45.2%, and yields +25.2% to +36.4% without thinking.

TTPO is the fourth instance of the disagreement-is-the-signal pattern, and the first to use disagreement as a dispatcher rather than a mask. The prior three (the 08-08 cluster on untrustworthy token-granularity teacher signal; AgentOPSD 08-07 locating pivotal turns by disagreement; R2-OPD 08-25 locating untrustworthy tokens by disagreement between teacher-derived and progress-derived rankings) all used disagreement to decide what to suppress. TTPO uses it to decide which algorithm to apply, which is a strictly more expressive use of the same signal.

And it is the best available answer so far to the shared risk this page flagged on 08-25. That note read: R2-OPD's independently estimated progress reward is itself an unvalidated learned model, a weakness shared with VoI-MoLE (08-05)'s reducibility estimator, so every method in this family depends on a second estimator nobody has validated. OPDVR escaped by using a verifier, which is not an estimator but needs ground truth. TTPO's second signal is a majority vote: no parameters, no training, a characterized failure mode (it fails when the model is confidently and coherently wrong), and it tightens as the policy improves. For the label-free regime that is a genuinely better construction than a learned estimator. The unquantified part is the asymmetry rate itself: how often a disagreeing rollout is actually correct bounds the damage the negative branch does, and on problems where the model is below chance the consensus is systematically wrong and the method could invert. Five competition math benchmarks is an unusually consensus-friendly setting.

Cross-page note. TaoLive Harness-Aware Training (08-28) uses General On-Policy Distillation as stage-two damage repair, to restore the generalization that harness-augmented SFT destroyed. That is structurally the same move QAH made two days earlier when it refused to distil from a degraded recovered checkpoint. Distillation as the thing that undoes the cost of a specialization or compression step is now a recurring role for it on this page, distinct from distillation as capability transfer.

Updated state of knowledge. The distillation program now has five axes: which tokens (TIP through Quality-Aware OPSD), which trajectories (FiRe-OPD), which teacher (QAH), whether the objective is bounded by the teacher at all (OPDVR), and now whether a teacher or a label is required in the first place (Self-OPD, TTPO). The direction is consistent and the accounting is not: the two 08-28 papers delete a dependency and neither publishes the compute that replaced it, which is the same omission the harness pages have carried since May.

2026-08-30: the teacher's right to abstain

FedCC (arXiv 2608.23031, Kurate cs.LG #1 this week) is this page's thread arriving in a distributed setting, with one twist worth naming. In distillation-based federated learning, clients never ship weights or data. They run their local model over a shared unlabeled public dataset and ship only predictions, which the server aggregates into pseudo-labels. Label distribution skew breaks this: a client that has only seen two classes does not produce a flat distribution on an unseen class, it produces a confident wrong one, and the server has no ground truth to calibrate it away with.

FedCC's fix is to let a client tag ambiguous samples unknown, plus calibrated pseudo-labels balancing majority-class confidence against minority-class uncertainty. In the extreme case where each of ten clients holds exactly one class, it reaches 67.3% accuracy where baselines collapse to near-random.

The twist relative to everything else on this page. The established finding here is that most teacher signal is worthless: TIP found most teacher-generated tokens carry no learning signal so roughly 10% suffices, and token teachability (06-01) made per-token teachability the selection criterion. FedCC's teacher is not merely uninformative on some inputs, it is actively harmful, and the fix is to let it decline rather than to have the student filter afterwards. Filtering at the source rather than at the sink, which in this setting is not a preference but the only option, because the sink has no labels to filter with.

What is not reported is the abstention rate, and it decides whether this is usable: if clients abstain on most public samples under severe skew, the effective communication is far sparser than the bandwidth argument implies, and there is a regime where the server has too few votes per sample to calibrate anything. Nothing here is at language-model scale either, and whether an unknown class means anything for a generative model, where the output space is not a fixed set of labels, is not a small extension.

Cross-page connection. FedCC lands the same day as two selection papers on the new model pruning and sparsity page: RoI selecting N weights of M, and MCL selecting training samples by marginal concept-coverage gain. All three replace a forced dense decision with a sparse one plus an explicit way to decline, and FedCC states most clearly why that matters: the alternative to a sparse decision is not a neutral one, it is a confidently wrong one.