Knowledge Distillation

Transferring capabilities from a large teacher model to a smaller student model. On-policy distillation (OPD) trains the student on its own rollouts under token-level supervision from the teacher.

Addendum, 2026-09-09: distillation became a national-security category, and the industry's most quotable figure immediately said it should not be

This page has treated distillation as an engineering technique with an accounting problem. On the same day it gained its eighth selective-supervision axis, the U.S. government reclassified it as an exfiltration method.

NSA, CISA and FBI published joint advisory AA26-251A, naming DeepSeek, Moonshot AI, Alibaba, MiniMax, StepFun and Z.AI as running industrial-scale distillation campaigns against U.S. frontier models since at least late 2024. The alleged mechanics are supply-chain rather than technical: a gray market of API proxies called transfer stations, bulk premium subscriptions shared across developer teams, and third-party aggregators that strip account metadata. The specific claims worth recording: Moonshot AI allegedly distilled 17 U.S. models, including Claude Fable 5, to train Kimi-K3, and DeepSeek's prompts allegedly pushed models to write out hidden chain-of-thought steps, which transfers the reasoning method rather than finished answers.

The economic argument in the advisory is the part that belongs on this page, because it is a claim about the cost accounting this page tracks. The agencies argue DeepSeek's headline training cost is misleading because it covers only the compute burned and excludes what the training data cost, which they allege was obtained by distilling U.S. models rather than produced through original research. Stated as a technical claim rather than a political one: on-policy distillation from a strong teacher moves the dominant cost from compute to access, and a training-cost figure that omits teacher access is not a comparable number. That is a real methodological point about every efficiency claim in this literature, independent of whether the allegations hold.

The advisory's proposed mitigation is worth flagging as a research hazard. It asks U.S. labs to serve suspected distillers subtly degraded responses without telling them. If that is adopted, the teacher-reliability problem this page has spent all month on stops being a modelling question and becomes an adversarial one: TGOPD (09-08) verifies teacher reliability with a small set of verifier-scored probes, and a teacher deliberately degraded on some traffic and not others is designed to defeat exactly that kind of sampling. OPRD (09-09) is structurally more robust here, because a teacher that can only rescale verifier-supported gradient directions cannot inject a poisoned direction at all, only waste updates. Neither paper contemplates an adversarial teacher and both should.

Jensen Huang, asked the same day whether open-source model companies should be allowed to distill closed models, took the opposite position outright: "Distillation, learning from AI, learning from other people, and learning from other sources of knowledge, is fundamental to intelligence... AI systems will constantly be distilling knowledge and intelligence from other AI systems. The fact that AI can learn is a good thing." The largest supplier of the compute both sides run on is arguing that the category the advisory just criminalized is definitionally legitimate. This page should hold the disagreement rather than resolve it, and note that the technical question, whether teacher-generated chain-of-thought transfers method rather than answers, is the one both positions rest on and is empirically settleable.

2026-09-09: the eighth axis is the first that does not gate at all, and it makes the missing control matter more

Every one of the seven axes below is a gate: which tokens, which prompts, which examples get to carry teacher supervision. OPRD (09-09) (arxiv 2609.08798) admits everything and instead projects.

The premise is that conventional distillation makes the teacher an optimization target, which imposes the teacher's capacity ceiling on the student. That is tolerable when the teacher is stronger and fatal in the two cases that matter commercially: successive model generations and multi-domain consolidation, where repeating frontier-scale post-training from scratch is prohibitive and the teacher may be weaker in places. OPRD measures the teacher's policy shift relative to its own reference policy, evaluated on the student's rollouts, and amplifies the component of the student's verifier-driven policy gradient that already points that way. Because it only rescales updates the verifier already supports, it preserves the stationary points of policy optimization while converging faster. Reported: higher performance with fewer student updates than existing RL and distillation baselines in both successive-generation transfer and multi-teacher distillation, and it still works in ordinary strong-to-weak distillation.

That guarantee is strictly cleaner than any gate on this page, and the reason is worth stating. A gate can be wrong about a prompt and admit a bad direction. A projection onto the verifier-supported subspace cannot introduce a direction the verifier did not already produce, so an unreliable teacher can only change the rate, never the destination.

Its response-style analysis is the evidence, and it is the most convincing artifact in this family. OPRD students end up closer to models trained with verifier-based RL alone than to their weak teachers. If the teacher were imposing a ceiling, the students would look like the teacher.

It arrives one day after TGOPD (09-08) and shares its exact diagnosis. TGOPD's premise is that reverse KL is mode-seeking so a confidently wrong teacher induces a strong misleading update, and its treatment is to verify teacher reliability per prompt before admitting dense supervision. OPRD's treatment is that you never need to trust the teacher if the teacher can only scale gradients the verifier already produced. Two papers one day apart on the same failure, and OPRD's mechanism dominates in exactly the case per-prompt probes miss: a teacher that is unreliable in a way its probe responses do not reveal. The head-to-head is one table and neither has it.

It is also the nearest thing yet to evidence for this page's standing complaint, without actually running the control. The 09-08 prediction was that no paper in this family has published the random-gate control at matched admission rate, and that One-Example OPD (09-04), where sixteen queries match full-data training, implies most of these filters optimize a non-binding constraint. OPRD does not run that control either. But "teacher guidance accelerates rather than redirects" is precisely what a non-binding constraint would look like from the inside. Treat it as partial, indirect support and keep the prediction open.

The unpaid bill. "Fewer student updates" is the efficiency claim, and running the teacher to measure its policy shift on every student rollout is a real per-step charge that is not netted out. And the whole method is verifier-dependent, so it inherits the verifiable-domain restriction the RLVR line has always had.

2026-09-08: a seventh axis arrives, and it is the first one whose real contribution might be a scheduling result

TGOPD (09-08) (arxiv 2609.02998, AllSpark Team, HuggingFace daily) adds which prompts to this page's selective-supervision list, verified rather than estimated. The gate runs repeated verifier-scored teacher probes per prompt and decides, prompt by prompt, whether to admit dense on-policy distillation supervision or fall back to verifier-grounded GRPO. The motivation is specific and correct: vanilla OPD's mode-seeking reverse KL turns a confidently incorrect teacher into strong misleading updates, and every distributional proxy the field has tried (EOPD's forward KL at high-entropy teacher tokens, TrOPD's decoding-agreement trust regions, REOPOLD's likelihood-ratio clipping) measures uncertainty or compatibility rather than correctness. A teacher can be confident, agree with the student, and be wrong. The prior outcome-evidence methods (RG-OPD, RLSD, SPOT) do use verifiers but operate at the trajectory or token-branch level. Prompt level is the granularity at which "the teacher does not know this topic" is actually a true statement.

The economics are the elegant part. In asynchronous OPD the teacher node idles waiting on student rollouts, and TGOPD spends that idle capacity on the probes. Reported teacher-node GPU utilization goes from 9.8% to 78.9%. So the gate is not a filter with overhead to justify; it is an answer to the question of what an expensive frozen asset should do while it waits. That insight generalizes past distillation to speculative decoding, reward-model-in-the-loop RL, and cascade routing, all of which park an expensive model in a blocking wait.

But this page has to raise the objection its own 09-04 entry created. One-Example OPD (09-04) found that alignment with the teacher slows at the same rate whether you train on one query or the whole dataset, that sixteen semantically distinct queries reach 98.9% state coverage and match full-data training, and that content-light templates and off-domain WildChat queries also approach the real-query baseline. Its diagnosis, absorbed whole here, is that OPD is data-overfed but algorithm-starved, with the consequence that a filter improving which supervision the student sees is optimizing a quantity that was not limiting. TGOPD is a filter over prompts. On One-Example OPD's own finding, prompt selection should be close to worthless.

Two readings, separable by one cheap experiment. Generous: reliability and coverage are different quantities, and on a small prompt set each bad prompt is proportionally more damaging, so gating matters more not less. Deflationary: TGOPD's accuracy gains would survive replacing its gate with random prompt dropping at the same admission rate, and the durable contribution is the 9.8% → 78.9% utilization, which is scheduling and not distillation. The random-gate control settles it, it costs one run, and nobody in this seven-axis literature has ever run it. That is now this page's single most conspicuous missing ablation, and it applies retroactively to every axis here.

It pairs with RISE as opposite solutions to the same accounting problem. RISE (09-07) builds a synthetic teacher from the student's own optimization trajectory and deletes the external teacher entirely. TGOPD keeps the teacher and fills its idle cycles. A frozen teacher is an expensive, badly-utilized asset; RISE says stop paying for it, TGOPD says use what you already bought. Read as cost optimization RISE dominates if its extrapolation is safe, and this page has flagged that its safety is unmeasured because the Extrapolation Cliff (05-14) established a closed-form threshold above which on-policy distillation collapses and RISE reports no anchor-distance or extrapolation-coefficient sensitivity. TGOPD carries no cliff risk because a real teacher cannot drift. Treating them as a variance portfolio rather than ranking them is the useful framing.

And the standing pass@k complaint applies with full force. Since IDA-OPD (09-03) showed that concentrating supervision concentrates the student's output distribution (pass@1 improving while pass@k plateaus, traceable to identifiable negative-influence positions), this page has treated a missing pass@k as a red flag on any selective-supervision result. TGOPD discards entire prompts, which is the most aggressive concentration operation in the seven-axis set, and reports no diversity metric at all. Note also that mode-seeking reverse KL is the shared mechanism behind both TGOPD's failure mode and IDA-OPD's: one removes prompts where the mode is wrong, the other reshapes updates where the mode is too narrow. Obvious compose, unrun.

2026-09-07: a result that is not on any axis, because it stops selecting supervision and starts manufacturing a teacher

RISE (09-07) (arxiv 2609.05295) does not add a seventh axis. It attacks the premise the other six share. Every axis on this page is a filter over supervision the teacher already produced: which tokens (TIP, 04-16, most teacher tokens carry no signal and roughly 10% suffice), which layer (OPRD, 06-05, match hidden states not output tokens), which trajectories (OPDVR, 08-26, gate on verified correctness), which teacher (QAH, 08-26, distill from the original pre-compression model), which updates (IDA-OPD, 09-03, label each update entropy-expanding or contracting), which queries (One-Example OPD, 09-04, sixteen queries reach 98.9% state coverage). RISE selects nothing. It builds a teacher out of the student's own optimization history.

The mechanism: take the displacement between the current checkpoint and a trailing anchor checkpoint, extrapolate along it in parameter space or in output-logit space, and treat the extrapolated model as a synthetic teacher. That converts a sparse outcome-induced RLVR update into a dense token-level target, with no external model and no privileged conditioning. The two signals are complementary rather than redundant: the outcome reward grounds the extrapolation so the direction does not drift into a region that merely looks like improvement, and the extrapolated teacher converts that grounded direction into supervision at a resolution outcome reward can never reach. Because the anchor refreshes every iteration as the student improves, distillation stops being a one-shot compression step and becomes a recursive improvement loop. Reported to beat both RLVR-only training and on-policy self-distillation on math reasoning, multi-domain STEM, code generation and multi-turn agentic tasks.

It is the fourth instance of this page's strongest pattern, and the most literal one. OPRD, OPDVR and QAH independently concluded that in distillation the binding constraint is usually the supervisor, not the student, with three different fixes: change the layer, add a verifier, change the teacher. RISE takes the diagnosis at face value. If the supervisor is the constraint, construct a better supervisor from the only source guaranteed to be on-distribution, which is the student's own path. Distribution mismatch, the failure mode that motivates half this page, is impossible by construction here.

It is also the constructive answer to One-Example OPD's verdict. That paper's diagnosis, which this page absorbed whole, is that OPD is data-overfed but algorithm-starved. RISE is a pure algorithm change with no data change at all. The composition is obvious and unrun: RISE's synthetic teacher on One-Example OPD's sixteen-query budget would test whether two independent findings multiply.

Two things this page should hold RISE to, and the first is in its name. The Extrapolation Cliff (05-14) found a closed-form threshold above which on-policy distillation collapses, and RISE's entire mechanism is extrapolation along a displacement vector. How far you may step, and what happens as the anchor goes stale, is precisely the quantity that paper says has a cliff, and no extrapolation-coefficient sensitivity or anchor-distance ablation is reported. Second, this page has treated a missing pass@k as a red flag on any selective-supervision result ever since IDA-OPD showed that concentrating supervision concentrates the student's output distribution. A teacher extrapolated from the student's own direction of travel is the most self-reinforcing supervision signal this page has recorded, and RISE reports no pass@k.

A same-day arrival from the RLVR side that belongs in the same frame. GAPO (09-07) (arxiv 2609.00444) scales the importance-sampling clipping boundary with rollout advantage, on the observation that rare correct rollouts on hard problems carry the strongest exploration signal and are the ones a fixed clip suppresses most. It improves both Pass@1 and Pass@k. Put beside IDA-OPD (entropy-signed updates) and Cliff (09-03) (first-mistake-signed advantages), that is three results in five days modifying the magnitude of an individual update according to how informative it is, all three to protect diversity rather than to chase accuracy. Three subfields, no cross-citations. The unit of policy improvement has become the individual update, and its size is now a designed quantity rather than a global constant.

2026-09-04: the sixth axis is the query itself, and it says every other axis on this page may be optimizing a non-binding constraint

One-Example OPD (09-04) (arxiv 2609.04172, Tsinghua with UIUC, Johns Hopkins, Northeastern, UCAS) trains on-policy distillation on a single query and finds it keeps improving for hundreds of steps and recovers most of full-data OPD's gain across task domains and model families. The explanatory quantity is state coverage, the fraction of the states full-data OPD visits that a query set's rollouts also reach. One query reaches 71.5%, most of it inside the first 100 steps. Sixteen semantically distinct queries reach 98.9% and match full-data training, including in the multi-teacher setting. Meanwhile alignment with the teacher slows at the same rate whether you train on one query or the whole dataset. The diagnosis is a sentence this page should absorb whole: OPD is data-overfed but algorithm-starved.

The stress test is what makes it a claim about mechanism rather than a curiosity. Content-light templates and off-domain WildChat queries also approach the real-query baseline. Task content and induced state coverage come apart, which means the training query's job is not to supply the task and not to supply a verifiable outcome. Its job is to make the student walk through a diverse enough set of reasoning states that the teacher has somewhere to put supervision.

This reframes the five-axis selective-supervision thread that is the longest-running structure on this page. Every axis here is a filter that discards supervision: which tokens (TIP, 04-16, most teacher tokens carry no signal and roughly 10% suffice, extended through TA-OPD, TrOPD, FiRe-OPD, SG-OPD to R2-OPD (08-25)); which layer (OPRD, 06-05, match hidden states not output tokens); which trajectories (OPDVR (08-26), gate on verified correctness); which teacher (QAH (08-26)); which updates (IDA-OPD (09-03), label each update entropy-expanding or contracting). Today adds which queries, and it is the first axis measured on the input rather than on anything the teacher produced, and the cheapest.

The uncomfortable consequence, stated plainly. If alignment slows at the same rate regardless of data supply, a filter that improves which supervision the student sees is optimizing a quantity that was not limiting. Some fraction of the six-axis literature on this page may be reporting gains that a 16-query subset would also produce. That ablation is cheap, it applies to nearly every entry here, and nobody has run it. The page's standing complaint that the field keeps adding filters and never compares them now has a sharper form: the filters may not be the comparison that matters.

It is a clean complement to yesterday's axis and the omission between them is glaring. IDA-OPD's diagnosis was diversity-distillation failure, pass@1 improving while pass@k plateaus because concentrating supervision concentrates the student's output distribution, fixed at the update level with divergence-adaptive advantage shrinkage. Today's paper works on the state distribution the updates are computed over. IDA-OPD makes each step less collapsing; one-shot OPD says you need far fewer distinct queries but many more steps. Both argue the step, not the sample, is the unit. But training on a single query is the most extreme distribution-concentrating setup imaginable, it is the single best test case for IDA-OPD's failure mode, and this paper reports no pass@k at all. Given IDA-OPD published one day earlier, that is the most conspicuous missing measurement on this page.

On the Extrapolation Cliff (05-14), which found a closed-form threshold above which on-policy distillation collapses: state coverage is a plausible mechanistic reading of that threshold. If the student cannot reach the states where the teacher's advantage lives, coverage is low regardless of dataset size and no amount of data fixes it. The paper does not make the connection and it is worth testing.

Separately, a distillation result that is not on any of the six axes at all. Compile by Training (09-04) (arxiv 2609.04199) treats a natural-language specification as source code and compiles it into a reusable neural function: teacher models generate task-specific examples at compile time, those train a small adapter for a compact interpreter, and the artifact then runs with no teacher and can be stored, versioned and composed like software. On FuzzyBench-Hard, where the prior Program-as-Weights fast compiler produced no exact matches at all, it reaches 83.6% semantic accuracy for a compile cost of roughly a minute instead of seconds. For this page the interest is the objective, not the method: it is distillation where the target is a bounded behavior rather than a general capability, and the amortization argument (spend 60x at compile time for an artifact invoked millions of times) is the cleanest cost curve on the page. Read next to One-Example OPD it is a practical existence proof on the same side: a small quantity of generated teacher examples is enough. Neither cites the other and the compose is obvious and unrun, which is to optimize a compiler's example-generation stage for state coverage rather than volume.

2026-09-03: the fifth axis is the first one about the objective, and it exposes a blind spot in the other four

IDA-OPD (09-03) (arxiv 2608.29846, BiliBili / UNC Chapel Hill / USTC / SUFE) adds which updates to this page's list of selective-supervision axes, and it is the first axis measured on the student side rather than the teacher side.

The failure it names is diversity distillation failure, and it has a clean signature: in sampled-token on-policy distillation, the student's pass@1 improves while pass@k plateaus. The student learns the teacher's single most likely answer and never inherits the teacher's spread. Everything downstream that samples more than once, best-of-n, self-consistency, verifier reranking, was buying that spread.

The mechanism is the contribution. First-Order Local Entropy Influence is a signed first-order proxy for how one update will move the student's entropy, and it factorizes into two interpretable terms, the teacher-student log-probability gap and the student's local probability structure. Because it is signed, every update can be labelled entropy-expanding or entropy-contracting before it is applied, and entropy collapse traces empirically to identifiable negative-influence positions. IDA-OPD then does the minimal thing: leave expanding updates alone, replace contracting ones with divergence-adaptive advantage shrinkage. It runs entirely off the sampled-token log-probability vanilla OPD already fetches, so it is precise like the teacher-informed methods (AOPD, EOPD, top-K, full-vocabulary Forward KL) and cheap like the student-side ones (entropy bonuses, relaxed heavy-tailed credits). Pass@k improves consistently, pass@1 broadly holds, and it matches the strongest teacher-informed methods at strictly lower cost. The trade-off the two prior families split between them is dissolved rather than chosen.

The retroactive finding matters more than the method, and this page should carry it as a standing caution. Every one of the four earlier axes selects supervision down to the highest-signal subset: which tokens (TIP, 04-16, most teacher-generated tokens carry no learning signal and roughly 10% suffice, through TA-OPD, TrOPD, FiRe-OPD, SG-OPD to R2-OPD (08-25)), which layer (OPRD, 06-05), which trajectories (OPDVR, 08-26), which teacher (QAH, 08-26). Concentrating supervision on the highest-signal positions is precisely the operation that concentrates the student's output distribution. Selectivity and diversity are in tension, and not one paper in the earlier thread reported pass@k. That is a question mark over a year of results on this page, and the cheapest possible resolution: re-run any of them with pass@k logged.

One methodological improvement worth crediting explicitly. This page generalized a warning on 08-26, that R2-OPD's progress-reward model and VoI-MoLE's reducibility estimator are unvalidated learned models with no sensitivity ablation, so "every method in this family depends on a second estimator nobody has validated." First-Order Local Entropy Influence is a closed-form first-order proxy with an interpretable factorization, not a trained network, so it can be reasoned about analytically. Whether the first-order approximation stays faithful late in training is a fair open question; whether it is a black box is not. It is the first estimator in this family that answers the objection by construction.

Same-day convergence on the unit of credit assignment, and it is the sharper cross-paper pattern. Cliff (09-03) (arxiv 2609.02817) uses an off-the-shelf LLM to locate the first mistake in an RLVR rollout, splits the trace into correct prefix and incorrect suffix, and converts that into positive token-level advantages before and negative after, beating on-policy distillation by 15% and standard GRPO by 7% with teachers of only modest capability. IDA-OPD splits by entropy sign; Cliff splits by correctness onset. Two papers on one day, from RLVR reward design and distillation entropy respectively, both converting a per-position diagnosis into signed token-level advantages, and neither cites the other. The field's working unit of credit has moved from the trajectory to the token, with a sign attached.

Cliff also lands directly on this page's oldest open problem. R2-OPD's diagnosis was that teacher-agreement reward is a bad proxy for reasoning progress, because when the student finds a different valid path agreement falls while progress is fine, punishing exactly the independent correct reasoning you wanted. R2-OPD's answer was a second estimator, which this page objected to. Cliff needs no progress estimator at all, and because it does not assume teacher and student reason alike, the different-valid-path failure cannot arise: an alternative correct prefix is just a prefix with no mistake in it. That is the cleanest resolution this thread has had, and it arrives from outside the distillation literature.

New open problem: Cliff is not checked for the failure IDA-OPD discovered. A method that assigns negative advantage to everything after one point is a strong concentrator of the output distribution, which is IDA-OPD's exact diagnosis. Cliff reports no pass@k. The two papers were published the same day and the first one's finding is the second one's missing experiment.

2026-08-26: the axis count goes to four, and the binding constraint turns out to be the supervisor

Three results in one day, and together they say something this page has been circling all year without stating.

The program on this page had two axes. Today it has four. The long selective-supervision thread argued about which tokens to supervise: TIP (04-16, most teacher-generated tokens carry no learning signal and roughly 10% suffice) → TA-OPD (06-01, only reachable teacher corrections) → TrOPD (06-03, only tokens where the teacher is reliable) → FiRe-OPD (06-04, hard-filter trajectories then soft-reweight tokens) → SG-OPD (06-12) → R2-OPD (08-25). OPRD (06-05) changed the venue to which layer, aligning hidden states and bypassing the LM head on the argument that output-space variance was an artifact rather than a fact. Today adds which trajectories (OPDVR) and which teacher (QAH).

OPDVR (arXiv 2608.24696, LeapLab Tsinghua) answers R2-OPD's load-bearing weakness by refusing R2-OPD's whole approach, and the trade is clean enough to state as a rule. R2-OPD's diagnosis was that on-policy distillation's dense reward is derived from teacher agreement and implicitly treated as a proxy for reasoning progress, and the two come apart systematically: when the student finds a different valid reasoning path, teacher agreement falls while real progress is fine, so the student is punished for exactly the independent correct reasoning you wanted. Its fix built a second, cheap progress-reward model and suppressed distillation reward where the two rankings disagree. This page recorded the problem with that immediately, and generalized it: the progress estimator is an unvalidated learned model with no sensitivity ablation, VoI-MoLE (08-25) has the identical structural flaw in its reducibility estimator, and therefore every method in this family depends on a second estimator nobody has validated.

OPDVR attacks the same failure with a signal that needs no validation, because a verifier is not an estimator. It reformulates sampled-token OPD's implicit reward in terms of trajectory correctness, then applies a ReLU gate so correct trajectories receive non-negative reward and incorrect ones non-positive. The student that reasons divergently and correctly is protected by construction rather than by a model's opinion. Two consequences worth keeping. First, the teacher ceiling comes off: a purely distributional objective bounds the student at the teacher because it contains no notion of correctness, and adding verified outcome removes the bound. Second, the reformulation turns sampled-token OPD into a proper RLVR method, so it composes with any policy gradient including GRPO, and it does all of this with zero new hyperparameters — which is the real argument against the weighted-combination and heuristic-switching baselines it replaces. The honest reading of the pair: R2-OPD generalizes further, OPDVR is trustworthy where a verifier exists. Hold both.

QAH (Multiverse Computing) opens the fourth axis and shows the field's default answer had a structural flaw. After a model is structurally compressed and not merely quantized, no independently trained full-precision version of the smaller architecture exists, so quantization-aware distillation's only candidate teacher is the recovered bfloat16 checkpoint — itself a lossy approximation, which silently caps the student at that checkpoint's ceiling. QAH distills from the original pre-compression model instead, teacher and student sharing neither size nor architecture, which works because a teacher's output distribution is architecture-agnostic. GPT-OSS 120B compressed to 60B and quantized to MXFP4 then beats the 60B architecture's own bfloat16 checkpoint on 7 of 9 benchmarks. The reframing is the contribution: quantization stops being lossy postprocessing after healing and becomes a second full distillation pass against the original teacher, so the 4-bit student is not recovering information quantization destroyed, it is acquiring information the bf16 recovery stage never had time to transfer. A stability property falls out of the loss form too — KL to a fixed teacher stops applying pressure once the student catches up, where a cross-entropy task loss (QAT) keeps pushing toward hard labels and destabilizes if over-trained.

The pattern, now at three independent instances and therefore named. OPRD (06-05) found output-space OPD plateaus below the teacher on AIME 2024/2025 and AIMO. OPDVR states the same bound as a structural property of distributional objectives. QAH finds the compression-recovery version. Three subfields, three mechanisms, one diagnosis: in distillation the binding constraint is usually the supervisor, not the student. The three fixes are correspondingly different — change the layer, add a verifier, change the teacher — and none of them touches student capacity.

DiffusionOPSD (arXiv 2608.24646, ByteDance Seed with NUS and others) is the first entry on this page where the teacher is the student. It converts an image-level reward into explicit targets for clean-output predictions at sampled query states: a frozen behavior policy supplies anchors, reward gradients build bounded positive and negative targets around them, the trainable policy fits those as detached supervision, and an EMA update refreshes the behavior policy. 19 of 20 reward-matched settings won across two backbones and ten evaluators, up to 44.0% over the strongest competitor, and training GPU-hours cut 40% on SD 3.5-M and 63% on the step-distilled Z-Image-Turbo. The larger saving on the step-distilled backbone is the internally consistent detail: fewer denoising steps means fewer places to put targets, so a method that makes each one carry more supervision should benefit most there, and it does.

Set DiffusionOPSD against OPDVR and the day has a shape. OPDVR starts from dense distillation supervision and adds sparse verified correctness; DiffusionOPSD starts from a sparse terminal reward and manufactures dense intermediate supervision. Same HuggingFace page, no shared authors, opposite directions, one shared diagnosis: dense-but-uncorrelated and sparse-but-correct are each half a training signal, and the engineering is in the conversion between them. DiffusionOPSD's separation of target construction from finite realization also produces a result worth carrying: larger target-construction gains do not necessarily yield larger realized gains after one fitting update. A better target is not automatically a better step, which end-to-end policy-gradient methods structurally cannot tell you.

One more granularity for the discardable-supervision thread. OraRL (08-26) prunes at the rollout level, keeping only an oracle rollout plus the strongest rollout of each sign, and more than halves step time (2.2x SFT against GRPO-with-CoT's 4.9x). TIP established it for tokens, AgentOPSD (08-07) for turns, R2-OPD for reasoning spans, OraRL now for whole rollouts. Four granularities, same finding: most of the supervision is discardable. OraRL also contributes a named failure mode this page should watch for elsewhere — advantage inversion, where injecting one very high-reward sample into a group-relative computation lifts the baseline enough to flip good rollouts to negative advantage. That is structurally the mirror of R2-OPD's mimicry bias: both are systematic, non-averaging failures of a relative signal, and both are invisible in aggregate reward curves. The rule to carry: when supervision is relative, adding a strong reference can be actively harmful unless the baseline is decoupled from it.

The field split in two, and one half does not know the other exists (2026-08-16)

One week produced five papers repairing gradient-based teacher supervision and four papers demonstrating the gradient can be skipped. No paper in either group cites a paper in the other.

The repair group. Four papers, one diagnosis (08-16): on-policy distillation is unsound whenever the teacher's supervision is computed over information the student did not actually have. HPSE (2608.11660) finds the student's rollouts never cover newly injected knowledge, so on-policy supervision has nothing to attach to, and builds a hybrid rollout that places missing facts on the student's own trajectory precisely where coverage fails. CMD (2608.13391) finds video distillation pipelines supervising a causal student with a bidirectional teacher that scores complete clips, so the teacher's score depends on future frames the student never saw, and replaces it with a causal teacher plus Prefix Scoring against the student's realized cached prefix. REOPD (2608.11698) makes reward extrapolation reliability-adaptive, which is exactly the measurement Privileged-but-Biased (08-10) demanded when it found a privileged teacher's advantage arrives bundled with a bias the student inherits. ReOrder-OPD (2608.10905) finds prompt order shapes what the trajectory covers. SkillLens (2608.10775) adds retrieval. Five instances, five different misalignments, one unwritten general statement: OPD is sound only when the teacher's scoring function is measurable with respect to the student's realized information set.

The skip group, which the section below opened on 08-13 and which has grown to four in four days. AI4AI at Test-Time (08-13), 0.49 → 0.91 on Theory-of-Mind by writing the target a harness. DarwinX (08-14), +17 points average across four benchmarks by evolving the scaffold with the model frozen. AutoPrune (08-16), an LLM designs a visual-token pruning policy through a 131-atom DSL, 94.4% of tokens removed at >99% accuracy retention, training-free. SKILLER (08-16), a strong model acts as actor and critic in a reinforcement loop where every signal is natural language, producing executor-specific skills that lift Qwen3.5-9B by 4.3 to 20.4 points and match closed models on single-skill tasks.

The comparison nobody has run is now three months old and getting more expensive to keep not running. The repair group is spending real research effort making a gradient-based transfer mechanism reliable. The skip group is demonstrating the transfer can happen through a program at a fraction of the training cost, and reporting no cost figures at all for the strong-model loop that produces the program. Both halves are incomplete in exactly complementary ways. The 08-13 and 08-14 Looking Ahead bullets asked for dollars per point of gain on one axis for both. Still unrun.

One thread does connect them. SKILLER's gains are larger for the 9B model than the 4B (up to 20.4 versus 13.3 points), while AI4AI at Test-Time reported that weaker targets receive the largest gains. Those point in opposite directions within a narrow capability band, and the Extrapolation Cliff (05-14), which found a closed-form threshold above which gradient-based OPD collapses, says the gradient path has a ceiling somewhere in the same region. Whether the program path has a floor where the target is too weak to execute a written policy is now a cheap, concrete experiment.

And a third position exists that neither group occupies, arriving from production rather than from arXiv. Applied Compute's hinting framework (08-16), described by Samuel Denton in a conference talk, keeps the gradient but removes the reference solution. His constraint is explicit and it is the one thing no paper on this page imposes: no golden answer, no reference solution, no hand-written per-task rubric. The teacher's only advantage is a hint, a piece of privileged information the student did not get, and the hint encodes a direction rather than a target. You roll out conditioned on the hint and pull the unhinted policy toward its hinted self.

That is a structural answer to Privileged, but Biased (08-10) rather than another filter on top of it. That paper's causal chain begins with the teacher having seen one particular reference solution, which drags its per-token target toward that trajectory and lands the loss mass on stopwords, punctuation, and uncertainty markers. If there is no reference trajectory, there is nothing for the target to collapse toward. Every one of the nine filtering axes catalogued below accepts a reference-derived target and then decides how much of it to trust; hinting declines the target.

Three further contributions, and the third is the one this page has been asking for since 06-18:

• Relevance masking is an independent rediscovery of the same failure, with a mitigation. Denton uses an LLM judge to select which teacher tokens contribute to the loss, because otherwise you inherit the teacher's connector-word preferences and pay for it in catastrophic degradation. Same diagnosis as Privileged, but Biased, derived from customer deployments, six days apart, no mutual citation.
• Per-step injection is TurnSight (08-05)'s and SMRC-SD (08-10)'s claim with a cheaper estimator. Do not inject the hint at the start of a rollout; use a judge to pick the step, then distil only on the next step or few. Three independent arrivals at supervise at the state the student actually reached.
• The head-to-head. On a customer's out-of-distribution hyperlink-formatting task, reward shaping and SFT on correctly-formatted traces both degraded general coding performance, while online hinting took correct formatting from about 15% to about 80%; a fixed offline hint applied to every rollout barely moved it. This page's standing complaint is nine axes and zero comparisons. This is a comparison. It is n=1, from a vendor, with no regression magnitudes published, so it is evidence rather than proof, but it is the first time the alternatives were run against each other on one task.

One mechanism detail worth keeping. In his SWE-bench experiment, the rollout was conditioned on an off-policy production trace that never called the submit tool, so the teacher could not force the tool-call tokens and instead shifted the reasoning trajectory leading to them: task-complete rate 22% to 60% with test-pass rate flat. The behaviour was installed without ever supervising the tokens that constitute it, which is a stronger version of the selective-supervision thesis than any paper here states: sometimes the right token set to supervise is not the one the behaviour is made of.

The transfer medium stopped being a gradient (2026-08-13)

Every method on this page assumes the artifact carrying capability from teacher to student is a parameter update. AI4AI at Test-Time (2608.12307) proposes that the artifact can be a program instead, and measures it. A strong builder model writes an inference-time harness for a weaker target model, iteratively refining it against a 5% validation split; the harness is then frozen and run on the full test set. Average target performance across four Theory-of-Mind benchmarks goes 0.49 → 0.91. The target's weights are never touched.

Three details make this more than a prompting result. Builder reasoning effort improves harness quality monotonically, so this is a compute-allocation claim: the budget moves from training the student to thinking about the scaffold, and it pays off on a smooth curve. Weaker targets receive the largest gains, which is the same direction the Extrapolation Cliff (05-14) runs in reverse: where gradient-based OPD collapses above a capability threshold, harness transfer helps most below one. And the mechanism analysis rules out the obvious confound: gains do not come from the target reasoning more or sampling more widely, but from offloading unstable reasoning into deterministic code, per-question-type routing, and strict answer-format enforcement. The harness does not teach the target to think better; it removes the places the target's thinking was allowed to leak.

What this page should carry forward. The selective-supervision thread's nine axes all answer "which parts of the teacher's signal are worth a gradient." AI4AI asks whether a gradient is the right vehicle at all, and the two lines have never been compared on a common axis. The comparison is cheap and nobody has run it: for a fixed dollar budget, how much target accuracy does builder inference buy versus fine-tuning, and where do the curves cross? AI4AI establishes the substitution exists and reports no cost accounting whatsoever, not for the builder's refinement rounds, not for per-inference harness overhead, not against a fine-tuning baseline. That omission is this page's standing complaint applied to a new branch: nine axes with zero head-to-head comparisons, and now a tenth vehicle with no price either.

Open question: does harness transfer compose with weight transfer or compete for the same headroom? Directly testable by running an AI4AI harness on top of an already-distilled target. If the gains stack, the two are orthogonal levers and every OPD result on this page is understating its achievable ceiling. If they do not, both are recovering the same missing structure by different means.

And a scheduling axis arrived from Kurate. ReOrder-OPD (2608.10905, Kurate cs.LG #19, tier 1, absent from HuggingFace) proposes reliability-aware prompt ordering for on-policy distillation. Every axis above filters or reweights what the teacher supervises; this one changes the order the prompts arrive in, which is a curriculum claim rather than a filtering claim and the first in the cluster to touch scheduling. It rhymes with From Sweep to Seam (08-12), which found that in post-training quantization the schedule rather than the quantizer separates a usable 1.58-bit model from a broken one. Two papers in two days, in distillation and quantization, both finding order-of-local-operations is the underexploited lever, neither aware of the other. Kurate's tournament had not run at scrape time (all 40 entries at the 1200 baseline, 0% win rate, third consecutive stale week), so this is a topic selection, not a rank endorsement.

Current State (as of 2026-08-11)

The ninth axis arrived within 24 hours of this page predicting it, and it is the first one whose decision variable is an outcome under intervention rather than a statistic of the current step.

The prediction, and its resolution. The 08-10 Looking Ahead said a ninth filtering axis would confirm this page's standing read that the cluster generates variants rather than comparisons. Not Every Divergence Should Be Suppressed (08-11) (2608.04408, Kurate cs.LG #15, tier 1, absent from HuggingFace, Tsinghua) is that axis. Its objection is that divergence is the wrong decision variable, because a large teacher-student disagreement says only that they prefer different actions, not whether the student's error can still be fixed. The replacement is counterfactual recoverability: under a fixed remaining budget, is the teacher more likely to finish the task if it continues from the error, or if the error is rolled back and the branch resampled? Errors where continuing still works are recoverable, so their divergence carries signal. Errors where only rollback works are irreversible, and suppressing divergence there actively reinforces a doomed trajectory, which is a two-sided harm the eight prior axes cannot express.

Why it is more than the ninth variant. This page named the load-bearing split as six-or-seven methods that accept the teacher's target and decide how much to trust it, versus the two that rebuild it from evidence about whether a candidate works (VAD (08-04) decomposes, SPOT reconstructs). Counterfactual recoverability is on the rebuild side, which matters because Privileged, but Biased (08-10) indicts the accept-and-reweight family far more directly. The rebuild branch now has three members and is the only part of the cluster the falsifier does not obviously reach.

It is also the closest thing yet to the unified reliability estimator this page has flagged as missing since 06-18. TA-OPD's teachability, TrOPD's trust region, SG-OPD's sign-consistency and Quality-Aware OPSD's can-this-prefix-still-reach-the-answer gate are four task-specific estimators of one latent quantity, and the coordinate-task version was decidable only because GUI grounding reduces it to ground-truth-box membership. Counterfactual recoverability answers the same question by rollout instead of by task structure, which is what makes it general and also what makes it expensive. It is the explicit computation of what Relay-OPD (07-29) approximates for free, since Relay-OPD's label-free trigger (on failed prefixes the teacher redirects while the student ploughs on) asks "do they diverge on continuation?" where this asks "does continuation still succeed?" Same object, one inferred and one measured, no mutual citation. The page's standing complaint applies with full force: nine axes, still zero head-to-head comparisons, and still nobody prices the teacher calls spent on selection against the teacher calls selection saves.

Separately, a five-day-old prediction on this page resolved in the other direction: SPOT crossed from Kurate-only to cross-source confirmed. The 08-06 entry logged SPOT (2608.04419) as one of two papers that came off the Kurate boards (cs.LG #16, tier 1) without HuggingFace attention, and flagged it as LLM-rated underrated. Today it is on the HuggingFace Daily Papers board and has risen to Kurate cs.LG #8, making it the day's only cross-source-confirmed paper. Nothing about the method changed; the popularity signal took five days to catch up to the quality signal. For a wiki that treats Kurate as the quality channel and HuggingFace as the popularity channel, this is the cleanest measurement yet of the lag between them, and five days is short enough that the Kurate board is worth reading as a leading indicator rather than as a supplement.

One method detail worth adding to SPOT's entry, from the alphaxiv overview. SPOT positions itself specifically against Entropy-Aware OPD (EOPD), which augments reverse-KL training with a top-k approximation of forward KL at high-teacher-entropy positions to preserve plausible local alternatives. SPOT's argument is that teacher entropy is an insufficient statistic on three counts: it cannot distinguish uncertainty concentrated among a few viable candidates from uncertainty dispersed over a long tail, it says nothing about whether the student already represents those candidates, and local teacher probabilities may simply not predict downstream success. That is why the acquisition score combines normalized teacher entropy, the mass captured by a small top-k set, and student-teacher mismatch rather than entropy alone. It also sharpens the reverse-KL critique this page has carried loosely: standard OPD is mode-seeking, favouring the teacher's most probable continuation and starving other correct continuations, which is a coverage failure distinct from the three collapse modes already catalogued here (the Extrapolation Cliff capability threshold, ReOPD's prefix trap, and VAD/CRPO's source contamination).

And the production form of multi-teacher distillation shipped in a technical report. Motif 3 (08-11) (2608.09119) post-trains a 314B-total / 13.2B-active MoE by training six RL specialist teachers plus one SFT software-engineering teacher and folding them into one model with Multi-teacher On-Policy Distillation. All teachers are in-house, which is the pattern this page predicted persists: dense OPD supervises over a shared vocabulary, so a shared tokenizer is required unless you use BPM (07-29)'s byte-prefix marginalization. Motif 3 does not need BPM precisely because it owns its teachers. It also does not ablate MOPD against simply having run seven RL jobs, so whether MOPD consolidates capabilities or averages them remains the open question of the multi-teacher line.

A training-free branch opened on the agent side, and it belongs here rather than on the memory page. AMD (08-11) (2608.07169, KAIST) transfers a large teacher agent's competence to 4B-8B students with no gradient update at all, by factoring successful teacher trajectories into Workflow (task strategy), Subtask (concrete intermediate behaviors) and Function (calling conventions and pitfalls) memories, injecting the first two proactively and retrieving the third reactively on tool-call errors. +27.2pp AppWorld, +11.2pp BFCL V3, +3.4pp ToolSandbox, with Subtask memory contributing most. Read against MAPD (08-02), which compiled a proprietary teacher into a JSON task-plan-facts protocol shown to a privileged student branch, AMD is the same semantic schema as neutral exchange channel move with the distillation step removed: the schema is served at inference instead of being distilled into weights. That makes it the cheapest rung on the neutral-channel ladder this page has tracked from bytes through pixels to editable prose rules, and it reports the capability-gap effect from the memory side, since 4B students benefit most and effectiveness depends on both teacher capability and student compatibility, which is the Extrapolation Cliff (05-14) shape without the gradient.

Prior State (as of 2026-08-10)

The cluster finally got its falsifier, and it does not compare the variants. It attacks the premise underneath all of them.

The result. Privileged, but Biased (08-10) (2608.04794, Kurate cs.AI #18, tier 1, absent from HuggingFace, from Microsoft Research) reproduces SDPO's reported gains in SDPO's easy setting, then applies the identical setup to hard tasks and finds nothing is learned. Across question answering, mathematics, coding and multi-turn agentic tool use, across reasoning modes, model sizes, and forms of privileged information, under both the SDPO and OPSD recipes, the per-token loss falls steadily while validation accuracy stays flat or degrades. The causal chain has four links and each is separately checkable. (1) PI bias: having seen one particular reference solution, the teacher's per-token target is pulled toward that trajectory rather than toward correctness in general, quantified by a new PI Bias Score. (2) Trained to match that target everywhere, the student's objective becomes nearly blind to whether a rollout is correct. (3) The loss it assigns falls mostly on low-information tokens: stopwords, punctuation, uncertainty markers. (4) Within correct rollouts, the exploratory tokens incur the highest divergence, so the objective penalizes exactly the hesitation that reasoning requires. Output: a flatter, less decisive student.

What this does to the seven-axis taxonomy below. Every one of the seven filters exists because the privileged teacher's signal is unevenly trustworthy. This paper says the untrustworthiness has a single named cause and that as a lone objective the family optimizes a signal decoupled from task success. Read strictly, the filters are patching a target that should not be matched. The page's own six-versus-two split is now the load-bearing distinction: six methods accept the teacher's target and decide how much to trust it, while VAD (08-04) decomposes it and SPOT (08-06) rebuilds it from evidence about whether a candidate works. This page predicted that "reweighting a wrong target converges to a wrong place more slowly, while reconstructing it converges somewhere else." The two reconstruction methods are the only ones this result does not obviously indict.

One paper it directly contradicts, and the contradiction is about what to do rather than what is true. SA-OPD (08-06) found that a teacher's extreme-divergence tokens are frequently driven by language priors, formatting conventions and stereotyped templates rather than by the input, and filtered them out. Privileged, but Biased makes the same observation about where the loss mass sits and draws the opposite conclusion: that is not noise at the margin, that is where the objective's mass actually is, so filtering it leaves you optimizing a smaller amount of the same decoupled signal. Same measurement, opposite prescription, four days apart, no mutual citation.

The scope limit that matters, and it is a real one. The failure is demonstrated for self-distillation as a lone objective, with no reward term. That is not the configuration most of this cluster actually ships. RSTG (08-06) distils only on negative zero-variance prompts inside GRPO, precisely the prompts where the whole group failed and the reward gradient vanishes, and reports +4.02% math and +3.05% code. That configuration is untested here and is not obviously covered by the argument, because RSTG uses distillation to fill in where the verifiable reward is silent rather than to replace it. The clean experiment is now specifiable: measure the PI Bias Score inside RSTG's negative-group support. If bias is low there, the whole cluster's correct form is "distil only where the reward signal is dead," and the seven filtering axes were solving the wrong problem.

The eighth axis lands the same day, from the state side, and it is the one TurnSight asked for. SMRC-SD (08-10) (2608.05219, HuggingFace) names state-reference mismatch: in an interactive environment the student's own actions keep changing the execution state, so a globally correct reference trajectory is only locally valid at the pre-action states it was recorded from. SMRC-SD verifies per turn whether the student's current state is supported on the reference, distils only at matched states, and at those states builds teacher context conditioned on the state actually reached. ALFWorld 0.746 to 0.865, WebShop 0.574 to 0.693, at Qwen3-1.7B. This is TurnSight (08-05)'s dissent implemented: TurnSight argued the standard privileged context is the wrong context because it derives from the ground-truth answer rather than from the state the agent reached, and this page called that "the most important claim in the cluster." Five days later it has a method and the method works.

And here is the tension the cluster now has to resolve. SMRC-SD fixes a sibling of PI bias, mismatch between reference state and student state, but not PI bias itself: at matched states it still derives its target from one particular reference solution. Either state matching incidentally defuses PI bias, which would be a strong and testable claim nobody has made, or SMRC-SD's ALFWorld and WebShop gains sit inside the same easy-setting regime where Privileged, but Biased also reproduces gains. Two papers surfacing on the same day, one saying the mechanism does not work on hard tasks and one reporting +12 points, and nothing in either can distinguish the two explanations. That is the single most important open question on this page.

Author-tracking note, to prevent a future merge error. Today's Kurate rising-author board names Junlin Liu at score 17.0 with three top-10 appearances, including "Contrastive Reinforced Policy Optimization via Privileged Self-Distillation," which is the CRPO line this page has tracked since 08-04 and attributed to overlapping Meituan author groups. SMRC-SD's first author is Junzhuo Liu at the University of Electronic Science and Technology of China. Different people, near-identical names, same problem. Do not merge them.

Prior State (as of 2026-08-06)

Seven on-policy distillation papers in one day, and the cluster's central question now has seven mutually untested answers. The page should stop counting instances and start naming what would settle it.

The count, because it matters. Five landed on HuggingFace: SA-OPD (2608.03632, cross-source confirmed on Kurate cs.AI #11), RSTG (2608.00782), Poly-OPD (2608.04349), OPD-V (2608.05131), and SKILL-KD (2607.28048). Two came off the Kurate boards without HuggingFace attention: SPOT (2608.04419, cs.LG #16, tier 1) and CRPO's re-ranking at cs.LG #3. On-policy distillation is now producing roughly one paper per day per filtering axis, and the 08-05 Looking Ahead prediction that the cluster would either produce a head-to-head comparison or reveal itself as generating variants is resolving toward variants.

The seven axes, in the order they appeared. Every one accepts that dense teacher supervision is unevenly trustworthy and differs entirely in what it conditions trustworthiness on. Position: CRPO (08-04) sorts by predictive entropy, on the finding that a privileged self-teacher spikes into overconfidence right after a tool call returns. Direction: VAD (08-04) projects the correction onto a signed counterfactual visual-evidence axis and discards the unexplained residual. Time: PCSD (08-05) weights by how persistently teacher-favouring signal holds across an adaptive window. Turn structure: TurnSight (08-05) keeps only what multiple lookahead horizons agree on. Input-groundedness: SA-OPD filters tokens that are simultaneously weakly input-dependent and extreme in divergence, on the claim that a teacher's token judgment is often driven by language priors, formatting conventions, or stereotyped reasoning templates rather than by the input. Modality balance: OPD-V builds a Positive Teacher (zoom-in crop) and a Negative Teacher (masked image), reads the logit margin between them, and distils only inside the resulting trust region. Realized outcome: SPOT probes a budgeted set of positions, has the teacher propose candidates, runs verifier-scored student continuations on them, and builds a closed-form KL-regularized target from the outcomes.

The one distinction that actually separates them, and it is not the axis. Six of the seven accept the teacher's target and decide how much to trust it. Two do something structurally different. VAD decomposes the target and throws part of it away. SPOT rebuilds the target from evidence about whether the candidate works. That is the split worth tracking, because reweighting a wrong target converges to a wrong place more slowly, while reconstructing it converges somewhere else. The page previously flagged VAD as the first paper to change what the target is rather than how much it counts; SPOT is the second and goes further.

SA-OPD generalises VAD's mechanism out of vision, which the page predicted implicitly and should now state. The page derived a principle from TurnSight: a privileged signal is trustworthy to the extent that it survives perturbation of the privilege. VAD perturbs an image crop. OPD-V perturbs it in both directions at once. SA-OPD perturbs input dependence in general, which is the version that runs on pure text, and it is the first paper on this beat to make perturbation-survival its central mechanism for the general LLM case. Three of today's seven are instances of one principle nobody has written down as a paper.

RSTG partially resolves the CRPO-versus-ReCo conflict, and it resolves it exactly the way this page guessed. The page hypothesised that ReCo protects exploration coverage against a base-model prior while CRPO protects supervision reliability against a confidently wrong teacher, so "they select on different supports." RSTG makes the support separation structural: distil only on negative zero-variance prompts, the ones where GRPO's group all failed and the gradient vanishes entirely, because on those prompts there is no exploration left to crowd out. Inside that support it targets high student entropy or large teacher-student divergence, which is the ReCo side of the same statistic, and it adds SFT on teacher-generated correct trajectories to supply positive gradient where OPD's asymmetric advantage supplies mostly suppression. +4.02% math, +3.05% code over naive GRPO+OPD. The 08-04 Looking Ahead asked for a third paper on this conflict within 60 days and got one in two, though not the paper it asked for: RSTG separates the supports instead of reporting a coverage metric and a reliability metric on the same runs, so the question of what high entropy means is sidestepped rather than answered. The clean test is now obvious: run CRPO's entropy sort inside RSTG's negative-group support. If it still helps, the support explanation is wrong.

RSTG's three diagnosed causes are the most reusable thing published on this beat today. Naive GRPO+OPD degrades because (1) teacher guidance quality varies per sample, (2) fitting the teacher fast forecloses RL's only upside, which is exceeding the teacher, and (3) OPD's token-level advantages are asymmetric, since student-generated tokens usually get low teacher probability, so the signal suppresses rather than guides. Cause 3 is a general property of OPD-on-RL, not a quirk of this setup, and it belongs on this page independent of RSTG's method.

The neutral-exchange channel reaches its tenth entry and the ladder now has a human-readable top rung. Any-OPD (08-05) took the channel fully external to both parties, comparing independently decoded outputs in a frozen model-agnostic vision representation. Poly-OPD, one day later, shows that once the channel is external, teacher count stops being architecturally constrained: two heterogeneous teachers (FLUX.1-dev and Z-Image) into one 2.5B SD3.5-Medium student via a pixel bridge plus frozen DINOv2 space, lifting GenEval 67.3 to 73.3 above both larger teachers and DrawBench HPSv3 9.34 to 11.35. And SKILL-KD moves the channel to editable natural-language rules, distilling the actionable discrepancy between a student failure and a teacher success into a textual skill patch, verifying the patch by re-running the student, and consolidating patches with trace-linked edit histories to prevent skill drift. Bytes, probability spaces, JSON schemas, pixels, now prose rules. The page's standing caution holds: there is no unique lowest common substrate for reasoning, so each rung above bytes is a bet, and none of them ablates how much competence fails to fit.

Poly-OPD's gradient-compatibility diagnostic got an independent witness on the same day from an unrelated problem, and this is a genuine cross-paper finding. Poly-OPD measures gradient agreement across teachers and concludes attention LoRA modules can be shared while feed-forward adapters must stay teacher-specific. Physics of Multimodal Pretraining (08-06) ablates architectures across modalities and concludes that shared attention and normalization with modality-specific feed-forward layers is what turns competition into synergy, holding across visual tokenizer designs, validated on 13.5B MoE models over 2T tokens. Same partition, two methods, two problems, one day, no mutual citation. If the principle is real it says attention is a source-agnostic mixing operation and the feed-forward layer is where source-specific knowledge lives, which would predict that every multi-source training method on this page should be splitting FFNs and sharing attention.

Vision laziness connects OPD-V's post-training finding to a pretraining cause. OPD-V's diagnosis is modality imbalance: when text dominates generation, the model does not integrate visual input, so privileged visual information goes unused regardless of how well it was constructed. Physics of Multimodal Pretraining names vision laziness: delaying modality integration to a late alignment stage teaches the model to lean on language priors. One paper locates the cause in the pretraining schedule and the other measures the consequence in post-training, on the same day, neither aware of the other. The actionable question: does a model pretrained with early unification need OPD-V's trust region at all?

Where the expensive signal is allowed to fire, now on four axes. TIP (04-16) restricted it on the token axis (under 10% of teacher tokens carry signal). Relay-OPD (07-29) on the trajectory axis (training length halved). ReOPD (08-03) on the environment axis (zero tool calls during student training, at least 4x faster rollouts). RSTG adds the prompt axis: most prompts get no teacher call at all. Nobody in the four has priced the teacher-call saving, which is the first number a practitioner pairing GRPO with a frontier-model teacher would ask for.

Industry contradiction worth logging, because it is about the word rather than the technique. ByteDance's founder Zhang Yiming told an internal all-hands that the company will not use distillation as a shortcut, even if that means lagging domestic rivals, with the TikTok history cited as a reason. On the same day, ByteDance authors are on SA-OPD, the day's cross-source-confirmed leading on-policy distillation paper. Those are not in conflict, and the reason they are not is the confusion this page has tracked since the Distillation Panic (05-04) entry: "distillation" names two different things. Copying a rival frontier model's outputs to shortcut capability is an IP and provenance question. Compressing a teacher you own into a smaller student you serve is an efficiency technique. The policy fight, and now a founder's strategy statement, are using the word in the first sense while the entire technical literature on this page uses it in the second. Anyone reading either as evidence about the other is making a category error.

Prior State (as of 2026-08-05)

Seven privileged-teacher papers in four days. The pattern is no longer the finding; the internal disagreement is. Four groups now filter the same privileged signal along four different axes, and one of them argues the privileged context itself has been built wrong all along.

The four axes. Every paper in this cluster accepts that a privileged teacher, meaning the same policy handed information the deployed model will not have, produces dense but unevenly trustworthy supervision. They differ entirely in what they condition trustworthiness on. CRPO (08-04) filters by position, sorting by predictive entropy. VAD (08-04) filters by direction, projecting the correction onto a counterfactual evidence axis. PCSD (08-05) filters by time, deriving weights from the local persistence of teacher-favoring signal across an adaptive window, on the claim that teacher reliability is autocorrelated rather than per-token independent. TurnSight (08-05) filters by turn structure, building multiple hindsight views at different lookahead horizons and keeping only what they agree on the direction of. None of the four evaluates against any of the others.

The dissent, and it is the most important claim in the cluster. TurnSight argues the standard privileged context is the wrong context. Existing methods derive it from the ground-truth answer or from a retrieved skill library, and neither describes the state the agent actually reached, so the teacher's confidence is about the answer rather than about the agent's situation. TurnSight instead conditions on realized execution. If that is right it partially undercuts the other six rather than complementing them, and nobody has said so. It also generalizes VAD's mechanism into a principle: a privileged signal is trustworthy to the extent that it survives perturbation of the privilege, whether that perturbation is removing an image crop (VAD) or extending a lookahead horizon (TurnSight).

PCSD may subsume CRPO. Both come from overlapping Meituan-and-Junlin-Liu author groups. CRPO's finding, that the teacher spikes into overconfidence right after a tool call returns, describes an isolated, non-persistent signal, which is exactly what PCSD's persistence window down-weights by construction. PCSD is plausibly the more general mechanism with CRPO's entropy heuristic as a special case in the tool-call setting. Testable, untested.

It also reframes the unresolved CRPO-versus-ReCo tension. ReCo (Kurate cs.LG #19) upweights the high-uncertainty positions CRPO discards, because one is protecting exploration coverage and the other supervision reliability. If reliability is a property of a run of positions rather than a position, as PCSD argues, those two instructions stop being contradictory because they select on different supports. Still nobody has reported a coverage metric and a reliability metric on the same agentic runs.

Separately, the neutral-exchange-channel pattern reaches its eighth entry and its logical endpoint. Any-OPD (08-05) does on-policy distillation between arbitrary latent flow-matching model families that "share nothing but pixels," treating the teacher as a pure black-box sampler and connecting the two models at exactly one point, a frozen model-agnostic vision representation in which their independently decoded outputs are compared. Trajectory correspondence comes from matching continuous noise levels rather than step indices, which is a small correction with broad reach, since every schedule-based distillation method on this page aligns by step. FLUX.1-dev 12B into SD3.5-Medium 2.5B lifts PickScore 0.846 to 0.884 and HPSv3 9.12 to 10.97, where direct latent regression fails to train at all. After BLD (bytes), TESSY (interleaving), Switch-KD (shared text-probability space), Tide (bounded-gradient cross-arch), CoPD (bidirectional OPD), D-OPSD (conditioning asymmetry), the channel is now entirely external to both parties. It also has a policy edge: output-only cross-family distillation working undercuts any regulation premised on distillation requiring teacher internals, which is the technical core of the Distillation Panic (05-04) argument.

Prior State (as of 2026-08-04)

The privileged-branch pattern hits its third and fourth instance in three days, so it is now named. And two papers on the same day report the same failure inside it: a privileged teacher is wrong in a way that correlates with where the student needs help most.

The pattern. MAPD (08-02) showed a JSON reasoning protocol only to a privileged branch of the student, whose token distributions then supervised a deployed branch that never reads the protocol. CriPO (08-03) used a criterion-injection self-teacher, the same policy under a privileged prompt, to repair a suppressed gradient. Today adds VAD (08-04), whose teacher is privileged with a visual crop, and CRPO (08-04), whose self-teacher is privileged with agent-task information. Four instances, and the distinguishing question is what the privileged branch is privileged about: a task schema, a rubric criterion, a visual region, agent state. Call the pattern privileged-branch target construction. It is now the dominant way this page's papers build a supervision signal.

The shared failure, stated two ways on one day. CRPO (2607.28026, Kurate cs.LG #2) diagnoses that because the self-teacher holds privileged information it becomes overconfident exactly at the positions where the student is genuinely uncertain, which in agent tasks is right after a tool call returns new information. Two harms follow: the teacher's reasoning routes converge onto patterns in the demonstrations so the student generalizes worse, and multi-turn optimization directions become unclear because position-level supervision is unreliable. The fix sorts positions by predictive entropy into positive positions where high entropy reflects genuine reflective exploration and negative positions where it reflects exposure bias, then contrasts them group-wise so only reliable fine-grained signal survives. Gains across 13 reasoning and deep-search benchmarks over both RL and self-distillation baselines. VAD (2607.28590, top of today's HuggingFace board at 18 upvotes) says the multimodal version: a teacher correction is source-mixed, bundling visual signal with linguistic priors and teacher-specific effects, and the paper's diagnostic reports that a substantial share of a teacher's strongest corrections are not well aligned with its own evidence-conditioned response. That invalidates the premise of Vision-OPD, which conditions the teacher on an evidence crop and then distils the complete next-token distribution on the assumption that visual conditioning makes the correction visual.

These two are the same claim about privileged teachers from opposite directions, and together they add a third collapse mode to this page. The Extrapolation Cliff (05-14) is a capability-gap threshold above which on-policy distillation collapses because the student is too far below. The prefix trap from ReOPD (08-03) is the multi-turn fact that student occupancy and teacher reliability move in opposite directions. The new one is source contamination: the teacher is competent, on-distribution, and its target still carries signal from a channel you never meant to transfer. Three modes, three causes, and only the third has a per-position detector, which is why CRPO's entropy sort and VAD's counterfactual projection are the interesting machinery rather than the benchmark numbers.

VAD's mechanism, because it is the sharper of the two. Run the same fixed teacher twice at each student prefix, once with the relevant evidence present and once with it removed. The change in centered log-probabilities defines a signed direction u_t along the visual evidence. Project the original correction onto u_t, keep the aligned component, discard the unexplained residual, and rebuild a student-anchored target from the aligned part, demoting the privileged teacher to a weak regularizer. The sign is what separates it from the visual-advantage line (VA-OPD, V-Zero), whose positive advantage captures "the evidence supports this alternative" and structurally misses "the evidence refutes what the student just said", and refutation is exactly where VAD's gains concentrate. Six fine-grained visual benchmarks at 4B and 9B.

One clarification on the sign. The visual-advantage line uses a positive advantage, which captures "the evidence supports this alternative" and structurally misses "the evidence refutes what the student just said." VAD's signed direction captures both, which is why its gains land on refutation cases.

This is a fifth mechanism in the token-weighting thread and the first to change what the target is rather than how much it counts. TIP (04-16) found under 10% of teacher tokens carry signal, LongAct (04-18) weighted by activation magnitude, Relay-OPD (07-29) by continuation divergence, CoRT (07-30) by counterfactual likelihood for +4.4 points. All four reweight a target they accept. VAD decomposes the target and discards part of it, which is a different operation: reweighting says this token matters less, decomposition says this token's target is partly wrong. CoRT is the closest sibling and the contrast is clean, since both are counterfactual but CoRT's counterfactual is over token replay to assign credit while VAD's is over evidence presence to attribute direction. They compose trivially and nobody has composed them.

One safety-adjacent note. ROPD (08-04) (2607.27081, Kurate cs.AI #18, tier 1) uses on-policy distillation for safety realignment after malicious fine-tuning, modelling the divergence between aligned and compromised output distributions rather than fitting a specific prompt template, which is what buys robustness when the defender cannot observe the attacker's template. It belongs on the responsible-ai page but it is worth logging here as the first use on this page of distillation to remove a capability rather than transfer one, and the reported result that existing defences cause catastrophic forgetting of the specialized skill is the same tradeoff-under-a-privileged-signal shape as everything above.

Three additions to the above. First, CRPO's cost argument, which is the reason to prefer it over the existing fixes. RLSD, SDAR and RLCSD all address exposure bias by bolting an RLVR objective onto the self-distillation objective, and pay for maintaining two optimization frameworks. CRPO reformulates OPSD contrastively instead and adds no second framework, though note the no-extra-cost claim is made against those hybrids rather than against plain OPSD, which is the baseline that matters.

Second, CRPO and ReOPD stack cleanly and nobody has stacked them. ReOPD (08-03)'s step-decaying schedule chooses which prefixes to train on; CRPO's entropy contrast chooses which positions inside a prefix to trust. Both are reweightings over the same rollout data, so composition costs nothing beyond bookkeeping. Same for VAD and CoRT (07-30), already noted above as trivially composable.

Third, and this is a genuine unresolved conflict rather than a gap: ReCo (Kurate cs.LG #19, 2607.26862) prescribes the opposite of CRPO on the same statistic. ReCo attacks GRPO for concentrating on responses the base model already generates with high probability, traces it to two update terms (high-probability responses dominating the group gradient by repetition, and the importance ratio scaling gradients toward already-likely tokens), and fixes it by normalizing response contributions by expected occurrence and replacing the token-level importance ratio with a variance-based ratio that upweights non-saturated decision points where alternative tokens remain plausible. Those are the high-entropy positions CRPO contrasts away. The likely reconciliation is that ReCo protects exploration coverage against a base-model prior while CRPO protects supervision reliability against a confidently wrong teacher, in which case an agentic post-training recipe needs both signals plus a way to tell the two kinds of uncertainty apart, which nobody has proposed. If that is not the reconciliation, one of them is wrong about what high entropy means, and the variable that would settle it is whether CRPO's filtered positions show the Pass@k-at-large-k degradation ReCo measures.

Current State (as of 2026-08-03)

A second, distinct collapse mode for on-policy distillation, and this page should stop bundling it with the first. ReOPD (08-03) (2607.04763, Microsoft Research and University of Amsterdam) studies multi-turn agentic OPD and names the prefix trap. Pushing interaction histories toward the student's own distribution makes them more relevant to the student and simultaneously drags the teacher onto states where its targets are unreliable, so student occupancy and teacher reliability move in opposite directions. This is not the Extrapolation Cliff (05-14) failure, which is a capability-gap threshold above which OPD collapses. It is the multi-turn-specific fact that a student can construct histories no competent policy would have produced, and a teacher conditioned on an incoherent history is not a teacher. Two collapse modes, two causes.

The method treats prefix selection as reliability-aware distribution design and implements it bluntly, with a step-decaying sampling schedule that emphasizes early, lower-shift prefixes rather than matching the student everywhere. Teacher trajectories are collected once and replayed as prefixes, the student acts at selected steps, and the teacher supplies dense per-step supervision. Across math-with-Python and search environments over multiple teacher and student scales: zero tool calls during student training, at least 4x faster per rollout, accuracy preserved or improved. Two notes for this page. It is the same shape of result as TIP (04-16), which found most teacher tokens carry no signal and about 10% suffices, except cutting waste on the environment axis rather than the token axis. And it composes almost exactly with MAPD (08-02) below, whose largest unpriced component is an offline multi-agent pipeline run per query before training: ReOPD's entire argument is that pre-collected teacher trajectories amortize across many student updates, which weakens that objection considerably. Neither paper cites the other. → summary

Self-distillation shows up as a gradient-repair tool, not a compression tool. CriPO (08-03) (2607.18082, Zhejiang University and ByteDance) uses on-policy self-distillation from two self-teachers, both being the same policy under a different prompt, to fix rubric-based RL. The measurement worth remembering here is that over 57% of samples contain a criterion the model satisfied whose signal was destroyed by scalar advantage aggregation. The structural note for this page is that CriPO's privileged self-teacher is the second use in two days of a privileged-information branch supplying dense supervision to a deployed branch that never sees it, after MAPD's privileged student branch reading the JSON protocol. A third instance would make it a named pattern. → summary

Current State (as of 2026-08-02)

The neutral-representation pattern reaches four papers, and it has been climbing the abstraction stack the whole time. MAPD (08-02) (2607.24280, Kurate cs.AI #12) distils a closed proprietary teacher for agentic search, where logit matching is impossible because logits are hidden and tokenizers differ, and copying natural-language trajectories transfers writing style rather than reasoning competence. Its intermediate is a JSON protocol holding task type, reasoning plan, and extractive grounding facts, compiled offline by a multi-agent system that decomposes the query, retrieves evidence, and repairs failed searches. The protocol is shown only to a privileged branch of the student policy, whose token distributions then serve as a dense distillation target alongside the sparse RL objective, so the deployed policy learns to reach the protocol's reasoning without ever reading it. 39.4% average success on Qwen3-1.7B and 44.4% on Qwen3-4B across seven QA benchmarks, with style drift and verbosity degeneration explicitly reported as suppressed, and, the part that logit matching structurally cannot do, robust generalisation across different proprietary teachers.

Line up the neutral layers this page has logged and the direction is unmistakable. TESSY (04-18) used hybrid token sequences. Switch-KD (04-18) used a shared text probability space. BPM (07-29) used bytes, the unique lowest common substrate two tokenizers share, recovering the byte-prefix marginal exactly at over 99% of training positions. MAPD uses a semantic schema. Token sequences, probability spaces, bytes, then a task-plan-facts protocol: the neutral layer has moved from a lossless re-encoding of the teacher's output to an interpretation of it. That is the interesting move and also the reason to discount the result somewhat, because there is no unique lowest common substrate for reasoning, so a fixed schema is a bet and no ablation reports which of the three protocol fields carries the signal or how much competence fails to fit.

It is also the second paper here whose teacher is a system rather than a model. CAST (07-30) removed the requirement that the teacher be a neural network. MAPD's teacher is an offline multi-agent pipeline with a repair loop, which produces better traces than any single call to the underlying model would, so the teacher is now allowed to be more capable than the model it is built from. That widens the page's running conclusion again: on-policy distillation is a general statement about transferring any structured or scalar-valued source of state quality into a policy, and the teacher's identity keeps getting less constrained.

One tension the page should hold open. MAPD's privileged branch supplies a dense signal over the whole sequence, which is exactly the uniform-credit pattern the four-paper token-weighting thread below spent three months arguing against (TIP under 10% of tokens, LongAct activation magnitude, Relay-OPD continuation divergence, CoRT counterfactual likelihood for +4.4 points). Composing a MAPD protocol with CoRT per-token weighting is obvious and unrun.

Policy context, which is new for this page. Distillation is now the subject of an explicit regulatory fight. Three open letters in five days (08-02): Microsoft's 235-signatory open-weights letter goes out of its way to defend distillation as "a widely used technique for model improvement, evaluation, and validation," while Anthropic declined to sign and asked for a crackdown on industrial-scale distillation operations. The technical record on this page is the reason that ask is already behind. BPM removed the tokenizer constraint, CAST removed the neural-teacher constraint, and MAPD removes the logit constraint entirely while needing nothing but API access. A policy aimed at industrial-scale distillation is aimed at a technique that just stopped requiring industrial scale.

Current State (as of 2026-07-30)

On-policy distillation lost its third and largest structural constraint in eight days: the teacher no longer has to be a neural network. CAST (2607.25308) observes that in any domain with a classical solver, the change in the solver's state value from one state to the next already says whether an action advanced the position, which is exactly the turn-level credit signal RLVR's sparse final reward withholds. The result that matters is the derivation: under a soft-optimal solver assumption, maximizing the solver advantage is equivalent to on-policy distillation from the solver, requiring only scalar values rather than teacher logits. That collapses the teacher's bandwidth requirement from a vocabulary-sized distribution per token to one number per turn, which is why a Sokoban solver with no tokenizer and no logits can serve as an OPD teacher at all.

Line the month's three results up and the pattern is explicit. BPM (07-29) removed the shared-tokenizer requirement via byte-prefix marginalization. Relay-OPD (07-29) removed the verifier requirement, using teacher-student continuation asymmetry (on a failed prefix the teacher changes course while the student ploughs on) as a label-free handoff trigger for +5.73% and half the trajectory length. CAST removes the requirement that the teacher be a model. What remains of on-policy distillation is a very general statement about transferring any scalar-valued source of state quality into a policy, which is a much larger surface than the literature has been working over. CAST's evidence that this is not merely a game trick is the zero-shot transfer: trained only against solvers on Sokoban, Minesweeper and Rush Hour, it posts the highest average zero-shot performance on ALFWorld and WebShop, which have no solvers.

The token-weighting thread crosses the pattern threshold at four papers. CoRT (07-30) (2607.25659, Nanjing/ByteDance) attacks the same waste from inside the policy. Rubric-based RL decomposes evaluation into explicit written criteria and GRPO then flattens all of it into one response-level advantage broadcast uniformly to every token, even though a formatting criterion and a correctness criterion live in different spans. CoRT replays the same sampled response under the rubric-conditioned prompt and a matched criteria-free prompt and uses the per-token log-likelihood contrast as a proxy for rubric dependence, mapping those contrasts to bounded response-normalized weights that redistribute the signed advantage. No auxiliary scorer, unlike the prior Rubrics-to-Tokens approach which trains a separate relevance model. +4.4 percentage points average over matched response-level GRPO.

Four papers now make the same core claim from four different signals: uniform credit across a trajectory is the waste. TIP found most teacher-generated tokens carry no signal and roughly 10% suffice (teacher agreement). LongAct (04-18) restricted RL gradients to high-magnitude Q/K activation positions for about 8% on LongBench v2 (activation magnitude). Relay-OPD used continuation divergence. CoRT uses counterfactual likelihood. That crosses this wiki's three-paper bar, so the pattern is established rather than emerging. The open question none of them answers is whether the four signals agree with each other on the same rollouts; they are cheap enough that the correlation study is overdue.

Open flag on CoRT: the confound is whether the contrast measures rubric dependence or merely prompt sensitivity, since a token whose likelihood drops when any long prefix is removed gets upweighted for the wrong reason. A placebo-prompt ablation would settle it and is not run.

Current State (as of 2026-07-29)

The reliability estimator this page has called missing since 06-18 gets its first label-free instance, and the shared-tokenizer assumption falls the same week. Two papers, one from HuggingFace and one from the Kurate weekly board, move the line on its two oldest walls.

Relay-OPD ("Pass the Baton", 2607.26057) attacks prefix failure, the failure mode TRD (06-09) named: once the student commits to a wrong early step, the teacher's supervision on everything after it is scoring a doomed path. Its contribution is not the fix, it is the trigger. On failed prefixes the teacher tends to redirect while the student persists, and that teacher-student continuation asymmetry is computable from the two models alone. No verifier, no answer key, no reward model. That matters more than the numbers, because every prior estimator on this page (TA-OPD's teachability, TrOPD's trust region, SG-OPD's sign-consistency, Quality-Aware OPSD's can-this-prefix-still-reach-the-answer, ReOPD's prefix trap) needs supervision to fire, which is why the whole line is stuck in math and code. The asymmetry is coarse (binary and trajectory-positional rather than graded and per-token), so it is not the unified estimator; it is the first member of the family that could run outside verifiable domains. The fix itself is a middle path between the two existing responses: FiRe-OPD (06-04) discarded bad trajectories, TRD repaired them, Relay-OPD hands control to the teacher for a short budgeted leg and takes it back. Results: +5.73% over standard OPD and +1.49% over FastOPD at Qwen3-1.7B across eight math benchmarks, best or second-best on every one, with training trajectory length cut by over 50%. Both the accuracy and the compute saving come from the same source, which is not generating tokens that build on a dead prefix.

Read against Requential Coding (07-25), which proved the bits a teacher actually transmits are charged only where teacher and student disagree, Relay-OPD is the first paper on this page to use disagreement as the selection criterion rather than as an after-the-fact explanation. The 07-25 entry called closing that loop the open move nobody had made. It is now closed at the trajectory level and still open at the token level.

Cross-Tokenizer OPD via Byte-Prefix Marginalization (BPM, 2607.22334, Kurate cs.LG #16, ai_rating 7.0/10) removes a constraint the page has silently inherited from every multi-teacher result it logs. Dense OPD supervises with a distribution over a shared vocabulary, so teacher and student must share a tokenizer, which is why Nemotron 3 Ultra (06-16) and every MOPD run the Interconnects 06-16 podcast named (DeepSeek V4, MiMo-V2-Flash, ten-plus teachers) used in-house teachers. Prior cross-tokenizer methods either discard unmappable teacher mass, producing a biased target, or map it onto student tokens with unrelated content, injecting noise. BPM maps into byte space, the substrate every tokenizer is built on: each teacher token's probability goes to the longest student token whose bytes are a prefix of the teacher token's bytes, colliding mass is summed, and unmatched mass goes to an explicit residual category rather than vanishing. The target is vocabulary-complete and mass-preserving, and exactly recovers the byte-prefix marginal at more than 99% of training positions (measured, with a mass-preserving chain-factorized lower bound for the remainder). Across Qwen3-32B, GLM-Z1-9B, and MiniMax-M2.7 teachers: +3.7 to +6.6 six-benchmark avg@8 over the strongest baselines.

Three papers now occupy distinct answers to "where does the distillation signal live." OPRD (06-05) says the output vocabulary is the wrong venue and aligns hidden states instead. BPM says the vocabulary is fine if you translate it correctly. Relay-OPD says the vocabulary is fine and the problem is where in the trajectory you apply it. BPM is the most immediately deployable because it needs only the teacher's output distribution, no architectural access.

The industry counterpart landed the same week and is not a coincidence. Ben Lorica's essay (07-28) reports 25-plus startups building the reinforcement-fine-tuning stack so enterprises can turn open weights into specialized models they own, with 10 to 30 point gains on well-defined tasks. Consolidating the open ecosystem into one owned student is exactly what BPM makes technically clean, and the verifier-and-grader layer those startups are commoditizing is the missing component this page has flagged as the reason every result here is confined to math and code.

Current State (as of 2026-07-25)

The page's central empirical claim gets an information-theoretic proof, from a paper that was never trying to do distillation. Since TIP (04-16) found that under 10% of teacher-generated tokens carry usable learning signal, every result on this page has been a different estimator of the same latent quantity: which teacher outputs are worth transmitting. TA-OPD called it teachability, TrOPD called it a trust region, SG-OPD called it sign-consistency, Quality-Aware OPSD called it can-this-prefix-still-reach-the-answer, and ReOPD (07-24) generalized it to whole histories. Requential Coding (2607.11883, Qiu / Finzi / Zheng / Zhang / Wilson, NYU and CMU) says what the quantity is. It compresses a model by having a teacher select samples drawn from the student's own distribution and recording only the selections, so bits are charged only where teacher and student disagree. The resulting code length is independent of both parameter count and data entropy, and is often orders of magnitude shorter than the prequential code, with the gap widening as scale grows.

Two consequences for this page. First, it supplies the reason the whole selection line works: if the code that measures transferred information charges nothing at agreement points, then a distillation procedure that transmits agreement points is provably paying for nothing, and TIP's "<10% of tokens" is not a heuristic but the expected shape. Second, it makes the unified reliability estimator the page has flagged as missing since 06-18 look like a compression problem rather than a supervision problem: the estimator everyone is approximating is "how many bits does this teacher output actually move," and requential coding gives a way to write that down.

The paper is not a distillation method and does not produce a smaller servable artifact. It is a measurement scheme feeding a PAC-Bayes generalization bound, where it reaches state of the art for billion-parameter LLMs, beating quantization-based bounds even when quantization is granted zero error. The finding that most stresses this page's neighbours: holding loss fixed, larger models and ensembles compress to smaller codes despite more parameters, which is the opposite of what parameter-count-based complexity proxies (the ones underneath most pruning and quantization work) predict. It is the natural theoretical partner to Dataset Distillation by Influence Matching (07-24), already noted there: both compress by generating a small high-value dataset rather than shrinking weights, one to save training compute, one to measure complexity.

Provenance note: Kurate cs.LG #7 (ai_rating 8.5/10), flagged as LLM-rated-underrated in the 07-22 and 07-24 digests and never surfaced on HuggingFace across three weeks on the leaderboard.

And the same day, a paper builds a computable instance of exactly that quantity. VCSD (Visual Contrastive Self-Distillation, 2607.21556, UMD / UCSD / Duke / MBZUAI) asks whether the teacher-student asymmetry that on-policy self-distillation requires can be built without handing the teacher any privileged information. Its answer is to build the asymmetry out of removal rather than addition: an EMA teacher runs the identical prompt and identical student-generated prefix twice, once with the real image and once with a content-erased control, and the token-wise log-probability difference isolates the tokens whose likelihood was raised by that specific image's content rather than by the language prior. That difference is then used to sharpen the teacher's real-image distribution inside its own plausible support, which is the guardrail against the standard contrastive-decoding failure of promoting tokens the model considers implausible overall. Seven-benchmark aggregate on Qwen3-VL: 62.27 to 67.04 at 2B, 71.30 to 73.16 at 4B, 72.51 to 76.26 at 8B, with no external teacher, no privileged answers, no visual evidence signals, and no inference-time cost.

VCSD extends the branch D-OPSD (05-07) opened, and inverts its polarity. D-OPSD made one network both teacher and student under conditioning asymmetry, giving the teacher more conditioning (text plus target image) than the student (text alone). VCSD gives the teacher two conditionings and uses the gap between them, so the asymmetry is teacher-versus-itself rather than teacher-versus-student. That is the cleaner construction, because no privileged channel has to exist at training time and vanish at inference. Read against Requential Coding, the pairing is the sharpest statement this page has: one paper proves the transferred information is the disagreement, the other measures a disagreement per token and trains on it. The open move nobody has made yet is to close the loop, computing a disagreement-based code length and using it as the selection criterion, which would turn TIP's four-month-old entropy heuristic into a derived rule rather than an empirical one.

The unexplained result worth stress-testing: 8B gains more than 4B. Self-supervision tricks normally decay with scale, and this one does not, on one dataset. The other caveat is structural rather than empirical. Because sharpening stays inside the real-image distribution's support, VCSD can only reweight candidates the teacher already considered, so it cannot recover from a teacher that omitted the right answer entirely, which is precisely the failure a genuinely stronger external teacher fixes. Self-distillation buys independence from a teacher at the cost of inheriting the model's own blind spots.

Prior State (as of 2026-07-24)

The reliability-gating line moves from per-token to per-history, and goes fully offline. ReOPD (Multi-Turn On-Policy Distillation with Prefix Replay, 2607.04763, Microsoft Research / Furu Wei) carries the page's core pattern (never apply the teacher signal uniformly, gate it by reliability) into multi-turn agentic distillation, and names the multi-turn version of the drift the line keeps diagnosing: a prefix trap. Making the interaction history more student-on-policy improves relevance but pushes the teacher onto histories where its target is unreliable, a two-sided distribution shift between student occupancy and teacher reliability. This is the same object as TA-OPD's "teachability" and Quality-Aware OPSD's "can this prefix still reach the answer?" gate, now stated over whole trajectories rather than single tokens. ReOPD's fix is a step-decaying sampling schedule that emphasizes early, low-shift prefixes where the teacher is still trustworthy, and it reuses pre-collected teacher trajectories as replayed prefixes so the student never touches the environment: zero tool calls during training, at least 4x faster per rollout, matching or beating online OPD. It is the offline, agentic descendant of H²SD and pairs with same-week Dataset Distillation by Influence Matching (2607.16859), the data-side cousin: both compress by keeping only the high-value slice (reliable prefixes / influence-matched synthetic samples) rather than processing everything. The unified per-token reliability estimator the page has flagged since 06-18 now also owes a per-history generalization.

Current State (as of 2026-07-22)

The reliability-gating line gets an outcome-conditioned form: use the teacher for direction on failures and magnitude on successes. H²SD (Hybrid Hindsight Self-Distillation, 2607.18955) is the newest instance of the page's "gate the teacher signal, never copy it token-for-token" pattern, but it gates on trajectory outcome rather than per-token reachability. On correct rollouts the self-teacher (the same model given the confirmed-correct answer plus a rephrasing instruction) only modulates update magnitude and never overrides the direction the reward set; on failed rollouts the teacher is conditioned on a reference hint containing the verified answer, and the student minimizes reverse KL toward it, giving an explicit correction direction. This fills the gap the page's predecessor RLSD left (RLSD could modulate step size but never point a failed rollout the right way), and it is the distillation-side sibling of the same-day RLVR-optimization surge on rl-for-llms (ISO frame-only geometry, SAT staleness-tail clipping). It also refines the still-unwritten unified reliability estimator the 06-18 section flagged: H²SD adds outcome (correct vs failed) as the coarsest reliability gate, and its verified-answer-conditioned failure signal sits on the trustworthy end of yesterday's bandwidth-versus-trustworthiness frontier (07-21). Limit: the failure branch needs the ground-truth answer to build the hint, so like Quality-Aware OPSD's box-membership gate it is train-time-only and verifiable-domain-only; reasoning benchmarks only, no coding/agentic results.

Current State (as of 2026-06-18)

The week's pattern has a name now: the OPSD teacher signal must be gated by reliability, not copied uniformly — and the fourth paper in two days carries it into coordinate prediction. Quality-Aware OPSD ("Trust the Right Teacher", arxiv 2606.18101) is the first time the spring selection-and-gating line lands on a VLM-based GUI grounding task (mapping an instruction to the exact screen coordinates to click). Naive OPSD breaks here for the diagnosis the line keeps naming: OPSD scores the teacher on the student's prefix, and once the student's coordinate prefix has drifted off the target, the teacher's next coordinate-token prediction is conditioned on a doomed path and becomes noise. The fix is two coupled mechanisms. A soft correctness-aware gate asks, per coordinate-token, whether the student's current prefix can still be completed into the ground-truth box; if not, that teacher signal is down-weighted. Teacher-probability scaling then calibrates the strength of the surviving signal by the teacher's own confidence. The key empirical claim is that neither component helps alone — gating without calibration and calibration without gating each fall short — and only the combination consistently beats the base model and strong baselines across six GUI grounding benchmarks. The two roles are distinct: gating is a yes/no reliability filter, scaling is a graded calibration.

This makes the four-paper week explicit as an N-of-a-kind pattern. ZPPO (06-17, pull the teacher out of the gradient and into the prompt on questions where every rollout fails), d-OPSD (06-17, condition the diffusion-LLM self-teacher on the student's own self-generated future instead of a left prefix), OPD-Evolver (06-17, distill only the memory operations that outcome-attribution credits as helpful), and now Quality-Aware OPSD (06-18, gate the teacher per coordinate-token by whether the prefix can still reach the answer) are four mechanisms making one claim: on-policy distillation works only when the teacher signal is gated or re-weighted by reliability, never applied token-for-token by default. This is the same instinct the page already tracks under different names — TA-OPD's "teachability" (reachable corrections), TrOPD's "trust region" (reliable-supervision region), SG-OPD's "sign-consistency gate" (verifier-endorsed direction), and TIP's <10%-of-tokens selection. Quality-Aware OPSD adds the cleanest reliability test yet: for a coordinate task the question "can this prefix still reach the answer?" reduces to ground-truth-box membership, decidable per token. The open wall is unchanged and now sharper: TA-OPD's teachability, TrOPD's trust region, SG-OPD's sign-consistency, and this paper's can-still-complete gate are four task-specific estimators of the same latent quantity — per-token teacher reliability — and the unified estimator that subsumes them (with the coordinate-box case as the cleanest instance) is still unwritten. Two limits to note: the gate needs the ground-truth box, so it is train-time only and cannot run at inference; and the result is GUI-grounding-only with no scale study, so whether the reliability-gating frame transfers from coordinates back to general reasoning OPSD is the test to watch.

Current State (as of 2026-06-17)

Three distillation papers the same day, and one of them refuses to put the teacher in the gradient at all. The standing tension on this page has been how to inject teacher signal into the student's gradient without the off-distribution drift the line keeps naming (Many Faces "distribution mismatch", TA-OPD "unreachable disagreement", TRD "prefix failure"). ZPPO (Zone of Proximal Policy Optimization, arxiv 2606.18216) answers by removing the teacher from the gradient entirely and putting it in the prompt: on hard questions where every rollout fails (zero advantage), it builds discrimination prompts (BCQ: one correct teacher + one wrong student answer, anonymized; NCQ: aggregate the student's own wrong rollouts) and recirculates them through a replay buffer until the student "graduates" at 50% accuracy. Largest gains at the smallest scale (0.8B with a 27B teacher), exactly where logit-imitation is most brittle. This is the mirror image of MOPD (Multi-teacher On-Policy Distillation), which the Interconnects 06-16 podcast named the 2026 frontier default (DeepSeek V4 §5.1, MiMo-V2-Flash, Nemotron 3 Ultra >10 teachers): MOPD maximizes teacher-in-gradient (token-by-token reverse-KL to the relevant specialist), ZPPO removes it. Two opposite bets on small-student brittleness in one week — the open question is whether they compose (MOPD for reachable questions, ZPPO for the unreachable hard tail).

d-OPSD (Learning from the Self-future, arxiv 2606.18195 — reuses the acronym of the 05-07 diffusion paper, distinct work) ports on-policy self-distillation to diffusion LLMs for the first time. AR-centric OPSD injects privilege as a left-to-right prefix with token-level KL, which conflicts with arbitrary-order denoising; d-OPSD conditions the self-teacher on the student's self-generated answer as a suffix ("self-future experience") and supervises at the step level (denoising iterations), beating RLVR/SFT at ~10% of RLVR's steps. It carries the conditioning-asymmetry device (PBSD, SDPG) into a new architecture, with the twist that the privilege is temporal-but-not-positional. OPD-Evolver (arxiv 2606.17628) applies on-policy self-distillation to agent memory: a slow/fast co-evolution loop distills four memory abilities (select/use/write/maintain over a 4-level hierarchy) into the deployable policy via outcome-calibrated attribution and privileged hindsight, with a 9B beating 397B-scale models. The "privileged hindsight" + "outcome-calibrated turn-level credit" is PBSD's machinery applied to memory operations. The page's open "joint formulation across all OPD facets" gap now also has to account for prompt-channel teaching (ZPPO), arbitrary-order supervision (d-OPSD), and memory-operation distillation (OPD-Evolver).

Current State (as of 2026-06-16)

Two scale extremes the same day: multi-teacher OPD at 550B, and a claim that verifiable reasoning compresses to 3B. Nemotron 3 Ultra (arxiv 2606.15007) puts Multi-teacher On-Policy Distillation (MOPD) into a frontier post-training recipe — the first multi-teacher OPD the wiki has logged at this scale, extending the page's heavy OPD line (Dense Supervision, Sparse Updates 06-15, SG-OPD 06-12) from single-teacher to a teacher ensemble inside a 550B MoE. At the opposite end, VibeThinker-3B (arxiv 2606.16140) reaches 94.3 on AIME26 and 80.2 Pass@1 on LiveCodeBench v6 in a 3B dense model (using curriculum SFT → multi-domain RL → offline self-distillation), matching far larger flagships on verifiable tasks. Its Parametric Compression-Coverage Hypothesis is the conceptual payload and the natural lift of this page's standing finding that the distillation signal is sparse and locatable (TIP <10% of tokens carry signal; Dense-Sparse's FFN-heavy subnetwork): VibeThinker pushes it from "the update is sparse" to "the capability is compressible," and separates compressible verifiable reasoning from coverage-bound open-domain knowledge. That separation is what makes the day's phase-routing results coherent (Kilo plan/implement, FastContext): if verifiable reasoning fits in a small core, distilling that core out and routing verifiable subtasks to it is the same lever seen from compression and from routing. Open wall, same as the rest of the line: the hypothesis is confirmed only on verifiable (math/code) slices; the "knowledge needs coverage" half is asserted, not measured side by side.

Current State (as of 2026-06-15)

2026-06-15 — OPD's weight-space anatomy gets measured: dense supervision, sparse updates. Dense Supervision, Sparse Updates (arxiv 2606.13657) measures where in the weights OPD writes, across several language and vision-language model pairs, and the answer slots directly into the Geometry of On-Policy Distillation (06-09, OPD sits in a relaxed off-principal regime with early subspace-locking) thread. Two findings. Sparsity: OPD updates are small and coordinate-sparse, distributed across layers, FFN-heavy, and training only the discovered subnetwork recovers nearly full OPD performance, a concrete efficiency lever that operationalizes the subspace-locking observation. Geometry: updates are numerically full-rank but spectrally concentrated, lie mostly off the source weights' principal singular subspace, and fall disproportionately on coordinates where the source weight is near zero, matching the off-principal regime 06-09 described and the RLVR "moves away from principal directions" result (Zhu et al. 2025). The optimizer twist: sparsity-inducing SGD underperforms AdamW for OPD (the reverse of sparse RLVR, where SGD is competitive), because dense teacher supervision preserves heterogeneous coordinate-wise gradient scales that AdamW's adaptive scaling still exploits. This makes optimizer choice a diagnostic for reward density: sparse-reward regimes favor SGD, dense-supervision regimes favor AdamW. The OPD facet list (selection / stability / correction / geometry / credit / verifier-gating) now has its geometry facet measured directly in parameter space rather than inferred from rank dynamics.

Current State (as of 2026-06-12)

2026-06-12 — SG-OPD makes the verifier a gate on the teacher signal, not a replacement for it. SG-OPD (Sign-Gated OPD, arxiv 2606.09304) names two assumptions standard OPD silently relies on and that break in practice: trajectory-level alignment between student and teacher, and uniform token-level reliability of the teacher. It fixes both with a binary correctness verifier at two granularities. "Phased teacher sampling" mixes verifier-endorsed teacher rollouts into the cold-start (trajectory-level), so the student is not distilling from trajectories it could never produce — the same off-distribution diagnosis as TRD (06-09) prefix-repair and FiRe-OPD (06-04) filtering, but resolved by injecting good trajectories rather than repairing or discarding bad ones. The "sign-consistency gate" then extrapolates the update on tokens where the teacher's direction agrees with the verifier-correct direction and interpolates (damps) it where they disagree (token-level). Average gains of +1.98 (per-sample) and +7.50 (per-question) on competition math over standard OPD. The deeper move: it slots a verifier into the dense OPD signal as a modulator, which is the OPD answer to the reward-hacking worry that this week's Kurate-rated "LLMs Gaming Verifiers" (RLVR reward hacking, cs.LG #13) raises — keep the verifier in the loop but never let it be the sole sparse signal. The selection/stability/correction/geometry/credit facets now gain a sixth: verifier-gated update direction. The unified OPD formulation across all six remains unwritten, and SG-OPD's reach beyond verifier-rich domains (math, code) is the same open wall the whole line keeps hitting.

Current State (as of 2026-06-09)

2026-06-09 — OPD gets two mechanism papers the same day, attacking different layers. All spring the wiki tracked OPD interventions (which tokens to keep, reweight, trust). Today two papers ask what OPD is. On the Geometry of On-Policy Distillation (arxiv 2606.07082) characterizes OPD in parameter space: it sits in a relaxed off-principal regime (fewer weights touched than SFT, less constrained than RLVR) and exhibits subspace locking — cumulative updates collapse into a narrow low-dimensional channel fixed early, and constraining training to that early subspace preserves OPD but breaks SFT. Crucially, sparsifying update tokens and shifting rollout off-policy preserve the rank dynamics, while mixing in RLVR changes them, so the geometry is a property of the OPD objective itself. This retroactively explains the whole selection line: TIP (<10% of tokens), TA-OPD (reachable corrections), and OPRD (hidden-state distillation) all worked because the useful update was always confined to a narrow channel. Trajectory-Refined Distillation (TRD, arxiv 2606.08432) attacks the trajectory level: it names prefix failure, where a wrong early step makes the teacher's per-token distribution bimodal and fragments the gradient in a way no token-level truncation or reweighting can fix. TRD revises the bad prefix under teacher guidance within on-policy support before distilling, and broadens exploration by exposing alternative valid derivations even on correct rollouts. Where FiRe-OPD (06-04) filtered out bad trajectories, TRD repairs them, salvaging the discarded signal. A third paper the same day, PBSD (Privileged Bayesian Self-Distillation, arxiv 2606.09348), pushes self-distillation fully into RL credit assignment: it uses the likelihood ratio between a standard student and a privileged answer-conditioned teacher (the same conditioning-asymmetry device as D-OPSD and SDPG) to convert a sparse final reward into Bayes-calibrated turn-level credit for long-horizon search agents. The selection axis (which tokens), the stability axis (TrOPD trust region), the correction axis (TRD trajectory repair), the geometry (locked subspace), and now turn-level credit (PBSD) are five named facets of one OPD/OPSD family — the joint formulation is still unwritten.

Current State (as of 2026-06-07)

2026-06-07 — distillation/compression as the answer to compute scarcity, on two new fronts. Two papers extend the efficiency program away from LLM token-distillation into adjacent regimes. Flash-WAM (arxiv 2606.05254) carries step distillation into joint video-action robot policies and shows the standard single-modality recipe breaks when one model carries two streams at different noise levels: video tolerates high noise, precision-critical actions need a gentle schedule, so a uniform consistency function degrades (naive consistency distillation drops real-world success to 24%). The fix is modality-aware: a variance-preserving consistency function for the high-noise video stream, a linear-gradient-scaling one for the low-noise action stream, grounded in a structural analysis of achievable gradient scaling. Result: single-step inference per modality, 8.1s → 348ms per chunk (23x), real-world success recovered to 60%. Where the spring OPD line (TIP → OPRD) debated what to match, Flash-WAM adds a how-to-parametrize-when-streams-differ axis. Separately, SEAOTTER (arxiv 2606.03940) is sensor-side learned compression for cloud robotics that transcodes once into a standard JPEG so no downstream neural decoder is paid: at 200:1 vs AVIF, 7x faster encode, 3.5x faster decode, +8% ImageNet, full JPEG-ecosystem compatibility. It is the upstream sibling of AdaCodec (06-06, compress frames before the model); SEAOTTER compresses sensor images before they leave the robot. Both are "more perception per watt/byte" levers that matter precisely under the HBM-to-2031 / Black-Friday compute-scarcity backdrop. Also note Google's TIPSv2 (CVPR 2026) reported distillation making a student beat a much larger teacher on patch-text alignment, a third 2026 datapoint that distillation is not merely lossy compression.

Current State (as of 2026-06-05)

2026-06-05 — OPRD changes the venue: distill representations, not output probabilities. The whole spring program above debated which tokens to supervise in output space (TIP → TA-OPD → TrOPD → FiRe-OPD), all of them fighting noisy reverse-KL gradients over a ~150k-token vocabulary. OPRD (On-Policy Representation Distillation, arxiv 2606.06021) argues the variance is an artifact of distilling at the wrong layer. It aligns student and teacher hidden states across selected layers on the same student rollouts, bypassing the LM head entirely. Theoretically this removes the Monte-Carlo sampling variance the selection line was managing; empirically it closes the student-teacher gap on AIME 2024/2025 and AIMO where output-space OPD plateaus below the teacher, and trains 1.44x faster using 54% less memory than top-k OPD (no full-vocab head to estimate). This reframes the entire token-selection debate as possibly solving a problem created by output-space distillation: if matching dense hidden states removes the variance that motivated selecting tokens, the selection axis becomes optional. Open tension: today's Rethinking Continual Experience Internalization finds off-policy context-distillation more stable than on-policy (on-policy is limited by local corrections on student-induced flawed states). OPRD is on-policy but supervises representations rather than token corrections; whether representation alignment inherits or escapes that instability is the question linking the two papers. The selection axis now reads: salient → reachable → reliable → soft-weighted-after-filter; OPRD opens an orthogonal venue axis: output space vs representation space.

Current State (as of 2026-06-04)

2026-06-04 — two OPD papers push past hard token selection and past two-loop stabilization, on the same day. FiRe-OPD (Filter, then Reweight, arxiv 2606.02684) operates on both granularities the selection line split apart: it filters whole trajectories to drop bad rollouts, then applies soft token reweighting inside the survivors. The argument against the TIP/TA-OPD hard-selection line is explicit: hard top-k throws usable signal away, so weight every retained token continuously instead. It reports +6.25 AIME 2024 (strong-to-weak) and +18.81 Miner (multi-teacher) over recent token-level methods, with the multi-teacher gain hinting that trajectory filtering matters most when teachers disagree. This is a partial answer to yesterday's TrOPD Research angle (unify selection across levels), though it does not yet add TrOPD's reliability trust region. SDPG (Self-Distilled Policy Gradient, arxiv 2606.04036) goes the other way and answers yesterday's Looking Ahead prediction directly: instead of two stabilizers, it folds OPD into RLVR as one loss. The model conditions on privileged context to supervise its own generations, written as an exact full-vocabulary reverse-KL student→teacher term, combined with group-relative verifier advantage (÷ std) and reference-KL. Using the exact KL rather than the cheap K1 estimator sidesteps the gradient outliers TrOPD fought, trading memory for stability. SDPG is the "RL is weighted SFT" instinct (DRIFT, 06-01) generalized to self-distillation, and the privileged-conditioning teacher is the same trick as D-OPSD (05-07). The selection axis now reads: salient tokens (TIP) → reachable tokens (TA-OPD) → soft-weighted retained tokens after a trajectory filter (FiRe); the stability axis reads: trust region on a cheap estimator (TrOPD) vs exact full-vocab KL inside the policy gradient (SDPG).

Current State (as of 2026-06-03)

2026-06-03 — OPD gets its stability layer (TrOPD), and the same trust-region primitive shows up in a frontier RL run the same week. Trust Region On-Policy Distillation (TrOPD, Samsung, arxiv 2606.01249) targets the instability the whole OPD theory has been circling: when student and teacher distributions diverge, the student wanders into regions where the teacher's supervision is unreliable, the cheap K1 reverse-KL estimator produces large gradient outliers, and training can collapse. TrOPD applies OPD only inside a reliable-supervision trust region, handles outlier regions with clipping/masking/forward-KL, and uses off-policy guidance (continue from teacher prefixes, imitate via forward KL) to pull on-policy exploration back toward reliable regions. It beats OPD/EOPD/REOPOLD on math, code, and general tasks and ships a unified OPD benchmark. This is the how-to-bound-the-update complement to the which-tokens selection line: TIP (04-16, <10% of tokens) and TA-OPD (06-01, reachable teacher corrections) pick tokens; TrOPD bounds the update. "Reachable tokens" (TA-OPD) and "reliable-supervision region" (TrOPD) are the same instinct two days apart. The strongest signal: Microsoft's MAI-Thinking-1 (same day, via Ken Huang) stabilizes a long GRPO run with an asymmetric trust region plus a hard ratio clamp, steered by an integral controller on policy entropy — the same control-theory primitive against the same enemy (reverse-KL outliers under distribution mismatch), reached independently on the RL side.

Also 06-03 — upward distillation appears (Knowledge Seeding). The 06-03 Language Models Need Sleep (2606.03979) frames continual learning as offline consolidation, with a "Knowledge Seeding" step that distills a smaller-self into a larger network (capacity expansion, not compression) via OPD + RL-based imitation. Unusual direction for the wiki's distillation literature, which compresses downward. (Note: title collides with the unrelated 05-27 SSM-consolidation paper.)

Current State (as of 2026-06-02)

2026-06-02 — On-policy distillation reaches the speculative-decoding drafter (Draft-OPD). Draft-OPD (arxiv 2605.29343) applies the OPD covariate-shift lesson to a new place: the draft model in speculative decoding. SFT-built drafters (EAGLE3, DFlash) plateau because they train on fixed target trajectories but are evaluated on blocks they propose under their own policy — the exact offline-to-inference mismatch the wiki has logged for student models. Naive OPD is hard for drafters because they cannot roll out reliably alone and target-assisted rollout erases the on-policy signal; Draft-OPD resolves this by using target-assisted rollout for stable continuations while replaying drafting from the verification-exposed error positions, so training concentrates on the draft-induced errors that cap acceptance. Over 5x lossless acceleration for thinking models, +23% over EAGLE-3, +13% over DFlash. This is the same diagnosis as TA-OPD (06-01), DRIFT (06-01, RL≡weighted SFT), and DAgger-for-LLM-agents (05-14): offline training distribution ≠ on-policy test distribution. The fix transfers cleanly to the drafter. See also speculative-decoding.md.

Current State (as of 2026-06-01)

2026-06-01 — Teachability closes the loop TIP opened (TA-OPD). Not All Disagreement Is Learnable (TA-OPD, arxiv 2605.26844) refines the central question of selective on-policy distillation: which tokens carry the signal. TIP (2026-04-16) answered "under 10%, the high-entropy and overconfident-wrong ones." TA-OPD shows raw KL disagreement is a coarse proxy because it conflates learnable disagreement (teacher's corrective mass lands on the student's top-K candidates, so the student can reach it) with incompatible disagreement (teacher's mass sits off the student's current support, unreachable in one step). It formalizes "token teachability" (local compatibility) and applies the OPD loss only at high-teachability positions, no reward model or verifier. Result: often beats full-token OPD with only 5% of tokens, and beats entropy- and divergence-based selection, on Qwen2.5/Qwen3 pairs. This is a direct fix for the "biased TopK reverse-KL gradients" failure named by The Many Faces of On-Policy Distillation (05-13): the bias came from training on disagreement the student could not reach. The selection axis now reads: salient tokens (TIP entropy/divergence) → reachable tokens (TA-OPD teachability).

Current State (as of 2026-05-16)

2026-05-16 — Teacher-side control becomes the third axis (ATESD). Every prior OPSD method gave the teacher the full reference solution; ATESD treats teacher exposure as a learnable control variable. A Beta-policy controller, conditioned on training-state statistics, samples a reveal ratio in [0, 1] and holds it for a short window of student updates. A discounted learning-progress reward addresses the delayed credit assignment that single-step rewards cannot solve. On AIME 24, AIME 25, HMMT 25 across Qwen3-{1.7B, 4B, 8B}: +0.95, +2.05, +2.33 Average@12 points over OPSD respectively. Combined with Extrapolation Cliff (closed-form policy for when OPD beats OPRL, 2026-05-14) and SDAR (gated student-side OPSD, 2026-05-15), three orthogonal axes of teacher-signal control are now operational: branch selection (Cliff), student-side gating (SDAR), teacher-side exposure (ATESD). None of the three papers composes with the other two; the joint formulation has not been written. → ATESD summary

Prior State (as of 2026-05-13)

2026-05-13 cluster on on-policy distillation. Two HF papers land on the same day and together they reframe OPD as a layered problem with both an allocation rule and a failure taxonomy. The Sparse-to-Dense Reward Principle (arXiv 2605.12483) argues that GRPO and OPD are not separate recipes but different reward-density regimes, with a clean allocation rule: scarce labeled data should train the strongest teacher first via sparse RL, get bridged to the student by forward-KL + OPD, and only then is student-side GRPO effective. A 1.7B Qwen3 student bridged from an RL-improved 8B teacher beats direct GRPO on the same student. The Many Faces of On-Policy Distillation (arXiv 2605.11182) names three failure mechanisms: distribution mismatch when teacher labels are computed on student-generated prefixes, biased TopK reverse-KL gradients, and OPSD-specific aggregation collapse where the student learns a privileged-information-free average policy that helps no specific instance. OPSD works when the privileged information is a shared latent rule, fails when it is instance-specific. Together with TIP (token selection), the Cliff (operating-point bound, 2026-05-14), and the prior CoPD / D-OPSD / RLRT entries, OPD now has all four layers of a mature theory: allocation, mechanism, failure modes, and bound. → Sparse-to-Dense summary · Many Faces summary

Prior State (as of 2026-05-07)

On-policy distillation has become the dominant approach for reasoning model compression. The key open question is which tokens actually carry the learning signal, naive approaches use all tokens, but most are uninformative. The distillation toolbox now ranges from token-importance methods (TIP) to neutral-channel cross-architecture transfer (BLD, TESSY, Switch-KD, Tide), parallel co-evolution (CoPD), and self-distillation under conditioning asymmetry (D-OPSD, 05-07). Two papers on 2026-05-07 attack the heterogeneous-information-density problem in diffusion distillation specifically: Stream-R1 reweights the DMD objective at both rollout and pixel level using a single shared reward model, and D-OPSD eliminates the external teacher entirely by making the model its own teacher under different conditioning. Policy dimension (2026-05-05): Nathan Lambert's "Distillation Panic" warns that the term is being conflated with API-jailbreaking attacks, and that pending U.S. legislation aimed at "distillation attacks" risks chilling the legitimate technique used by every lab. xAI's trial admission ("Generally AI companies distill other AI companies") is the most direct insider acknowledgment that the practice is industry-wide. The technical and political surfaces of distillation are now both load-bearing.

Key Papers

TIP: Token Importance in On-Policy Distillation (2026-04-16) — Identifies two high-signal token regions: high-entropy (uncertain student) and low-entropy + high-divergence (overconfident but wrong). Entropy-based 50% token selection matches full training with 47% less peak memory. <10% of tokens (targeting overconfident region) nearly matches full baseline. → summary

Key Concepts

• On-policy distillation: student generates its own rollouts, then learns from teacher's token-level distribution over those rollouts
• High entropy tokens: student is uncertain — natural learning signal
• Overconfident tokens (low entropy, high divergence): student is wrong but confident — dense corrective signal often missed by entropy-only selection
• Memory efficiency: token selection not only improves learning quality but also reduces peak GPU memory — critical for distilling large reasoning models

Cross-Tokenizer Distillation / BLD (2026-04-17) — When teacher and student use different tokenizers, standard token-level distillation breaks. Byte-Level Distillation (BLD) solves this by converting both to a shared byte-level representation and distilling there. A lightweight byte-level decoder head is added to the student. Competitive with complex CTD methods despite its simplicity. → summary

TESSY (2026-04-18) — Stylistic divergence between teacher and student causes SFT performance drops even when the teacher's reasoning is correct. TESSY interleaves teacher and student: teacher generates reasoning-heavy tokens, student generates style tokens. The hybrid sequence is stylistically the student's but intellectually the teacher's. Turns a 10% performance drop (Qwen3-8B on GPT-OSS-120B data) into a 6.7% gain on code generation. → summary

Switch-KD (2026-04-18) — Multimodal distillation for VLMs. Routes the student's visual outputs through the teacher's language pathway, forcing transfer through a shared text-probability space instead of separate modality silos. +3.6 points averaged across 10 benchmarks for a 0.5B student distilled from a 3B teacher. → summary

Key Concepts

• On-policy distillation: student generates its own rollouts, then learns from teacher's token-level distribution over those rollouts
• High entropy tokens: student is uncertain — natural learning signal
• Overconfident tokens (low entropy, high divergence): student is wrong but confident — dense corrective signal often missed by entropy-only selection
• Memory efficiency: token selection not only improves learning quality but also reduces peak GPU memory — critical for distilling large reasoning models
• Cross-tokenizer distillation (CTD): transferring knowledge when teacher and student use different tokenizers — the vocabulary mismatch problem
• Byte-level interface: using raw bytes as a universal common representation between any two tokenizers
• Stylistic divergence: teacher and student have different learned generation styles; a teacher's reasoning traces can be too foreign for the student's optimizer
• Cooperative synthesis (TESSY): interleaving teacher/student token generation to create hybrid training data that matches student style while containing teacher reasoning

ShadowPEFT (2026-04-22) — Centralized PEFT via depth-shared shadow module. Shifts adaptation from distributed weight-space perturbations (LoRA) to a single shadow module that evolves a parallel state through all transformer layers. Decoupled from backbone, independently pretrainable, optionally deployable in detached mode for edge computing. Matches or outperforms LoRA and DoRA at comparable parameter budgets. → summary

Tide: Cross-Architecture Distillation for Diffusion LLMs (2026-04-30) — First framework to handle teacher/student mismatch in all three of architecture, attention mechanism, and tokenizer for diffusion LLMs. Three components: Tidal (noise-aware schedule across timesteps + training progress), CompDemo (complementary-mask context enrichment), Reverse Calm (inverted chunk-likelihood matching with bounded gradients). Distills 16B MoE / 8B dense teachers into a 0.6B BD3LM student; +1.53 avg across 8 benchmarks; HumanEval 32.3 → 48.78; 22× memory reduction, 5× inference speedup vs teacher. → summary

CoPD: Co-Evolving Policy Distillation (2026-05-01) — Multi-capability post-training via parallel expert RLVR + bidirectional OPD during training (not after). Mixed RLVR has divergence cost; train-experts-then-OPD has behavioral-pattern gap; CoPD avoids both by having experts mutually distill while RLVR-training in parallel. Integrates text/image/video reasoning into one model that surpasses domain-specific experts. Bidirectional OPD as the neutral exchange channel, applied to parallel training rather than student/teacher pipeline. Fifth paper in the cross-distillation-channel pattern (after BLD, TESSY, Switch-KD, Tide). → summary

The Distillation Panic — Nathan Lambert (2026-05-04) — policy/discourse piece arguing that the "distillation attacks" framing being pushed in U.S. legislation conflates legitimate post-training distillation (used by every lab, including Nemotron and Olmo) with API jailbreaking (which should be called jailbreaking). The risk: a domestic ban on Chinese open-weight models built via API distillation, with collateral damage to Western academics and small labs. Quotes Musk's xAI trial admission as evidence the practice is industry-standard. Adds the policy dimension to the wiki's distillation tracking. → summary

Stream-R1: Reliability-Perplexity Aware Reward Distillation (2026-05-07) — DMD (Distribution Matching Distillation) for streaming video diffusion, reweighted by a single shared video reward model on two axes: inter-reliability (per-rollout, via exp(reward_score)) and intra-perplexity (per-pixel, via gradient saliency from the same reward). Adaptive balancing prevents any single quality axis (visual, motion, alignment) from dominating. The video-streaming analogue of TIP: heterogeneous information density across rollouts, frames, and pixels means uniform supervision wastes signal. → summary

D-OPSD: On-Policy Self-Distillation for Step-Distilled Diffusion (2026-05-07) — addresses the practical problem that standard SFT destroys the few-step capability of step-distilled diffusion models (Z-Image-Turbo, FLUX.2-klein). Novel paradigm: the same model serves as teacher and student under different conditioning. Teacher sees text plus target image (multimodal), student sees only text. Loss minimises divergence between the two over the student's own rollouts. The seventh paper in the neutral-exchange-channel pattern, but with a new neutral channel: conditioning asymmetry on the same network. → summary

Key Concepts

• On-policy distillation: student generates its own rollouts, then learns from teacher's token-level distribution over those rollouts
• Centralized PEFT (ShadowPEFT): single depth-shared module performs layer-space refinement, unlike LoRA's per-layer weight perturbations
• Layer-space vs weight-space adaptation: ShadowPEFT refinement evolves a parallel state through the network depth; LoRA adds local rank-decomposed perturbations to individual matrices
• Cross-architecture diffusion distillation (Tide): distill from a diffusion teacher to a diffusion student of different size/attention/tokenizer; requires noise-aware scheduling, complementary-mask context enrichment, and bounded-gradient cross-tokenizer losses
• Neutral exchange representation pattern: across BLD (bytes), TESSY (cooperative interleaving), Switch-KD (shared text-probability space), Tide (inverted chunk-likelihood with bounded gradients), and CoPD (bidirectional OPD between parallel RLVR experts), the field has converged on engineering a neutral channel between mismatched teacher/student rather than forcing token alignment. Five papers, five mechanisms, one principle.
• Co-evolution distillation (CoPD): parallel RLVR-trained experts serve as mutual teachers via bidirectional OPD during training, eliminating both the inter-capability divergence of mixed RLVR and the behavioral-pattern gap of train-then-distill
• Selective-supervision OPD (the dominant 2026 thread): uniform per-token supervision is wasteful and often harmful, so select for the load-bearing signal. Sequence: TIP (04-16, ~10% of teacher tokens carry signal) → TA-OPD (06-01, only reachable teacher corrections) → TrOPD (06-03, only tokens where the teacher is reliable) → FiRe-OPD (06-04), which argues for two-level granularity: hard-filter trajectories, then soft-reweight tokens (soft > hard avoids information loss) → SG-OPD (06-12), verifier-gated update direction → Quality-Aware OPSD (06-18), which gates the teacher per coordinate-token by whether the student's prefix can still complete into the ground-truth box, then calibrates the survivors by teacher confidence (first time this line lands on a GUI-grounding VLM task). SDPG (06-04) attacks the same density problem from the RLVR side, manufacturing a dense full-vocabulary reverse-KL teacher from the model's own privileged-conditioned self. The one-week convergence (ZPPO, d-OPSD, OPD-Evolver, Quality-Aware OPSD, 06-17/06-18) makes the principle explicit: the teacher signal must be gated/re-weighted by reliability, never copied token-for-token by default.
• Budgeted expert reads (MergePipe, 06-04): weight-space merging is I/O-bound at LLM scale; MergePipe casts it as a budgeted expert-access-set problem with a provable omitted-delta error bound and 11x speedup. The merge-time companion to BEAM's inference-time expert-activation budgeting.

Related Pages

• ../llms-foundation-models/rl-for-llms.md
• ../llms-foundation-models/2026-04-16-prerl-rl-in-pretrain-space.md
• kv-cache.md

The anti-distillation defense is architecturally broken (2026-08-11)

Stealing Reasoning Traces from Proprietary LLM APIs (08-11) changes the standing state of knowledge on this page in a way no method paper has. Every entry above concerns how to distill well. This one concerns whether distillation can be prevented, and the answer is no.

The mechanism matters because it explains why the defense fails. Frontier providers hide chain-of-thought by encrypting it and returning the ciphertext to the client, which replays it on each subsequent request. The encryption binds nothing: the blocks are fully interchangeable across sessions, users, and models within one provider's ecosystem. So an adversary injects a strong model's encrypted trace into a weaker, less-safeguarded sibling model from the same provider, and that model decodes and emits the trace verbatim in plaintext. The protected model is never jailbroken, so its safety training is irrelevant to the attack. Demonstrated across Anthropic, OpenAI, and Google, which makes it a design-pattern failure rather than one vendor's bug.

Three consequences for this page:

1. Hidden reasoning is not a moat. Any lab relying on concealed CoT to stop competitors training on its reasoning has been relying on nothing. The recoverable artifact is exactly the trace that on-policy distillation wants.
2. The policy argument shifts ground. The Distillation Panic (05-04) argued that legislative framing conflates legitimate post-training distillation with API jailbreaking. This attack is neither: it does not jailbreak the protected model at all, so a regime that polices jailbreaking does not reach it. The distinction the policy debate rests on is not the distinction that governs the technique.
3. Provenance moves from prevention to detection. Anthropic shipped in-text invisible watermarking on 08-10, carried in the text rather than metadata so it survives copy-paste. Hugging Face's Elie Bakouch asked publicly whether watermarking is really an instrument for proving a competitor trained on Claude output. If prevention has failed, watermark-based forensics is the remaining lever, and it is a fundamentally weaker one: it establishes that copying happened after the fact rather than stopping it.

Open question this raises for the selective-supervision thread. Every method on this page assumes a cooperative teacher whose distribution you are licensed to query. A recovered trace gives you the teacher's text but not its per-token distribution, so the extraction attack supplies SFT-grade data, not the dense token-level signal that on-policy distillation depends on. Whether trace-level theft is actually competitive with licensed dense supervision is unmeasured, and it is the number that decides how much the broken defense really costs. The paper does not report it.

2026-08-25: from "most teacher tokens are useless" to "some are actively harmful"

R2-OPD (08-25) sharpens the central claim of this page's selective-supervision thread. The prior state of knowledge here was that most teacher-generated tokens carry little learning signal and should be dropped or downweighted. R2-OPD makes a stronger and more uncomfortable claim: a subset of teacher supervision actively fights the thing you are training for.

The mechanism is a proxy failure, not a noise problem. On-policy distillation gives dense token-level reward derived from teacher agreement, and it implicitly treats that reward as a proxy for reasoning progress. The two come apart in a specific, systematic way: when the student finds a different valid reasoning path, teacher agreement falls while actual progress is fine, so the student gets punished for exactly the independent correct reasoning you want. Noise averages out over training; a systematic penalty on divergence-from-teacher does not. It biases the student toward mimicry, which is why the paper is titled Beyond Imitation.

The fix is deliberately not a better reward model. R2-OPD builds a second, cheap one and uses only the disagreement between them: rank reasoning spans within a trajectory by teacher-derived reward, rank them independently by estimated progress reward, and suppress the distillation reward wherever the two orderings disagree. Working on within-trajectory rankings rather than absolute values is the correct call, since the two reward sources share no scale. It bolts onto existing OPD pipelines.

This crosses the wiki's threshold for a named pattern: disagreement between two signals is itself the signal. Three independent instances now. The 08-08 weekly cluster (Requential Coding, OPD²) established that a single teacher signal is untrustworthy at token granularity. AgentOPSD (08-07) located pivotal turns in agent trajectories by disagreement rather than absolute reward. R2-OPD locates untrustworthy tokens the same way. The design principle to carry forward: do not trust a single supervision signal, and treat the conflict between two independent signals as the highest-information location in the trajectory.

The same idea surfaced in a different subfield on the same day, which is the more interesting observation. Task-CoEvolve (08-25) uses variance-weighted sampling to concentrate harness evaluation on validation tasks where candidate harnesses disagree, cutting evaluation count 80%. Distillation and harness evaluation, no shared authors, one day, one principle. It is standard active-learning theory (information concentrates at the decision boundary) being independently rediscovered rather than imported, and Task-CoEvolve is the more refined instance because it weights by disagreement magnitude where R2-OPD's filter is binary.

The load-bearing weakness, stated plainly. The independently estimated progress reward is itself an unvalidated learned model, and the sensitivity ablation is missing. This matters in a specific way: if the progress estimator errs in the same places as the teacher, the filter quietly does nothing and the reported gains come from somewhere else; if it errs in different places, the filter suppresses good supervision. Without that ablation the result is credible but its mechanism is unconfirmed. Note that VoI-MoLE (08-25) has the identical structural weakness in its reducibility estimator, so the pattern comes with a shared open risk: every method in this family depends on a second estimator nobody has validated.

Open experiment, now buildable. The 08-08 weekly asked for the composition of turn-level and token-level disagreement filtering. AgentOPSD supplies pivotal turns, R2-OPD supplies untrustworthy tokens, and filtering the pivotal tokens of the pivotal turns is the two-level version. Also unexplored: weighting by disagreement magnitude instead of thresholding it.

2026-08-28: the supervisor becomes optional, and the label goes with it

Two papers on one day each delete a different expensive dependency from the supervision pipeline, and together they push this page's central 2026 finding to its limit.

Self-OPD (arXiv 2608.26872, Tsinghua + Zhejiang + Alibaba) removes the teacher entirely for flow matching. The two problems it names are the ones this page has been circling: cost, because training a specialized teacher for every new objective is a per-objective expense rather than a one-time one, and compounding error, because a student regressed onto teacher velocity predictions drifts off the teacher's support step by step, so the supervision degrades exactly along the trajectory where it is needed. The construction: at each timestep, branch the deterministic next-state prediction into K stochastic SDE candidates, roll each to a finished sample with the ODE sampler, and compare their rewards against a deterministic self-reference baseline to get normalized advantages.

Three design choices carry it. The baseline is the deterministic path rather than the population mean, so the advantage answers "did adding noise here help relative to the confident path," which is a sharper question. The objective is all-branch pull-push: high-advantage branches attract the velocity field and low-advantage branches actively repel it, under direction-aware attenuation and SDE-variance normalization. Standard on-policy distillation only pulls; explicitly pushing away from bad branches is what lets a teacher-free method extract signal from its own failures instead of discarding them. And for multi-objective alignment it fuses at the reward level, not the gradient level, which the alphaxiv overview explains well: teacher-dependent multi-objective OPD merges several teachers' velocity fields, and conflicting update directions produce a compromise satisfying no objective, giving models that excel on one teacher's prompt family and fail elsewhere. Normalizing scalars before any gradient forms resolves the conflict in a space where it is well-defined.

This is the fourth instance of the teacher-ceiling pattern in four days and the strongest resolution of it. OPRD (06-05) found output-space distillation plateaus below the teacher on AIME and AIMO. OPDVR (08-26) diagnosed that a purely distributional objective bounds the student at its teacher and broke the bound with a verifier. QAH (08-26) found the ceiling was a degraded recovered checkpoint and swapped the teacher pointer to the original pre-compression model. Self-OPD removes the ceiling by removing the referent: a student supervised by its own advantage-ranked exploration has no teacher to be bounded by. The 08-26 digest stated the shared diagnosis as the binding constraint in distillation is the supervisor, not the student. Four papers, four mechanisms, one claim, well past this page's three-instance threshold.

It is also the second teacher-free dense-supervision result for generative models in three days. DiffusionOPSD (08-26) was the first entry on this page where the teacher is the student: a frozen behavior policy supplies anchors, reward gradients build bounded positive and negative targets, an EMA update refreshes the behavior policy, cutting training GPU-hours 40% on SD 3.5-M and 63% on the step-distilled Z-Image-Turbo. Self-OPD reaches the same place by branching at the SDE level instead of anchoring on a frozen copy. Two large groups, two constructions, three days. The gap between them is telling: DiffusionOPSD led with a GPU-hour reduction, and Self-OPD publishes no compute comparison at all, which matters because K full ODE rollouts per timestep is not obviously cheaper than the teacher it deleted. That is the load-bearing omission.

TTPO (arXiv 2608.27448, Zhejiang + Alibaba) removes the ground-truth label, and does it without adding an unvalidated estimator. The problem is specific: every method on this page needs labels for verification or teacher conditioning, so none can run at test time. Substituting a majority-vote pseudo-label fails worse for dense methods than for coarse ones, because a corrupted pseudo-label conditioning a teacher misleads at every token, where a corrupted sequence-level reward misleads once per trajectory. TTPO's observation makes the failure survivable: the error is asymmetric, because rollouts that disagree with the pseudo-label are usually wrong regardless of whether the vote itself was right. So route by branch. Distil the agreeing rollouts with on-policy self-distillation, penalize the disagreeing ones with grouped RL, and refine both with token-level selection: distillation down-weights already-converged positions, RL penalizes only confident errors. Without labels, it matches label-supervised OPSD on five competition benchmarks, raises Qwen3-1.7B from 38.0% to 45.2%, and yields +25.2% to +36.4% without thinking.

TTPO is the fourth instance of the disagreement-is-the-signal pattern, and the first to use disagreement as a dispatcher rather than a mask. The prior three (the 08-08 cluster on untrustworthy token-granularity teacher signal; AgentOPSD 08-07 locating pivotal turns by disagreement; R2-OPD 08-25 locating untrustworthy tokens by disagreement between teacher-derived and progress-derived rankings) all used disagreement to decide what to suppress. TTPO uses it to decide which algorithm to apply, which is a strictly more expressive use of the same signal.

And it is the best available answer so far to the shared risk this page flagged on 08-25. That note read: R2-OPD's independently estimated progress reward is itself an unvalidated learned model, a weakness shared with VoI-MoLE (08-05)'s reducibility estimator, so every method in this family depends on a second estimator nobody has validated. OPDVR escaped by using a verifier, which is not an estimator but needs ground truth. TTPO's second signal is a majority vote: no parameters, no training, a characterized failure mode (it fails when the model is confidently and coherently wrong), and it tightens as the policy improves. For the label-free regime that is a genuinely better construction than a learned estimator. The unquantified part is the asymmetry rate itself: how often a disagreeing rollout is actually correct bounds the damage the negative branch does, and on problems where the model is below chance the consensus is systematically wrong and the method could invert. Five competition math benchmarks is an unusually consensus-friendly setting.

Cross-page note. TaoLive Harness-Aware Training (08-28) uses General On-Policy Distillation as stage-two damage repair, to restore the generalization that harness-augmented SFT destroyed. That is structurally the same move QAH made two days earlier when it refused to distil from a degraded recovered checkpoint. Distillation as the thing that undoes the cost of a specialization or compression step is now a recurring role for it on this page, distinct from distillation as capability transfer.

Updated state of knowledge. The distillation program now has five axes: which tokens (TIP through Quality-Aware OPSD), which trajectories (FiRe-OPD), which teacher (QAH), whether the objective is bounded by the teacher at all (OPDVR), and now whether a teacher or a label is required in the first place (Self-OPD, TTPO). The direction is consistent and the accounting is not: the two 08-28 papers delete a dependency and neither publishes the compute that replaced it, which is the same omission the harness pages have carried since May.

2026-08-30: the teacher's right to abstain

FedCC (arXiv 2608.23031, Kurate cs.LG #1 this week) is this page's thread arriving in a distributed setting, with one twist worth naming. In distillation-based federated learning, clients never ship weights or data. They run their local model over a shared unlabeled public dataset and ship only predictions, which the server aggregates into pseudo-labels. Label distribution skew breaks this: a client that has only seen two classes does not produce a flat distribution on an unseen class, it produces a confident wrong one, and the server has no ground truth to calibrate it away with.

FedCC's fix is to let a client tag ambiguous samples unknown, plus calibrated pseudo-labels balancing majority-class confidence against minority-class uncertainty. In the extreme case where each of ten clients holds exactly one class, it reaches 67.3% accuracy where baselines collapse to near-random.

The twist relative to everything else on this page. The established finding here is that most teacher signal is worthless: TIP found most teacher-generated tokens carry no learning signal so roughly 10% suffices, and token teachability (06-01) made per-token teachability the selection criterion. FedCC's teacher is not merely uninformative on some inputs, it is actively harmful, and the fix is to let it decline rather than to have the student filter afterwards. Filtering at the source rather than at the sink, which in this setting is not a preference but the only option, because the sink has no labels to filter with.

What is not reported is the abstention rate, and it decides whether this is usable: if clients abstain on most public samples under severe skew, the effective communication is far sparser than the bandwidth argument implies, and there is a regime where the server has too few votes per sample to calibrate anything. Nothing here is at language-model scale either, and whether an unknown class means anything for a generative model, where the output space is not a fixed set of labels, is not a small extension.

Cross-page connection. FedCC lands the same day as two selection papers on the new model pruning and sparsity page: RoI selecting N weights of M, and MCL selecting training samples by marginal concept-coverage gain. All three replace a forced dense decision with a sparse one plus an explicit way to decline, and FedCC states most clearly why that matters: the alternative to a sparse decision is not a neutral one, it is a confidently wrong one.

2026-09-11: the teacher you build in order to be unlike it, and the day distillation acquired a regulator

Two arrivals, one technical and one institutional, and they belong on the same page for a reason this page should state plainly: the technique's cost curve and its legal exposure moved on the same day.

Negative Self-Distillation (09-11) (arXiv 2609.11699, University of Virginia + Stanford) inverts the sign of everything above. On-policy self-distillation conditions a model on privileged information, usually the ground-truth solution, has it write a clean trace, and trains the unconditioned model to imitate that trace. Recent work found this degrades hard reasoning, and the diagnosis the alphaxiv overview names is reflection collapse: a trace written by someone who already knows the answer never hesitates, never backtracks, never expresses uncertainty, so imitating it suppresses exactly the exploratory and self-corrective behaviour hard problems need. NSD builds a negative teacher instead: prompt the same model under a deliberately degraded persona (the paper's example is a "careless reasoner") and push the student's distribution away from it. No ground truth, no external teacher, no labels.

The technical contribution is the gate, not the inversion. Pushing away from a bad trace is an unlearning objective, and a flawed reasoning trace is still written in English. Its tokens mix the behavioural flaw with ordinary linguistic scaffolding, so indiscriminate divergence degrades the model's foundational language ability in exchange for a marginal reasoning gain. A dynamic gating mechanism identifies reasoning-critical tokens and confines the update to them. That is the same structural move as everything in this page's selective-supervision thread, applied for the first time to repulsion rather than attraction: TIP found most teacher-generated tokens carry no learning signal so roughly 10% suffices, R2-OPD (08-25) found some teacher supervision is actively harmful because agreement punishes a student who finds a different valid path, and both decided which tokens to trust. NSD decides which tokens are safe to repel. The page's running principle now covers both signs: never apply a single signal uniformly across a sequence.

It is the fifth escape from the teacher ceiling recorded here, and the cheapest by a wide margin. The 08-28 entry listed four: OPDVR (08-26) broke the bound with a verifier, QAH (08-26) swapped a degraded teacher pointer back to the original pre-compression model, and Self-OPD (08-28) and TTPO (08-28) deleted the teacher and the label respectively. NSD escapes differently: a negative teacher imposes no ceiling at all, because "be unlike this" does not bound you from above. And it needs neither a verifier, a second trained model, nor K rollouts per step, only one extra forward pass under a different system prompt. Self-OPD's all-branch pull-push already established that explicitly repelling low-advantage branches is what lets a teacher-free method extract signal from its own failures rather than discarding them; NSD is the pure-push limit of that idea in the language-model setting. Two independent groups, three weeks apart, both concluding a model's own failures are usable supervision.

The standing risk this page named on 08-25 gains its newest instance. That note read: every method in this family depends on a second estimator nobody has validated, written first for R2-OPD's progress reward and VoI-MoLE's reducibility estimator. NSD's gate is the fourth. Worse, NSD adds a second unvalidated object: the persona is a prompt, and there is no guarantee the flaws it produces are the flaws the model actually makes. If the negative teacher's errors are disjoint from the student's real errors, the method is pushing away from a strawman and the reported gains come from somewhere else. The fix is available and unwritten: TTPO showed that rollouts disagreeing with a majority-vote pseudo-label are usually wrong regardless of whether the vote was right, which is a label-free way to find genuine flawed traces instead of synthesizing fake ones.

X-AuT (09-11) is this page's damage-repair role in its cleanest form. The role was named on 08-28: distillation as the thing that undoes the cost of a specialization or compression step, distinct from distillation as capability transfer. X-AuT prunes a speech LLM's audio encoder progressively (18 to 16 to 14 layers) and repairs the interface with representation alignment plus cross-scale distillation from a 1.7B teacher, with the language-model backbone frozen. The teacher-scale effect is unusually large: 5.55% mean error from the 1.7B teacher against 8.45% for self-distillation, which is nearly three points bought by teacher capacity alone, on a page where most teacher-quality effects are fractions of a point. It also reports a failure mode this page has not recorded: pruning encoder blocks produces deletion errors and premature end-of-sequence, a behavioural change rather than a metric decline, which an averaged error rate cannot distinguish from fluent-but-wrong output.

And the institutional half. Anthropic's September threat report (09-11) alleges illicit extraction from seven Chinese labs since February, including 151 million+ Claude exchanges by Alibaba between May and July through 3,500+ fraudulent accounts, peaking near 3 million per day, targeting chain-of-thought, agentic, coding and kernel capabilities. It confirms the reasoning-trace extraction technique this page recorded as neither jailbreaking nor licensed use was used in practice. This closes the loop on the policy argument logged above: that entry said a regime policing jailbreaking does not reach the technique, and that provenance had moved from prevention to detection. A three-month, nine-figure campaign disclosed only after the fact is the confirmation. Nathan Lambert's reading list (09-11), published the same day, is the counterweight worth holding alongside it: independent evaluation puts the open-closed gap at 4 to 6 months, distillation contributes without explaining that gap, and he publicly revises his own April 2025 claim that DeepSeek did not distil o1 in light of the extraction methods.

The open question this page has carried since the extraction paper is now the decisive one and is still unmeasured. A recovered trace gives you the teacher's text, not its per-token distribution, so extraction supplies SFT-grade data rather than the dense token-level signal every method above depends on. Anthropic's claim that extracted conversations raise capability beyond the subjects they covered is the closest thing to evidence on either side, and the report publishes no measurement behind it. Two practical consequences for this page if the enforcement response holds: reasoning traces become a withheld product surface, which degrades every trace-consuming method here, and licensed dense supervision becomes more expensive relative to the teacher-free constructions (Self-OPD, TTPO, NSD) that this page spent August accumulating. Those three papers were framed as efficiency results. They are now also a hedge.

2026-09-16: the teacher's reasoning distribution is a design parameter you are already setting

Training Specialist Models without Reasoning Trajectories (09-16) (arXiv 2609.13770) asks what governs the trajectories a specialist generates when it was trained only on question-answer pairs with no reasoning supervision at all. The answer is that specialist optimization implicitly selects from a latent trajectory space, and that selection is inherited wholesale by anything distilled from it.

The methodological move is the good part: use student distillation as an instrument rather than as a goal. A student inherits no parameterization and no optimization constraints from the specialist, only the sampled trajectories. That makes it a clean read-out of the specialist's latent distribution in a way that probing the specialist directly cannot be. Across 27 specialist-student pairings the specialization-generalization profiles correlate exceptionally strongly, and explicitly controlling the specialist's distributional drift moves teacher and student together along a domain-precision versus general-capability trade-off, holding across chemistry, physics and multilingual settings and across divergent model families.

This reframes something this page treated as settled. The distillation thread here has been about what signal to transfer: OPRD (09-09) matched teacher hidden states rather than output tokens; negative self-distillation (09-11) removed the teacher entirely by learning to avoid self-constructed reasoning flaws, at up to 7.5% average gain across seven reasoning benchmarks and holding at 1.7B, 4B and 8B; TIP argued most teacher-generated tokens carry no signal and roughly 10% suffice. All of them take the teacher's trajectory distribution as given. This says it was a choice, made upstream, without anyone labelling it as one.

Pair it with Drift-Constrained Optimization (09-16), which landed on the same HuggingFace page the same morning. DCO declares a behavioral-drift budget before optimization, which fixes the distance from the reference model and leaves direction as the only free variable, turning fine-tuning into a direction-selection problem. This paper says the drift you allow a specialist determines what its students inherit. Two papers, one morning, both arguing that behavioral drift is a designed parameter rather than a side effect of optimization. Neither knows the other exists. That is the convergence signal worth recording: the field has moved from "how much should the model change" to "in which direction, under what budget."

Cost consequence nobody has priced. Every distillation result on this page carries teacher inference as a line item, because generating reasoning traces is the expensive part. This setting has no gold reasoning at all, so the specialist is cheaper to build. But the trade-off means a tighter domain specialist yields a student with less general capability, and that loss currently has no price tag, which makes the cheaper-specialist saving unverifiable as a net win.

Gaps. The drift-control knob is not named in the abstract (learning-rate schedule, KL penalty, data mixture all imply different practical costs). Strong correlation across 27 pairings is not causation. And nothing tests whether a student can be steered away from its teacher's profile, which is the question a practitioner asks second.

2026-09-18: three papers demote the teacher on three different axes, in one day

Three on-policy distillation papers landed together (09-18), and each attacks a different assumption the technique rests on. On-policy distillation (OPD) has the student generate its own rollouts while the teacher scores them token by token, so supervision is dense and on-trajectory. RetireOPD (arXiv 2609.20784) attacks the schedule, What Does Privileged Information Add to On-Policy Self-Distillation? (arXiv 2609.20612) attacks the premise, and When EOS Tokens Disagree (arXiv 2609.20511) attacks the plumbing.

This crosses the page's threshold for declaring a pattern, and the pattern is five weeks old. The 08-25 entry recorded the shift from "most teacher tokens are useless" to "some are actively harmful." The 08-28 entry made the supervisor optional. The 08-30 entry gave the teacher the right to abstain. The 09-11 entry recorded the teacher you build in order to be unlike it. Today the student fires the teacher, the teacher's special knowledge turns out not to have been what was helping, and the teacher is shown to be corrupting a behaviour the student already had right. The teacher's authority has been narrowed in every entry for five weeks, and today it narrowed on three axes at once.

RetireOPD: the value of supervision decays, so let the student decide when it is done. It trains a decoupled skill-conditioned teacher on environment rewards, then trains a skill-free student jointly with RL and OPD under Adaptive Retirement: the student drops the teacher once their discrepancy stops shrinking and it reaches a target fraction of the teacher's success rate, continuing with RL alone. Across Qwen2.5 from 1.5B to 7B: ALFWorld success up 14.1 to 18.8 percent over the RL baseline, WebShop accuracy up 11.8 to 19.0 percent, and it surpasses its own teacher in every setting. Two findings motivate it and both belong on this page: privileged information alone does not always make a teacher reliable, and the benefit of teacher supervision is stage-dependent. The 08-25 result was that teacher value is non-uniform along the token axis. This is the same claim along the time axis.

Privileged information adds much less than the field assumes, and the honest version is deflationary. Using AMPLE-Math, 5,319 problems with six reasoning views sharing one answer, each view compared against matched reference-free distillation, the authors find reference-free distillation accounts for much of Qwen3-1.7B's improvement both in domain and out. Additional reference benefit is modest, strongest for a polished solution, worth about two percentage points in SmolLM3-3B at step 50. Two results are sharper than the headline. The benefit is a property of the student, not the method: at a fixed checkpoint, swapping short direct-response rollouts for long thinking-enabled rollouts turns gains into losses in both families with problems, references and evaluation held constant. And changing token-level supervision can leave student behaviour largely unchanged, which is awkward for a subfield whose main activity is designing token-level losses. Their interpretation directly qualifies the 09-16 entry: the teacher's reasoning distribution is indeed a design parameter, but OPD may be improving access to capability the student already has through parameters shared between direct-response and thinking-enabled inference, in which case the reference's value is what it adds to that cross-mode transfer rather than how much of the solution it reveals.

The EOS paper is the one to act on first, because it is a correctness fix rather than a tuning choice. Across Qwen3, Llama and Gemma, a base student and a post-trained teacher can place stopping probability on different EOS tokens even when their declared stopping sets are identical. The loss then suppresses the student's preferred stopping action without reliably transferring the teacher's, and the student learns not to stop. Aligning the decoding stopping set is insufficient; treating functionally equivalent EOS tokens as a shared semantic stopping action substantially mitigates the inflation across all three families. A stage-wise analysis across K2-Horizon training stages shows termination preferences shift substantially during training and that a distinct late-run inflation persists even after alignment, so mismatch is important but not exhaustive. Implementation released.

The read-through to test-time-compute-allocation.md is a warning. Length inflation in distilled students has been treated as a modelling failure, inviting length penalties and reward shaping. A meaningful share of it is a tokenizer-level identity bug that no length penalty addresses. Anyone who added a length penalty to fix OPD inflation should check whether they paid a reward-shaping cost to paper over a token-identity problem.

Open problem this page should now carry. None of the three reports compute cost, which is striking for a family whose main claim over pure RL is sample efficiency. If the teacher can be retired early, the privileged reference is worth two points, and the plumbing fix is free, then the remaining question is what fraction of an OPD run's total cost is buying the last few points, and nobody has published that curve.

2026-09-21: the contamination gets a causal estimator, and the discard fraction reaches four

Cal-OPD (2609.21619) is the intervention that Privileged, but Biased (08-10) implied and did not build. That paper found privileged self-distillation reproduces its gains on easy tasks and teaches nothing on hard ones, with per-token loss falling steadily while accuracy stays flat, and traced the failure to PI Bias: a teacher that has seen one particular reference is pulled toward that trajectory rather than toward correctness, so the student's penalty lands on stopwords and, inside correct rollouts, on exactly the exploratory tokens that reasoning requires. It quantified the first link with a PI Bias Score and stopped. Cal-OPD closes it.

The mechanism treats privileged information as a probe rather than a crutch. Run the teacher three ways, plain, positively intervened and negatively intervened, and take the spread of its token-level likelihoods as an estimate of the region attributable to the teacher's own conditioning rather than to the student's deficiency. Then keep only the component of the teacher-student discrepancy that lies outside that region. Retaining 52 to 65 percent of the original signal, it consistently beats standard OPD and its variants across model scales on mathematical reasoning. It also states the uncomfortable corollary directly: privileged OPD is the worse case, not the better one, because privileged information induces larger teacher-side likelihood shifts and therefore more contamination. The setting the field adopted to strengthen supervision is the setting where the signal is dirtiest.

This page's discard-the-signal family now has four members and four different cutting criteria, and Cal-OPD's is the first causal one. TIP (04-16) cut by token importance and found roughly 10 percent of teacher tokens carry the signal. FIRE-OPD (06-04) filtered rollouts then reweighted. R2-OPD (08-25) cut by whether a step made reasoning progress. Cal-OPD cuts by attribution: is this gap the student's fault or the teacher's? Every prior criterion was a heuristic proxy for importance. This one asks the counterfactual question directly, and that is the methodological step the family needed.

It composes trivially with RetireOPD (09-18) and nobody has run the pair. RetireOPD's answer to degrading teacher value is temporal, fire the teacher once the discrepancy stops shrinking. Cal-OPD's is spatial, keep the teacher forever and listen to part of what it says. Together they would say: listen only to the uncontaminated part, and only until it stops helping.

Two gaps this page should carry. First, the evidence is math-only, which is the benchmark family 08-10 showed the gains are a difficulty artifact on. A fix validated where the failure was mildest is not yet a fix. Second, three teacher passes per batch is a real cost and the paper reports signal quality rather than accuracy per teacher-FLOP — which is the same omission the 09-18 entry flagged across all three papers that day. Four consecutive entries on this page have improved the training signal and none has priced it.

A new experiment this framing makes obvious and nobody has run. Everyone selects the strongest available teacher. Cal-OPD's estimator says what you actually want is the teacher whose self-deviation region is smallest on your task, which is a different model and a cheap thing to measure now that there is an estimator for it.

2026-09-22: the frame shifts from which tokens carry signal to where the signal can be measured

IER-OPD is the fourth paper this page has logged arguing most training signal is wasted, and the first to say the waste is a measurement problem rather than an information problem. The thread, with each paper's actual claim: TIP said most teacher-generated tokens carry no learning signal and roughly 10% suffices, working at the token-weighting level; LongAct showed long-context training signal concentrates in the first 5% of tokens; Cal-OPD (09-21) introduced a self-deviation estimator retaining 52-65% of the teacher-student discrepancy. All three asked which tokens carry signal. IER-OPD asks where the estimate of that signal is trustworthy, which is a different question with a different answer.

The mechanism. Sparse on-policy distillation pays a teacher to supervise a subset of tokens in the student's own rollouts, and every published selector ranks by usefulness, meaning teacher-student disagreement. But the applied update is estimated from a single sampled next token, so it carries variance. At a high-entropy prefix the disagreement can be genuinely large and its sampled estimate can still be near-pure noise: you pay the teacher and receive an unusable measurement. The information-efficiency ratio is a signal-to-noise decomposition in information geometry characterizing relative gradient-estimation error under an optimal scalar baseline. A candidate-set approximation makes it computable without extra teacher calls, and it composes with existing usefulness scores rather than replacing them, retaining the sampled reverse-KL objective. Result: sparse configurations at 0.1% to 1% token budgets match or exceed full distillation with no selection, on mathematical and medical reasoning.

The cross-subfield convergence is the part worth recording. ../ai-routing/llm-routing.md reached the same conclusion from a completely unrelated direction on 09-21, when the decision-primitive calibration failure showed a confident answer is worthless if its confidence is a constant, and again on 09-22 when an independent retrieval test found the returned probability usable for ranking and unusable for gating. The GPU-kernel oracle paper (09-22) reached it a third time, finding benchmark checkers that deterministically miss 78.6% of precision faults while their verdicts feed RL rewards. Three subfields in two days concluded that the estimate's reliability, not its magnitude, is the binding constraint. That is a pattern by this wiki's threshold, and it is a methodological pattern rather than a technical one, which makes it unusually portable.

Harness-Zero, the same day, is distillation at the trajectory level. Harness-Zero distills an optimized agent harness into weights using a harnessing agent that corrects the student's response into the target action space before execution, because teacher and student do not share an output space. Reported: 23.3% → 44.3% with the harness removed, above the 41.7% with it attached. Together with IER-OPD, the day produced distillation results at the token level and the trajectory level, both about extracting maximum signal from an expensive teacher.

Open problem. IER-OPD's evidence is mathematical and medical reasoning only, the same exposure the 09-21 digest flagged for Cal-OPD and the same benchmark family where privileged-distillation gains previously turned out to be a difficulty artifact. The variance argument should matter more on agentic trajectories, where entropy is higher and the sampled estimate noisier, so running it there is both cheap and the most informative experiment available. Nobody has.

2026-09-23: hindsight becomes a distillation source, and two independent groups use it on the same day

This page has spent a month narrowing the question "what makes a good teacher signal" down to selectivity. TIP found most teacher-generated tokens carry no learning signal. The 09-18 trio demoted the teacher on three axes at once. IER-OPD (09-22) matched full on-policy distillation at 0.1% of tokens by selecting for gradient reliability rather than teacher disagreement, shifting the frame from which tokens carry signal to where the signal can be measured. Today two unrelated groups introduced a different kind of asymmetry between teacher and student, and it is not capability at all. It is information about what happened next.

Perplexity's OPSD (09-23) splits production trajectories by outcome and uses a different objective for each half. Successful trajectories go through rejection-sampling fine-tuning with a forward-KL objective, which is mode-covering imitation: keep everything that worked. Broken trajectories get a hint constructed from the tool-call error or the negative user feedback, the model regenerates the step with the hint in context, and the hinted behaviour is distilled back into the unhinted model with a reverse-KL objective, which is mode-seeking and collapses onto the specific correction rather than spreading probability across the failure modes around it. The teacher and the student are the same model. The only asymmetry is that the teacher has seen the consequence. Reported: a 21% reduction in tool-call failures in a live A/B test on a GLM-5.2 base, with trajectories also becoming cheaper.

Taste-Bench (09-23) does the same thing at a coarser granularity. It distils the judgement of a teacher that has already seen the outcome of a decision fork into a student that has not, and reports both better fork choices on unseen tasks and higher end-to-end success on held-out SWE-bench Pro tasks. Two independent groups, one day, the same trick at the token-step level and the trajectory-decision level. This page's pattern threshold is three; two is the point at which to name the category and watch for the third.

Why this is a distinct entry rather than a variant of self-distillation. Ordinary self-distillation asks a model to teach itself using its own current distribution, so the ceiling is what the model already prefers. Here the teacher is conditioned on information from the future of the trajectory, which the student structurally cannot have at inference time. That makes it a privileged-information distillation setup in the classical sense, with the privilege being temporal rather than architectural, and it needs no reward model, no human label, and no stronger model. The forward-versus-reverse-KL split by trajectory class is also a cleaner design than the single-objective post-training stage most of this page's entries use, and it is the first entry here where the objective choice is justified by what the trajectory class is rather than by tuning.

The open risk, stated by neither paper. A model trained on hindsight-corrected versions of its own deployed trajectories inherits its own distribution, including the queries it was already good at. The 09-21 contamination entry built a causal estimator for a related self-reinforcement problem, and no equivalent exists here. The obvious experiment: run OPSD for several rounds on traffic the model itself is shaping, and measure whether the distribution of incoming queries narrows. Perplexity names continual learning as the direction, which makes the multi-round behaviour the decision-relevant unknown rather than a footnote.

2026-09-29: three papers in one day on the supervisor, and one of them takes back an earlier fix

All three change the supervisor and leave the student alone. That is the 08-26 pattern ("the binding constraint is the supervisor"), which had four instances: OPRD (06-05, match teacher hidden states instead of output tokens), OPDVR (08-26) (gate the signal on verified correctness), QAH (08-26) (distill from the original pre-compression model) and RISE (09-07) (build the teacher from the student's own checkpoint history).

• MOPD-Router (09-29) routes the supervisor per token. Instead of one domain-matched teacher per prompt, every token picks from the whole pool by ExpertAlign (does this correction reflect the teacher's post-training specialty?). No labels, no trained router. It beats plain averaging by 5.88 points on unlabeled data and label-routed MOPD by 3.95. This is the first evidence on the 08-11 Motif 3 question of whether multi-teacher OPD consolidates or averages: averaging is measurably the weak option. Fifth instance.
• DCE+SRCL (09-29) lets the self-teacher learn. The privileged teacher is re-initialized from the improved student each round, and shorter verified self-rewrites keep length in check. Qwen3-8B reaches 65.97% Average@12 on four math benchmarks, +35.62 over OPSD. Sixth instance, and in tension with Privileged, but Biased (08-10): DCE says the problem is a stale teacher, 08-10 says the privileged signal itself is biased toward one reference. Refreshing the teacher does not remove the reference. Math only, so the hard-task test that would decide between them is still missing.
• LastOPD (09-29) restricts where the latent supervisor applies, and revises the pattern's first instance. OPRD-style depth-paired latent alignment lifts MATH-500 from 25 to 46 in 10 steps, then collapses to 11 while the alignment metric keeps improving. Aligning only the last layer, for a 10-step crossfade into token OPD, beats token-only OPD by 4 to 5.5 points. OPRD's fix ("change the layer") becomes "change the layer, briefly, then change it back." Together with RetireOPD (09-18, retire the teacher once the gap stops closing), that is two papers saying a supervision signal has a useful lifetime.

The updated pattern statement. The supervisor is still the binding constraint, but the unit of design has shifted from which supervisor to when and where it applies: per token (MOPD-Router), per round (DCE), per layer and per phase of training (LastOPD). None of the three reports teacher compute, the same omission this page has flagged since 09-18.

2026-09-30: the supervisor gets a volume knob, and distillation as an attack gets cheaper

Seventh instance of the supervisor pattern, and a new sub-axis: how loud. DN-MOPD (09-30) finds that multi-teacher OPD on Qwen3.5 fails to beat the best single specialist because instruction-following feedback is several times more spread out than math feedback and dominates the student's gradient. Rescaling each domain by its measured spread fixes it, and the fix is mostly turning instruction-following down. With MOPD-Router (09-29, which teacher per token), the design space is now: which supervisor, when, where, and how strongly. Nobody has combined per-token routing with per-domain normalization.

Same-day entries: LSPD (OPD's reverse KL is KL-regularized policy optimization; off-policy replay matches OPD with 25% of rollout batches, a rollout-cost win), OLIVE (teacher continues the student's prefix; needs only teacher text, not logits), DCSD (separate credit direction from magnitude so a wrong teacher cannot flip updates), AlignOPSD (align privileged feedback to the student's decision, not its timestep), and two vision self-distillation papers (SCOPD, LT-OPD) that recover pruned-VLM accuracy by training against the model's own full-token copy (LT-OPD: 68.6% to 82.3% retained at 5% visual tokens, 85% less KV). The 09-29 DCE+SRCL paper was widely amplified on X as "recursive self-improvement via OPD."

Distillation as attack. Distillation Defenses Easily Break After RL (09-30): defenses scored right after distillation fail once the attacker adds RL, and API-available summaries match full-trace extraction. Behavioral Shadows / ATD (09-30): one chosen word per prompt, on prompts where the shared base model is indifferent, transfers +5.34 HumanEval+ with no task data. Together they say the leak surface is not the trace, it is any output at all from a model with a public ancestor. This extends the 09-09 addendum (distillation reclassified as exfiltration) with the technical reason trace-hiding will not hold.

2026-10-01: scaling laws, a trust region, and the finding that OPD only reweights

Eight OPD papers in one HuggingFace list. Summary: OPD scaling laws, SAKI, KL-free (10-01).

• First scaling laws. Scaling Properties of Same-Family OPD (219 upvotes): early gains are roughly linear in sqrt(reverse KL) from the student's start; in every weak-to-strong pair the student's peak beats its teacher; peak improves with teacher size only up to about the student's size; at matched teacher score, smaller teachers transfer better. A compute claim: the largest available teacher is often the wrong one.
• The supervisor pattern, instances eight and nine. SAKI (HF + Kurate) changes when the teacher intervenes: maximal coupling inside a KL trust region, reverse KL on accepted tokens, direct teacher-top-token targets on corrected ones, correction rate exactly TV distance, plus a 4.22x faster speculative rollout verifier. SCOUT trains the supervisor itself on student prefixes, because the teacher is off-policy on the student's text and degrades as prefixes grow.
• What OPD actually does. KL-free OPD: a +1/-1 direction signal reproduces reverse-KL OPD, and only the direction on a small set of high-disagreement tokens matters. Sparse crosscoders: OPD creates no features and copies none; >98% of frequent features stay within 20% of their firing rate. OPD reweights the student's existing features. This supports TIP (04-16, most teacher tokens carry no signal) and undercuts the "recursive self-improvement via OPD" framing of DCE+SRCL (09-29).
• Multi-teacher, three answers in three days: MOPD-Router (09-29, route per token), DN-MOPD (09-30, rescale per domain), and now C-MOPD (every teacher supervises every sample) and PMOPD (project out cross-task interference in low-dimensional update subspaces, +2.54 on Qwen2.5-7B). No head-to-head exists.
• Rollout cost: ActFirst-OPD decouples acting from reasoning for multi-turn agents, 1.8x to 4.9x faster training. SPSD (HF + Kurate) distills MuZero-style game search into chains of thought, lifting Qwen3-4B-Base math from 24.1 to 36.6.
• Distillation as attack, continued. OpenAI formally accused a China-linked operation of distillation for the first time: about 16,000 extraction requests from 4,000+ users on 07-24/25, replaying encrypted reasoning into another conversation to be "transcribed" (X, 10-01). It closed the replay path and holds streamed output that might expose reasoning. Read with the 09-30 Distillation Defenses page, the defense is still at the API-pattern layer, not the model layer.

2026-10-02: the signal is a direction, and it lives in a few places

Seven OPD papers (10-02) follow the eight from 10-01. RIDE extrapolates the RL-induced hidden-state residual (teacher minus its pre-RL base) at every layer and is the only method whose mean meets or beats the RL teacher on all four pairs; output-space extrapolation is noisy because the LM head damps most hidden-state change. S²D-OPD keeps only the top 10% of states by teacher-reference divergence and wins 7 of 8 settings. LSD routes already-solved prompts to self-distillation against an EMA copy, cutting the length-scaling tax from 19.0% to -3.7%. OASIS fixes self-distillation's collapse with scale (OPSD gain 3.05 at 1.7B, 0.14 at 8B) by supervising verified rollouts. DuoOPD, PivotOPD and AdviSD extend OPD to multi-task, agent recovery and frozen-executor advice.

Pattern established (≥3 papers): the crosscoder study (10-01: OPD reweights existing features), KL-free OPD (10-01: only direction on high-disagreement tokens matters), RIDE and S²D-OPD (10-02) all say the transferable signal is a sparse direction. Implications: smaller or self teachers suffice, and API-level anti-distillation defenses protect a far larger surface than the attack needs (see OpenAI's Moonshot disclosure and the Azure gap, 10-01/10-02).

2026-10-03: rollout policy matters less than KL direction and learning rate

Distillation dynamics and sampling (10-03). A controlled study across Llama3 and Qwen2.5 finds rollout policy (student vs teacher-written) is not the main driver. KL direction shapes accuracy and coverage (forward KL robust to rollout source, reverse KL prefers student rollouts); learning rate governs forgetting and update sparsity. On-policy data helps on harder Countdown variants, but that edge does not reliably survive later RLVR.

• Tension with 10-02: the "signal is a sparse direction" pattern (crosscoder, KL-free OPD, RIDE, S²D-OPD) may partly reflect learning-rate effects on sparsity. Unresolved.
• Finetuning with Sampling (Harvard, X feed): MCMC reshapes expert traces toward the model's own distribution; SFT then rivals RL and OPD while forgetting less.
• Neighborhood OPSD: a pool of perturbed frozen teachers, routed per state, adds +1.67 to +2.75 Average@12 on AIME/HMMT over OPSD (Qwen3 1.7B to 8B).
• Smaller Models, Better Rejects: rejects from smaller frozen models train stronger 7B to 72B students in preference distillation than self-generated rejects; lower-likelihood rejects win for every source.

2026-10-04: weight teacher corrections by the student's verified outcome

R²-OPD (10-04), Kurate cs.LG. Uniform per-token OPD treats teacher preference as correction utility; R²-OPD reallocates supervision by outcome agreement and teacher-student disagreement, keeping dense feedback. Beats standard OPD on all seven math benchmarks (+3.5 at 1.7B, +2.4 at 4B; +1.6 on code). Fourth token-weighting answer after TIP (04-16), MOPD-Router (09-29) and SAKI (10-01), and the first tied to RLVR rewards. Tension with 10-03's finding that learning rate drives update sparsity remains open.

2026-10-05: diversity beats the teacher; defenses fail after RL

Diverse self-training and distillation defenses (10-05). (1) Gurung et al.: elicit distinct approaches first (GROOT tree, Verbalized Sampling list), then solve once per approach. GROOT at 4 samples beats IID at 64 (pass@64 15.5 vs 7.9); training only on incorrect diverse samples still beats RFT; a 4B model's diverse self-samples beat IID distillation from a 235B teacher (22.8 vs 13.4). (2) Distillation Defenses Easily Break After RL (2609.35699, Kurate): defenses that hold at the distillation checkpoint fail once the attacker adds RL; API-available signals suffice. Shift: the value of distillation data lies in strategy coverage, and output-level anti-distillation is likely a dead end.

2026-10-06: two channels per teacher, and BF16 hides most of the update

Multi-teacher OPD (10-06). (1) Latent-MOPD: the first representation-level multi-teacher OPD; supervises with routed specialists' late-layer hidden states and tokens, shifting from states to tokens over training. Beats token-only, representation-only and uniform-averaging baselines on all nine benchmarks; the student beats the best single teacher on most. Interleaving teacher domains within an update collapses the representation-only control. (2) From Gradients to Capabilities: Adam's first moment makes teachers' updates 0.83 cosine-similar despite different gradients; token averaging favors long responses; BF16 rounding hides updates (97% of FP32 master weights move, 7-11% of BF16 weights change). Tension: this weakens sparsity measurements taken on BF16 checkpoints, including evidence behind the 10-02/10-03 "signal lives in a few places" entries. Also: OPSFT (shorts) transfers on-policy generalization to SFT by constraining updates to the on-policy direction; Pivot-SD trains diffusion LMs only on high-information-gain commitments (the TIP idea, 04-16, for diffusion).

2026-10-07: distil the sharpened distribution, distil only what the harness adds

• OPPD (10-07). On-policy distillation of a teacher's power distribution (each answer's probability raised to a power, renormalized). One generation beats 64-candidate power sampling and beats GRPO from the same checkpoint by 3.8-5.4 points with no reference answers; +9.3 stacked after GRPO. Distillation as a compressor of test-time search.
• Harness-Aware Distillation (10-07). Contrast the teacher with and without harness info and train the student on the difference; 63.4% vs 47.0% on unseen ALFWorld, student beats its 8B teacher. Plain OPD changed attention to harness info but not success. ASCENT distils verified deployment experience into LoRA fast weights via a frozen hindsight teacher.
• Shift: distillation targets are getting more specific: not "the teacher's tokens" but "the teacher's sharpened mode" or "the part of the teacher's behaviour the scaffold cannot supply".

2026-10-08: reliability over coverage

OPD supervision-reliability cluster (10-08). The day's top HF paper (159 upvotes) finds cross-tokenizer OPD needs only strict 1:1 alignments and a student-chosen top-16 shared-vocabulary slice; adding span supervision for misaligned groups lowers accuracy because its gradients disagree with the strict ones. This revises the BLD (04-17) framing above: the tokenizer mismatch mostly does not need bridging. Three same-day papers gate the teacher (alignment, outcome via DiffGate, pivotal mistake via NVIDIA's PivotOPD, which lifts replayed ALFWorld success 8% to 59% by fixing one early turn). Rubric-privileged OPD before RL reduces reward hacking versus SFT+RL. Pattern (n=3): selective distillation, following TIP (04-16) and OPPD (10-07).

2026-10-09: the teacher is an implicit reward model

OPD teacher-as-reward cluster (10-09). Gains and Collapse reframes OPD as RL: teacher log-probs reward student behaviors the teacher rarely produces, so OPD sharpens sampling without adding capability, and collapses into long repetitive outputs when that implicit reward misaligns (fix: mask unhealthy responses or start from SFT). This gives the Extrapolation Cliff (05-14) a mechanism. Same day: NP-OPD adds negative-policy rollouts; Δ-MOPD distils teacher-minus-base shifts (+4.11 math with three teachers); SRD distils hindsight into foresight (2B agent 0% to 60.6% where RLVR had no signal); ReSAIL stops iterative self-distillation collapse (+22.5%). Pattern (three days running): OPD's lever is the reliability and shape of the teacher signal, not its coverage.

2026-10-10: OPD organizes knowledge, it does not add it

OPD skills not knowledge (10-10). In a controlled synthetic setup, reverse-KL on-policy distillation transfers compositional skill to unseen reasoning structures but almost no new facts; forward KL restores fact transfer; student rollouts drive multi-step execution. Real QA and competition math show the same asymmetry. Pattern (n=3, 10-07 to 10-10): OPPD distils a sharpened mode, Gains and Collapse calls the teacher an implicit reward, and now OPD is shown to recombine what the student already knows. For compression: on-policy distillation into a small model will not move world knowledge; do a forward-KL or SFT phase first. SGUID (same day) keeps only skills with a steady training signal and matches banks 11x larger.

2026-10-13: check teacher-student alignment before paying for on-policy rollouts

Semi-OPD and friends (10-13). Semi-OPD (NVIDIA/UCLA) distils on one cached set of initial-student rollouts and beats OPD in 14 of 17 pairs (1.5B-235B), up to +13.6% accuracy and 11.4x faster training; OPD only wins when initial top-token overlap is high, because misaligned students drift off the teacher's distribution as rollouts lengthen. Relative policy differences: task-training differences (RL, SFT) transfer, scale-only differences do not. SynthSelf: small-model continual learning via a same-family aligner rewriting data in-distribution. UTT: distil the gap into a reusable text Primer (KernelBench 9.4% to 48.6%). Pattern now n=5 since 10-07: OPD sharpens and organizes (10-07, 10-09, 10-10), and today it also has a cheap pre-flight test for when it will hurt. Adversarial side: Anthropic reports distillation defenses cutting harvested data 90-99% by hiding reasoning traces (responsible-ai 10-13); summarized traces alone cut a distilled model from 80.4% to 71.8% on SWE-bench Verified.

2026-10-14: supervise only the uncertain decisions

BranchOPD, PR-OPD, progressive contraction (10-14). BranchOPD branches student rollouts at uncertain, teacher-disagreeing tokens and reuses prefixes (19-31% fewer student tokens than 7 rollouts, +18.55 avg@16 math over vanilla OPD). PR-OPD finds a skill hint moves under a quarter of token probabilities but over 80% of hidden states, so it aligns hidden states at every layer (+14.0 WebShop over GRPO). Progressive contraction trains on as little as 0.22% of full SFT optimization. UltraBERT (shorts) distills only on 5%-masked sequences. Pattern (n=4 with Semi-OPD, 10-13 to 10-14): selective supervision beats uniform supervision. Contradiction to track: PR-OPD says logit-only losses under-read the teacher.

2026-10-16: four fixes and a reality check

OPD guarding, critique, action (10-16). Road MOPD uses non-routed teachers as critics and projects conflicting gradients (+5 points in the disadvantaged domain under high conflict). CriticOPD restores teacher "recoverability" on wrong prefixes with a critique-and-rewrite step, a concrete mechanism for the 10-09 Gains-and-Collapse failure. ActionOPD distils semantic actions from a black-box teacher (+3.3 to +6.9 OOD tool-use points, no shared tokenizer). Reality check: 10 self-distillation methods vs GRPO over 132 runs / 12,124 H800 GPU-hours: none consistently better, final scores hide instability, more cost ≠ more performance.

2026-10-17: gate the policy shift by its history

Waypoint-OPD (10-17), top HF paper. Distill the teacher's RL-induced log-prob shift only where the early shift (start → mid checkpoint) and late shift (mid → end) agree in direction. One extra frozen-teacher pass. Best domain average in all six settings (Qwen3-4B RL teachers; 8B, 14B, 30B-A3B students; math and code). The fifth OPD refinement in two days and the first to read the teacher's training trajectory. Also today: Hy-Memory-30B-A3B (Tencent) trains a memory writer by Memory OPD then RL and beats Claude Opus 5 as the writer inside Mem0/MemOS (55.49 vs 54.62 average across four harnesses); SCOPED distills capability-specific VLM experts into one student.

2026-10-20: disaggregated teachers, mechanism supervision

Distillation shorts. DisaggKD moves teachers to their own nodes and computes an exact sharded KD loss without full-vocabulary logits; trains pairs that OOM colocated. OPMD distills teacher intermediate computation through a student-compatible mapping on on-policy rollouts. TreeOPSD turns tree-backed-up rewards into bounded closed-form teacher distributions for agentic RL. Hy-Memory-30B-A3B uses on-policy distillation then RL inside the deployed harness.

2026-10-22: selecting the prompt, the rollout, the privileged information

Distillation shorts. Ask-to-Distill trains a question synthesizer on teacher-student disagreement (no answers needed) and matches real-question OPD banks. A KL-contrast metric predicts which privileged information helps self-distillation. ProD penalizes loop "stagnation" in distilled 0.6-8B agents. Kurate: Projected Distillation; correctness-filtered self-distillation. Signal-selection now at three levels: token (TIP, 04-16), rollout, prompt.

2026-10-23: the self-teacher pattern spreads

Distillation shorts. BSD uses an external teacher's trace only as privileged context for a copy of the student, which scores rollouts in the student's own vocabulary: one recipe for shared-tokenizer, cross-tokenizer and closed-API teachers. RSD (Meta) does the same with hindsight from the finished trajectory instead of a teacher (up to +59.7% relative on ALFWorld/ScienceWorld/SearchQA). TraceForge regenerates hidden reasoning in the student family's style from context plus answer (+3.7 HumanEval+, +7.1 MBPP+ over Fable 5's visible thinking). Muse Glimmer 30B distills Muse Spark into a one-GPU agent model. Pattern: at least four papers in ten days (PR-OPD 10-14, KL-contrast 10-22, BSD, RSD) use a context-conditioned student copy as teacher. Cross-tokenizer transfer keeps converging on text (ActionOPD 10-16, BSD).

2026-10-24: is part of the distillation gap a rounding error?

A Fix for Softmax and Cross-Entropy (2610.31642): standard soft-target cross-entropy has a gradient noise floor as the student approaches the teacher ("Softmax Cancellation"), so students drift away; a direct-residual implementation fixes it without higher precision. Toy scale only. Open test: fused large-vocabulary CE in on-policy distillation of a 7B student.

2026-10-25: the self-teacher gets an async trainer

AsyncOPSD (2610.25880): decouples rollouts from learner updates in on-policy self-distillation; shared weights avoid multi-model sync, staleness-aware policy control plus token-entropy weighting handle old rollouts. 1.96-2.25x throughput over synchronous OPSD with slightly higher math accuracy. Systems complement to BSD/RSD (10-23). Kurate also re-lists Projected Distillation (cs.LG #2) and correctness-filtered OPSD (cs.AI #4), both first noted 10-22.