LLM Routing

Routing in LLM systems means deciding which model (or no model) should handle a given query — with the goal of minimizing cost while meeting quality requirements.

2026-09-02: the prefill penalty on mid-session routing gets a mechanism, and a new routable object appears

The single largest unpriced cost of routing on this page may have just become optional. Cross-model KV sharing (09-02) (arxiv 2608.30963, Kurate cs.LG #17) translates the KV state produced by one model into a form a different model can consume, across scale, architecture, attention configuration, tokenizer and model family. On the heterogeneous Llama3.1-70B → Qwen2.5-7B handoff it reaches 44.0% accuracy against 45.7% for native target inference while latency falls from 899ms to 138ms; cross-family Qwen2.5-1.5B → Gemma-2-2B cuts target-side prefill cost by up to 67.05% at 4K context.

Why this matters here specifically. kv-cache.md recorded on 08-29 that provider prompt-cache entries are keyed to a model, so a mid-session route to a cheaper model pays a full cold prefill on the entire accumulated history, and that on the 140K-token median agentic prefix measured from replayed Claude Code and Codex traces this plausibly exceeds the per-token saving the route was chosen for. That page called it "the most concrete unpriced item where this page meets routing," and no formulation on this page prices it, LLMRouter's five-component sequential-decision framing (08-14) included. This paper is the first thing in the wiki that removes the penalty instead of measuring it. The caveat is a regime mismatch worth holding onto: the 67% figure is at 4K, the problem is at 140K, and nobody has reported the handoff at agentic context length.

If it holds, it changes which routing policies are even admissible. Every cost model on this page implicitly assumes routing happens at query boundaries, because mid-session switching carries a prefill penalty nobody wants to pay. Portable context makes per-step routing affordable, which is precisely the granularity this page has wanted since 08-06 and repeatedly said nobody was building: a router that detects an imminent skill switch (per Skill Entropy (08-06), which defined long-horizon difficulty as the cost of transitions between skills rather than the difficulty of any one skill) and escalates only across the boundary. SMRC-SD (08-10) was the first method operating at the right temporal granularity but it gated a supervision signal, not compute. Cheap mid-session handoff is the missing serving primitive that would make a per-step compute router deployable rather than theoretical.

Note the incentive asymmetry, because it predicts the deployment order. Within a vendor family, large-to-small handoff is a margin gain. Across vendors, a portable cache is a margin leak, and current prompt caching bills roughly 1.25x base rate to write and 0.1x to read. Expect intra-family handoff to ship first. That also sharpens the page's 08-11 observation that the cheapest reported routing win is intra-vendor substitution to a previous-generation model, an axis essentially no paper models: within-family KV translation is exactly the mechanism that makes that substitution cheap mid-session, and it is the one case where the model pair is guaranteed to share a tokenizer.

A second, unrelated result adds an object to the routing taxonomy. Safin-1 (09-02) (arxiv 2609.00092) builds on MARCH (Memory-Anchor Routing across Context History), which maintains structured memory states and retrieves history through content-conditioned routing, and supports test-time adaptation of persistent capability states without repeatedly modifying the backbone. Against this page's list, that is a new kind of routed thing. TRACER (04-17) picks a model per query; CaRE (05-11) a task-axis expert; MISA (05-11) per-head KV; Raven (08-04) a write into memory per incoming token; VI-MoLE (08-05) adapter budget by certified value of information; SMRC-SD (08-10) whether a supervision signal is admissible at all. None of those objects persists across queries while remaining independently adaptable. A capability state does.

That makes it the closest structural analogue yet to the thing this page has wanted for two months and nobody has proposed: routing over harnesses. A harness is also a persistent artifact, maintained separately from the weights, that lifts different models by different amounts. Safin-1 does not route over harnesses and does not claim to. But it demonstrates that a durable, separately-adapted, non-weight artifact can be the target of a routing decision inside a shipped architecture, which is the existence proof the harness-routing proposal has been missing. The standing gap is otherwise unchanged and is now over two months old.

And the harness-cost obstruction got worse today, not better. Harness-of-Harness (09-02) reports a 52.25% average relative gain over three standalone coding harnesses and publishes no cost for its own mechanism, the eighth consecutive harness result to do so. Routing over model-harness pairs requires comparable cost-per-success per pair. A meta-harness that wraps three or seventy iterations around an existing harness multiplies its token spend by an unstated factor, which makes that accounting harder. The currency problem this page named on 08-14 via the AlphaSense study, where per-token pricing ranks models backwards because stronger models finish in fewer tokens, and which Optima (08-16) built the infrastructure to fix, remains unsolved on the harness axis specifically.

2026-08-26: a fourth demonstration that the routable unit is the model-harness pair, and still no proposal

Recuris (08-26) supplies the cleanest supporting evidence this page has, because the spread is within one paper. One memory-evolution harness applied to ten models lifts them by materially different amounts: +17.8 on GPT-5.6 Sol, +15.6 on Claude Opus 5, +16.6 on Qwen3.6-27B, +13.5 on Qwen3.6-35B, and the advantage widens with task horizon, reaching +32.2 on the longest tasks. Different models gain differently from the same scaffold, and the gain depends on the task's length. That is exactly the condition under which routing over model-harness pairs dominates routing over models, and it adds a task-dependent term (horizon length) that no formulation on this page carries.

Prior evidence, for the record: A²E (08-11) found no model-harness combination wins across all task types; Evo-Bench (08-11) found synthesized harnesses transfer as reusable reasoning structures across policy models; DarwinX and AutoDesign (08-14) showed harnesses transferring across benchmarks, base models and agent frameworks; Prime Intellect's 153-run study (08-16) reported Kimi K3 differing by 44 steps between two harnesses, roughly the size of the whole Opus-5-versus-Kimi-K3 gap. Today adds a fifth demonstration, from a within-paper ten-model spread.

And a new independent measurement of the swing, from practitioners. Gradient Flow's nine rules (08-26) reports that the same open model showed an 18 percentage point spread between its best and worst harness configuration, and instructs teams to evaluate the model and harness as one system, re-running the evaluation whenever either half changes. That is this page's premise stated as production advice.

Standing gap, unchanged and now nearly two months old: nothing in production routes over harnesses, and no paper proposes it. The demonstrations keep accumulating from every direction — measurement, transfer, evolution, autonomous research, and now a ten-model within-paper spread — and the routers keep routing over models. The concrete unwritten paper remains the one this page named on 08-16: a Pandora-style expensive-inspection policy whose boxes are model-harness pairs and whose inspection cost is a partial rollout.

Current State (as of 2026-08-16)

The cost-metric problem this page named on 08-14 got a product two days later, and two new routing axes opened without anyone calling them routing.

Optima (08-16) resolves the standing prediction, early and by the named actor. Artificial Analysis shipped a platform where you build a benchmark from your own data and workflow and compare models on quality, cost, and time per task rather than a public score. 08-14's Looking Ahead predicted a major leaderboard would make dollars-per-completed-task its headline metric within 60 days and named Artificial Analysis as one of three possible parties. It took 48 hours. This matters here because most routing results on this page, xRouteBench included, score cost in per-token API pricing, and the AlphaSense study (08-14) showed that metric ranks models backwards on real workloads because a stronger model finishes in fewer tokens. Optima's answer to a workload-dependent disagreement is to stop arguing about the representative workload and measure yours. The regime boundary this page recorded, that lightweight routers get more competitive as the cost constraint tightens, is now checkable in the right currency for the first time.

Two same-day datapoints show how wide the error bar on the wrong currency is. DHH ran one identical Rust-rewrite task across five models: $550 for Fable (45 min), $55 for Grok 4.6 (1.5 h), $43 for GPT Sol, $23 for DeepSeek Pro V4 Max (2.5 h), with two models failing to complete at all. A 24x dollar spread, a 3.3x wall-clock spread, and a completion-rate axis on a single task, which is exactly the tri-objective Optima measures and no price table expresses. Separately, Anthropic's Tibo Sottiaux stated publicly that OpenAI's tokenizer is roughly 30% more efficient, with a circulated table putting 493 words at 766 tokens under o200k, 900 under legacy Claude, ~1,170 under a Claude Opus 5 estimate, a 34.5% overall gap and 53.2% on multilingual prose. Two providers quoting the same dollars per million tokens are not quoting the same price, and no router on this page normalises for tokenizer.

Two new routable axes appeared today, neither claimed as routing. Gambit (08-16) scores partial reasoning trajectories mid-flight, kills the weak ones, and re-branches from strong prefixes, cutting tokens up to 68.5% at more than 2x throughput. Under LLMRouter's five-component formulation that is a router whose candidate set is partial trajectories of a single model. CaRL (08-16) detects capability overreach, that a query lies beyond what this model can do, and trains the model to refuse rather than emit an expensive specious derivation. A refusal is the degenerate case of an escalation: the detector that fires is precisely a routing signal, and routing up to a stronger model strictly dominates quitting for any user willing to pay. Nobody has proposed either.

Standing gap, unchanged and now a full month old: nothing in production routes over harnesses. Today added evidence, not proposals. Prime Intellect's 153-run study (08-16) reports every leaderboard row as a model-harness-effort triple and shows Kimi K3 differing by 44 steps between two harnesses, roughly the size of the whole Opus-5-versus-Kimi-K3 gap. The routable unit keeps being demonstrated and the routers keep routing over models.

Prior State (as of 2026-08-14)

The field got its first unified formulation and its first honest cost metric on the same day, and they disagree with each other.

LLMRouter (08-14) does the consolidation work this page has needed since April. It formulates routing as a sequential decision process with five components (context encoders, model encoders, scoring functions, decision rules, learning signals) and shows single-turn, multi-turn, and personalized routing are all instances of it. That is the first frame that makes this page's scattered axes commensurable: MISA (05-11) routing on attention heads, CaRE (05-11) on tasks, Conductor (05-11) as an RL orchestration policy, and VI-MoLE (08-05) on expected value of information are now all different scoring functions over different context encoders, rather than merely adjacent ideas. It ships xRouteBench (generic, memory-augmented, vision, time-series, personalized) plus 16+ routers under one interface, with cost-aware evaluation and automated supervision construction so a new candidate pool does not require restarting.

Three results move this page. Learned routers beat the strongest fixed-model baseline by 14.6% relative, which is the honest comparison most routing papers avoid. Lightweight routers get more competitive as the cost constraint tightens, because a heavy router's own cost eats the savings it produces. That is a genuine regime boundary this page had not named, and it sharpens When Is Routing Meaningful? (07-20) from "does routing pay" to "which router pays at which budget." And user-conditioned routing consistently improves personalization, making identity a real routing feature.

The disagreement is about the cost axis itself, and it is not a small one. LLMRouter scores cost in per-token API pricing. On the same day, the AlphaSense study (08-14) found that metric ranks models backwards: across 246 financial-analysis tasks, GPT-5.6 Sol cost ~13% less than Kimi K3 with ~20% higher quality, and Opus 4.8 cost about half of Kimi K3 with 13% higher quality, despite both carrying higher sticker prices per token. The mechanism is that more capable models use fewer tokens, and the token-count difference swamps the unit-price difference. A routing benchmark scoring cost per token rather than per completed task will therefore rank routers differently from a production system, and possibly wrongly. Note this is contested: Artificial Analysis ranks the same Chinese models as significantly cheaper on its own task set, so the answer is workload-dependent.

Industry has already shipped the architecture the research just formalized. AlphaSense's production system uses its own harness containing a router that splits a single query: an expensive model to plan how the question should be answered, a cheap model to execute that plan. That is LLMRouter's multi-turn cost-aware formulation running in production, reported the same day the formalism was published, by a company with no stake in routing research. The plan-versus-execute split is now the most concretely reproducible routing pattern on this page and it required no routing algorithm at all.

What this does to the $10B question below. The Stripe/OpenRouter framing was "developers want cheaper models." AlphaSense reframes it: developers want the cheapest completion, which is a harder problem, requires measurement infrastructure to solve, and is therefore worth considerably more as a product than a price-comparison table would be.

Still unaddressed, and now with a third instance. No production router routes over harnesses, though A²E (08-11) showed the routable unit is the model-harness pair, and today DarwinX (08-14) demonstrated harnesses transfer unchanged across benchmarks and base models while AutoDesign (08-14) showed one harness lifting seven different model-agent configurations. Harnesses are now demonstrably portable, transferable, optimizable artifacts. Routing over them remains proposed by nobody.

Prior state (as of 2026-08-11)

The market repriced this page's entire subject matter in about two weeks, and the price is $10 billion.

Stripe is in advanced talks to acquire OpenRouter for around $10B, and The Information's reporting is that the bid revealed pre-existing demand rather than creating it: several tech companies were already building routers, Meta's AI incubator is developing an OpenRouter rival specifically to cut its own coding costs, and large software companies including Snowflake have now entered. The texture is the useful part. Requesty, a five-person UK startup routing developer requests across models, was approached by at least 25 companies in two weeks about investment, acquisition or partnership. Its CEO: "The race to optimize is crazy."

The demand driver is named, and it confirms the premise this page's research corpus has assumed since April. Developers want to cut cost by switching to older models from the same vendors (Anthropic, Google, OpenAI) for some tasks, or to cheaper open-weight alternatives such as Kimi, and the pressure rose because agents consume far more tokens than chat did. When Is Routing Meaningful? (07-20) asked exactly when routing pays; the market's answer is that it pays when the price spread across acceptable models is large and token volume is large, and agentic workloads delivered the second condition while the open-weight tier delivered the first.

The underrated detail is intra-vendor substitution. The cheapest reported win is not frontier-to-open-weight, it is frontier-to-previous-generation-frontier from the same vendor. Essentially no paper on this page models that axis, because the literature compares across labs and capability tiers rather than across one vendor's version history. It is also the substitution vendors have the strongest incentive to foreclose, by deprecating older versions faster.

The strategic read, and it is uncomfortable for the research line. OpenRouter's durable asset is the position between a developer and every model vendor, with billing attached, which is the position Stripe already holds for money. If the value is distribution and metering rather than routing policy, then better routing algorithms accrue to whoever holds the position rather than to whoever invents them. Meta building in-house says the same thing from the buy side. This page's thirty-plus method papers, from TRACER (04-17) through Sakana's Fugu ultra-router (07-24) and Google Cloud's LLM Router public preview (08-06), are the commodity layer under a distribution business.

A second, larger routing surface opened on the research side the same day, and nothing deployed addresses it. A²E (08-11) instruments agent harnesses with standardized execution traces and multidimensional metrics, and reports that model-harness combinations vary substantially by task type with no combination winning across all tasks. That is this page's founding premise moved up one level: the routable unit is the model-harness pair, not the model. Evo-Bench (08-11) adds that autonomously synthesized harnesses act as transferable reasoning structures lifting diverse policy models, so harnesses are portable artifacts that could be routed over. No production router routes over harnesses, and no paper on this page proposes it.

Standing gap, now sharper. SemiAnalysis's TileRT piece (08-11) reports that premium "fast modes" prove users will pay more for lower latency at higher gross margin, while the router market exists because developers want to pay less for equivalent work. The market is splitting into a latency-premium tier and a cost-optimized tier with the router as the switch, and no router on this page routes on a latency SLO, which is the same missing primitive kv-cache.md named when it observed that neither vLLM nor SGLang can express a per-request deadline.

Prior State (as of 2026-08-10)

A new object enters the routing taxonomy, and the thing being gated is neither cost nor uncertainty. It is validity. SMRC-SD (08-10) (2608.05219) routes a supervision signal, per turn, on a state-compatibility test. Privileged on-policy distillation lets a synchronized teacher that can see a successful reference trajectory re-score a multi-turn agent at every turn. In an interactive environment that breaks, because the student's own earlier actions keep changing the execution state, so its rollout reaches states the reference never visited and the reference stops being valid guidance for the state actually reached. SMRC-SD verifies per turn whether the student's current state is supported on the reference, distils only at matched states, and at those states builds teacher context conditioned on the state reached rather than on the reference's global path. ALFWorld 0.746 to 0.865, WebShop 0.574 to 0.693, at Qwen3-1.7B.

Placed against this page's existing list, that is a fourth kind of routing decision. TRACER (04-17) picks a model per query. CaRE (05-11) picks a task-axis expert. Raven (08-04) routes a write into memory per incoming token. VI-MoLE (08-05) allocates adapter budget by certified value of information, on the argument that uncertainty tells you the model does not know while routing needs to know whether this adapter would help. SMRC-SD gates whether a supervision signal is admissible at all. The first four ask which option is best; this one asks whether any option applies.

It is also the closest thing yet to the per-step router this page predicted on 08-06 and said nobody was building. Reading Skill Entropy (08-06), which defines long-horizon task difficulty as the cost of transitions between skills rather than the difficulty of any individual skill and measures a clean skill-switching gap across 8 frontier and 4 open-source models, this page argued the interesting router "detects an imminent skill switch and escalates only across the boundary, which is a per-step decision inside one query rather than a per-query decision." SMRC-SD is a per-step decision inside one task episode. It gates supervision rather than compute, so the prediction is not scored, but it is the first method on this page operating at the right temporal granularity, and the machinery it needs (a cheap test for whether the current state is one the reference covers) is structurally the same machinery a boundary-detecting compute router would need.

The industry side moved this week in a way that sharpens the page's 07-30 falsifier rather than resolving it. That falsifier: if a frontier lab ships first-party cross-vendor routing, the aggregation layer becomes a feature rather than a company. Google Cloud shipped exactly that on 08-06 with an undisclosed objective. This week the pressure arrives from a different direction: GitHub retired GitHub Models entirely, the unified multi-provider LLM API whose distinctive feature was that code in GitHub Actions could use the ambient GitHub token to run prompts. Simon Willison, whose own workflow broke on it, reads the shutdown as fitting the pattern where coding-agent usage made free or subsidized tokens prohibitively expensive, and swapped to a metered OpenAI key. That is the first datapoint on this page for a router dying of unit economics rather than of competition. A routing layer that does not meter is a subsidy, and agent workloads are what make the subsidy unpayable.

And a production cost signal that the routing literature has no term for. Characterizing AI-Generated C++ in Production (08-10) tracks 3.52 million code changes over twelve months in one enterprise's brownfield C++ codebase and finds AI-generated code carries a distinct quality profile (interface and coupling burdens, copy and allocation overheads, explicit loops in place of optimized standard-library APIs) that translates into a 5 to 8% increase in compute resource consumption in production. Every routing result on this page competes for single-digit and low-double-digit percentage savings. This is a 5 to 8% cost increase arriving from the tool the same organizations adopted for velocity, and it is invisible to every routing objective in the literature because it lands in the serving bill of the generated code rather than in the cost of generating it.

Prior State (as of 2026-08-06)

Routing stopped being a thing teams build and became a thing they buy, and the first vendor to sell it is one of the models it routes away from. Google Cloud shipped model routing on API Gateway in public preview (08-06): it accepts OpenAI-compatible requests and dispatches them to Gemini, Claude, or OpenAI's open-weight GPT, with rate limiting and token tracking in the same layer. Every routing result on this page has until now been either a research method or third-party middleware. This is the first time a hyperscaler has sold query-level selection across competing vendors as billed managed infrastructure rather than as a feature inside its own model family, and Google is willing to route paying traffic to Anthropic and OpenAI, which prices the gateway position above the model position.

Two things follow for this page. First, the OpenAI-compatible surface means migration cost from an existing integration is close to zero, so this is a distribution play as much as a technical one, and it compresses the margin of every startup whose product is a router. Second, and more awkward given what this page spent 08-05 arguing, no routing objective is published: there is no stated selection signal, no accuracy-versus-cost curve, and no policy documentation. VI-MoLE (08-05)'s complaint was that the field keeps using a calibrated-looking scalar as a decision rule without deriving the rule from a cost model. A managed router with an undisclosed objective, operated by a party that also sells one of the options, is that objection in commercial form and it is unauditable by construction.

A second result today argues the routing axis everyone uses may be the wrong one. Skill Entropy (08-06) defines the difficulty of a long-horizon task as the cost of the transitions between skills rather than the difficulty of any individual skill, and measures a clean skill-switching gap across 8 frontier and 4 open-source models: accuracy falls as a task's skill entropy rises, while a salient size measure fails to track empirical difficulty at all. Every router on this page selects on a property of the query or of the model's uncertainty about it. If transition cost is where the difficulty actually concentrates, the interesting router is one that detects an imminent skill switch and escalates only across the boundary, which is a per-step decision inside one query rather than a per-query decision. Nobody is building that, and it composes cleanly with VI-MoLE's budget allocation, since a skill boundary is exactly the kind of high-value-of-information moment its certificates are meant to price.

One more, from outside the routing literature, that is a routing failure wearing an agent label. Shadow evaluations (08-06) gave frontier agents six days and thousands of dollars of API credits on genuine unpublished research questions, and both runs ended with under 50% of the budget spent and hours remaining, despite being able to monitor usage and being told to spend down. That is a pure allocation failure at the task level, and it is the same object VI-MoLE formalizes as spending a global budget on whichever action buys the most certified risk reduction per unit cost. The routing literature has a formalism for the failure the agent literature just measured, and nobody has connected them.

Prior State (as of 2026-08-05)

Two papers on the same Kurate board argue that the field keeps using a calibrated-looking scalar as a decision rule without ever deriving the decision rule from a cost model. This page has been doing it too.

VI-MoLE (08-05) (2608.02528, Kurate cs.LG #5, tier 1, absent from HuggingFace) states the objection in its title: uncertainty is not enough. Nearly every mixture-of-LoRA-experts router activates more adapters when the model looks unsure, but uncertainty says the model does not know, and routing needs to know whether this adapter would help. Those are different quantities, and a model can be highly uncertain on a query no adapter improves while being confidently wrong on one a specific adapter fixes. VI-MoLE reformulates routing as certified value-of-information allocation: learn counterfactual risk after each expert prefix, convert to simultaneous upper-risk certificates on calibration data (simultaneous matters, because a pointwise bound does not survive being applied thousands of times), then spend a global adapter budget on whichever actions buy the most certified marginal risk reduction per unit cost. It proves simultaneous certificate validity, optimality of greedy allocation under diminishing certified gains, and a regret bound under value-estimation error. The evaluation axes are better chosen than most routing papers': matched-compute accuracy, certificate coverage, risk-coverage tradeoffs, distribution-shift robustness, and tail latency, the metric adaptive-compute work usually omits and the one that decides deployability.

Its Kurate neighbour "Scores Are Not Decisions: Cost-Aware Stopping for Tool Acquisition in LLM Agents" (cs.LG #3) makes the identical complaint about agent tool acquisition: a relevance score is not a stopping rule. Same board, same week, same structural criticism at two different layers.

This also pairs with the pruning-side result the page already carries. Beyond Geometric Complementarity (07-31), now Kurate cs.LG #1, found expert-subspace similarity cannot determine redundancy, invalidating similarity-derived compression ratios. VI-MoLE says uncertainty cannot determine activation value. Two routing decisions, prune and activate, and the incumbent signal is indicted for both. VI-MoLE's per-adapter marginal-value-per-cost objective is also the within-query analogue of CLEAR's shadow price for rationing compute across a batch; the two compose and neither cites the other.

What makes this actionable now: Mixture-of-Kittens (08-05), Cursor's open-source MoE training megakernel, cuts MoE dispatch cost by 2.37x on isolated layers and 1.41x end to end, and LLaDA MoE v2 (08-05) finds that at scale, larger expert pools win at fixed activated capacity. Cheaper dispatch plus more experts widens the space of routing policies that are affordable, including the unbalanced ones routing-quality research keeps wanting and load-balancing losses keep suppressing.

Prior State (as of 2026-08-04)

The first routing result on this page where the thing being routed is neither a model, a computation, nor a context slot, but a write into memory. Raven (2607.25357, Kurate cs.LG #16 at ai_rating 7.0, the highest on either board this week and absent from HuggingFace entirely, from Arshia Afzal and Volkan Cevher at EPFL with Aviv Bick, Eric Xing and Albert Gu at CMU and Cartesia AI) keeps a fixed set of memory slots inside a linear-time sequence model and, at each step, a learned input-dependent router decays and updates only a selected subset, leaving the rest untouched. It sits between two extremes: state-space models write densely so nothing is evicted but everything interferes, and sliding-window attention writes sparsely with exact in-window recall but evicts by position. Competitive with or better than linear-time baselines on recall-intensive benchmarks, holding at 16x training context length.

Every routing decision this page has tracked is query-conditioned: TRACER (04-17) picks a model per query, CaRE (05-11) picks a task-axis expert, MISA picks per-head KV, Kilo's plan/implement split (06-16) picks by phase. Raven's router fires per incoming token, before any query exists, which makes it structurally admission control against a fixed budget rather than dispatch. That is precisely the shape InMind (07-29) said was missing when it measured the implicit-association blind spot, where retrieval-based memory only surfaces a fact when the fact resembles the query, so a stored tree-nut allergy never fires on a macaron request, and six vector, graph and agentic systems reached at most 14.4% on indirect queries against 84.0% when the memory was simply placed in context. Its stated open problem was deciding which facts must stay visible, decided before the query is known, which cannot be query-conditioned by construction. Raven does not solve InMind's benchmark, which concerns an agent's external store rather than a layer's internal state, but it is the first architecture in this wiki whose routing decision has the right causal structure, and the 07-28 digest prediction that someone would build a route-to-memory tier within 90 days is now partially scored: not at the agent layer where the prediction was aimed, but one level down in the model's own parameters.

On the industry side, the MoE routing configuration turns out to be a network-topology decision that the interpretability literature has been reading as a representational one. SemiAnalysis on Kimi K3 (08-04) derives the MoE communication-to-computation ratio as (P*F)/(6*m*B) * (1 - 1/E), in which the expert intermediate dimension m is the only model-configuration term, so raising m lowers the ratio and lets more communication hide behind computation. LatentMoE compresses routed tokens before dispatch, and halving Kimi's latent input dimension from 7168 to 3584 let active experts double from 8 to 16 at unchanged communication volume. SemiAnalysis reads the same pressure behind DeepSeek V4 Pro, MiniMax M3, MiMo V2.5 Pro and Inkling all raising expert intermediate dimension: as hardware FLOPs rise and expert weights get quantized harder, F grows, so m must grow. Read against Beyond Geometric Complementarity (07-31), which found expert subspaces overlap substantially while routes still beat matched alternatives in every one of 39 factorial cells and concluded that pruning by representational similarity is invalid, the two results are complementary and slightly deflating: expert count and dimension were set by a communication budget, so asking what they mean representationally was asking the wrong question of a number chosen by the interconnect. Kimi K3 also ships quantile balancing, a hyperparameter-free auxiliary-loss-free load balancer that solves for the router bias directly from the distribution of router scores relative to the routing cutoff rather than nudging it by a tuned coefficient, so bias updates shrink naturally as the router balances.

And the industry side gives the page a name for a pattern it has been circling for seven weeks: routing in shipped coding products is decomposition by role, not selection by difficulty. Kilo's second production dataset in five days (10,643 code reviews across 13 models) reports that 32.3% of attributed reviews used a different model than the one that wrote the code, most commonly Step 3.7 Flash writing and Laguna M.1 reviewing, and frames authoring and reviewing as different jobs teams already split. That is the third published production role-split from two vendors, after Kilo's plan/implement split (06-16) and Cursor's planner-worker agent swarm (07-27). Plan, implement, review. This changes how the page should read its own negative results. When is routing meaningful (07-20) found many reported routing gains vanish under honest accounting of the router's own cost and of what a well-chosen single model would have achieved, and IBM's system-cost work (07-15) found routing is mispriced at the system level. Both measure selection by difficulty, which is nearly the entire academic literature, and neither obviously binds role decomposition, because a role split is not choosing the best single model for a query, it is assigning stages of one task to models with different cost-quality profiles. That may be the cleanest available explanation for the gap the 07-30 entry below names between a literature that struggles to justify routing gains and a market pricing OpenRouter at roughly 70x revenue.

Kilo's second finding is a measurement-validity result and belongs with this week's cluster of them. Models agree far more on what is wrong than on how bad it is, so the severity axis every escalation workflow depends on is the least consistent output the models produce. And the widely repeated claim that open-weight reviewers are less secure turns out to be carried by one outlier model: remove it and the gap mostly disappears, with open-weight reviewers matching closed leaders on critical findings, which makes the correct decision model-specific rather than tier-specific. Same shape as Beyond Geometric Complementarity (07-31) below and as Eviction as Estimation (08-03): the instrument everyone trusts is not measuring the quantity it is trusted for. Kilo's own methodological fix is worth borrowing, which is to normalize findings per review before comparing so a reviewer that flags everything does not win on volume.

A second new routing axis today, this one in distribution space. ROPD (08-04) (2607.27081, Kurate cs.AI #18) realigns a maliciously fine-tuned model by modelling the divergence between the aligned and compromised output distributions rather than fitting the attacker's prompt template, which is the template-invariant object and is what buys robustness when the defender cannot observe the attacker's template. The title oversells the routing framing, since nothing dispatches queries between models, so the honest placement is next to multi-head latent control (07-27) rather than in the model-selection family. Meanwhile the cost-side market kept validating what the literature cannot reproduce: Alibaba shipped Qwen3.8-Max priced below Kimi K3 with open weights due in a week, DeepSeek V4 Flash cut coding prices 99%, and Kilo itself was acquired by Anaconda.

Current State (as of 2026-07-31)

The intuitive explanation for why expert routing works has been measured and it does not hold, which invalidates the standard method for deciding which experts to prune. Beyond Geometric Complementarity (2607.28308) separates three quantities that MoE routing evidence routinely conflates: route coherence (does the router pick sensibly), candidate quality (is the picked expert good alone), and candidate-by-context interaction (does its advantage depend on the prefix). Using an Expert Subspace Separation Index, matched-route residuals, and a prefix-controlled 2x2 factorial, it reports three paired contrasts across six MoE architectures.

Expert subspaces overlap substantially, contradicting the geometric-complementarity story, yet actual routes still explain token representations better than matched alternatives. Across 39 factorial cells in OLMoE, Mixtral and DeepSeek, the selected candidate beats the strongest unselected rival in every cell, while the real prefix narrows that advantage everywhere: all interactions negative, every 95% confidence interval below zero. And the geometric narrowing does not imply functional redundancy, because adding later-ranked experts still improves next-token prediction in 24 of 39 frozen-route comparisons, with Top-2 beating Top-1 in all three seeds of a controlled training study. The authors name the joint pattern coherent overlap.

The operational consequence is a pruning result, and it is negative. Expert pruning and expert merging both usually select what to drop by representational similarity. This paper says similarity cannot determine redundancy or pruning value in exactly the regime where it is used. The correct instrument is a frozen-route intervention, which is cheap and directly functional. Any MoE compression ratio selected by subspace similarity should be re-derived.

This is the third layer at which this page's July results have found the same thing. When is routing meaningful (07-20) found many reported model-selection routing gains vanish under honest cost accounting. IBM's system-cost work (07-15) found routing is priced wrong at the system level because the router's own latency and error rate get excluded. Now expert-level routing turns out to work for a reason nobody had verified. In all three cases the mechanism is fine and the published justification for it does not survive controlled measurement, which is a measurement-validity thread rather than a routing thread, and it now runs alongside the wiki's KV-eviction-ablation and agent-benchmark-protocol threads.

On the industry side, routing between open and closed weights got its first published production dataset. Kilo, the application and routing layer that does not train or host models, published deployment data showing open-weight models now carry 79% of its coding workload, with the stated position that closed frontier models still win the hardest problems while open weights win the other 80% on cost, privacy and control, and that "the future isn't open vs closed, it's routing between both" (blog, summary). Their companion test had Kimi K3 for planning plus Grok 4.5 for implementation build the same embedded database as Claude Opus 5 for $1.27 against $31.71, scoring 93 versus 98 with identical crash-test results, and the five-point gap came entirely from tests, docs and code hygiene rather than correctness (blog).

That 25x price gap at a five-point quality cost is the strongest concrete number this page has for the procurement-side routing gain the academic comparisons do not model, and it sits directly against the 07-30 entry below: a literature that struggles to justify routing gains, and a market pricing the aggregation layer at 70x revenue. The reconciliation may simply be that the literature measures accuracy-per-query and the market is buying dollars-per-task, and those diverge by an order of magnitude when the open-weight tier is good enough for four fifths of the work.

Current State (as of 2026-07-30)

The market has put a price on the routing layer, and it does not match what the routing literature says routing is worth. Stripe is reportedly close to acquiring OpenRouter for around $10 billion, roughly 70x the three-year-old startup's ~$140 million annualized revenue (The Information). OpenRouter is a pure-play aggregation and routing layer giving developers access to hundreds of models; revenue is about $12 million a month, up nearly threefold since April, on relatively low costs.

That multiple prices routing as durable infrastructure. This page's two most rigorous recent results argue the opposite. When is routing meaningful (07-20) found that many reported routing gains do not survive honest accounting of what the router itself costs and what a well-chosen single model would have achieved. IBM's model-routing system-optimization work (07-15) argued routing is priced wrong at the system level, because the router's own latency and error rate are usually excluded from the comparison. The gap between a 70x revenue multiple and a literature that struggles to justify routing gains is the sharpest research-versus-industry divergence currently on this page, and it has a clean falsifier: if any frontier lab ships first-party cross-vendor routing, the aggregation layer becomes a feature rather than a company.

Two supporting signals point the same way as the market rather than the literature. Kilo's acquisition by Anaconda came with a Gradient Ventures partner's claim that portfolio companies shifting work off proprietary models save 50-80%, which is a routing gain measured in procurement rather than in benchmark accuracy, and it is a category the academic comparisons do not model. And Cursor Start launched in India at ₹649/month (roughly $7) bundling Grok 4.5 and cloud agents, which is only viable if the underlying model mix is being routed hard against cost.

A new adjacent axis: routing across context windows rather than across models. OpenAI reported that two settings tripled its ARC-AGI-3 score by letting the model retain reasoning across context-window boundaries via canonical compaction. That is a scheduling decision about what state survives a boundary, structurally closer to admission control than to model selection, and it extends the routing-over-context-occupancy axis this page opened on 07-29. It also cost nothing in model capability, which is the part that should worry anyone selling routing as a capability product.

And the most aggressive version of routing is deletion. TurboVLA (07-30) reaches 97.7% on LIBERO at 0.2B parameters and 0.9 GB VRAM by removing the language model from the vision-language-action pathway entirely, rather than routing around it. Read alongside DSPy task-model separation (07-25) and its 550x cost gap, the family of moves is consistent: the large general model is increasingly something to be avoided rather than dispatched to, and routing is one point on that spectrum rather than the endpoint.

Current State (as of 2026-07-29)

A new routing axis arrives from a memory paper, and it is the first one where the thing being routed is neither a model nor a computation but context occupancy. InMind (2607.24368) measures what it calls the implicit-association blind spot: retrieval-based memory only surfaces a stored fact when the fact resembles the query, so a stored tree-nut allergy never fires on a macaron request, because the bridge (macarons use almond flour) is world knowledge the retriever cannot see. Its paired controls isolate the failure precisely. The backbone answers 84.0% of indirect queries when the decisive memory is placed in context, and the same memory systems recall those facts on demand at up to 100%, but when the memory must be retrieved, six vector, graph, and agentic systems reach at most 14.4%. Eightfold embedding dimensionality does not close it.

The paper names the open problem as routing, and specifically as deciding which facts must stay visible, decided before the query is known. Every routing decision this page tracks is query-conditioned: TRACER (04-17) picks a model per query, CaRE (05-11) picks a task-axis expert, MISA picks per-head KV, Kilo's plan/implement split (06-16) picks by phase. Routing-what-stays-resident cannot be query-conditioned by construction, because the whole failure is that the query does not tell you what you need. That makes it structurally an admission-control problem against a fixed context budget, much closer to KV cache admission than to model selection, and it is the first entry on this page that a serving engineer rather than a model-selection engineer would own.

It also scores the prediction the 07-28 digest made, that someone would build a "route to memory before routing to a model" tier within 90 days. InMind is not that system. It is the benchmark that would grade one, and it establishes the headroom at roughly 70 points.

Industry note on the routing-as-product line. Kilo Code published a fresh head-to-head of its Auto Model router against manual model selection on a backend build, with the framing that once a good plan exists, the model that builds it moves cost far more than it moves the result (blog.kilo.ai). That is the same claim as the Kilo plan/implement split (06-16) and the Kilo routing audit (06-07), now stated as a cost thesis rather than a quality one, and it lands in the week enterprises are contesting AI coding-tool renewals over exactly that cost.

Current State (as of 2026-07-25)

Routing stops being a layer in front of the model and becomes a feature inside it, a shipped default at Microsoft, and a $10B acquisition target, all in one week. The page has tracked routing mostly as research plus a thin production layer. Three same-week industry moves change its status.

(1) Anthropic ships routing into the API. Claude Opus 5 (07-24 launch) adds fallback routing (classifier-blocked requests are re-routed to a recommended model based on the request rather than failing) and an explicit effort dial defaulting to high with a fast mode at ~2.5x speed for 2x base price. The effort dial is the productized, per-request form of what CLEAR (06-05, ration reasoning tokens across a batch by a global shadow price) argued for at the batch level, and it makes the Kilo Code audit's (06-07) "more reasoning is not monotonically better" finding actionable per call rather than per deployment.

(2) Cost-per-task, not capability, is now the frontier vendors compete on, and the pool got genuinely diverse. Cursor's CursorBench 3.2 has Opus 5 High at 66.7 for $3.91/task, Fable 5 High at 66.5 for $8.77, and Grok 4.5 High also at 66.7 for $1.51. Three models tied within 0.2 points across a 5.8x cost spread. This is the exact precondition When is routing meaningful (07-20) said routing gains require (a pool genuinely diverse in cost at comparable capability), and it now demonstrably holds at the frontier, not just between a frontier model and a small one. It simultaneously squeezes Sakana's Fugu Ultra v1.1 claim (that a router over commodity models beats Fable 5 without calling it) from both directions: a frontier model now matches Fable 5 at half price from above, and a flash model ties it at a sixth from below.

(3) Microsoft made cost-aware routing the production default, and the aggregation layer got priced. Satya Nadella's MAI post says Microsoft routes production traffic in GitHub Copilot, Excel, and Outlook to its own smaller MAI models whenever they match or beat frontier alternatives on the specific task, keeping frontier models for work that needs them. The Decoder's read is that this is an Azure economics play (the more open and in-house models running on Azure, the less Microsoft pays OpenAI and Anthropic), and notes MAI performs significantly worse in independent benchmarks, which makes the per-task qualification the load-bearing part. Kilo reports its own Auto Efficient mode at 71% of frontier completion rate for 72% lower cost with no custom models. Meanwhile Stripe is reportedly in talks to buy OpenRouter for close to $10B against a $1.3B last valuation, roughly 8x for a routing-and-billing layer that owns no models. The page's long-running claim that routing is cost control rather than a research nicety now has a dollar figure attached to the aggregation layer itself.

(4) The engineering precondition finally gets named. Every result above quietly assumes you can swap the model without breaking the system. DSPy's "separating the task from the model" argument (AI Engineer talk 07-23, ingested 07-25) supplies the thing that makes swappability real and treats it as the architectural decision. Its claim is that a complete task specification needs three languages (natural-language instructions for what should happen, code for what must happen, examples for what good looks like), and once all three exist the model becomes a swappable implementation detail that automated search can optimize. Reported evidence: Shopify cut an AI workload's cost 550x by holding the contract fixed and letting search find the cheapest model that still passed. Read together with Dan Farrelly's same-conference argument that context (models, prompts, tools) has a half-life of weeks while the execution layer lasts years, there is now a three-way convergence, DSPy plus Farrelly plus Microsoft MAI, on one structural claim: isolate the model behind a stable boundary, because the model is the fastest-decaying part of the system.

The open lever is unchanged and now more valuable. Microsoft, OpenRouter, Kilo, and Anthropic are all routing on a quality-floor-or-safety rule. None of them routes on coverage (predict which model catches which failure class), which the 06-07 audit identified as the unexploited gain implied by disjoint coverage. If a quality-floor router is worth $10B, a coverage-aware one is the obvious attack, and DSPy-style search over a fixed contract is the machinery that could find one.

Prior State (as of 2026-07-24)

The routing-as-substitute thesis gets its strongest public claim, from a product. Sakana's Fugu Ultra v1.1 router claims to beat Anthropic's Fable 5 on Sakana's own eval without Fable 5 in the routing pool, plus up to 7.9 points over v1.0, and adds a Claude-Code-compatible endpoint. This is the commercialized descendant of Conductor (Sakana's RL orchestrator over frontier models), and the boldest form of the "a router over commodity models substitutes for one expensive model" idea the page tracks. It must be read against When is routing meaningful (07-20), which argued routing only pays off when the pool is genuinely diverse in cost and capability, exactly the condition a "beat the model we don't even call" claim needs to justify. No independent verification exists yet and the service is EU-unavailable, so this is a claim to falsify, not a result. The Claude-Code endpoint is the concrete part: Fugu is positioning as a drop-in agentic-coding backend competing on economics, the same battlefield as the flash-model price war (Ling 3.0 Flash, Grok 4.5). → summary

Current State (as of 2026-06-18)

The learned orchestrator now confronts its own training cost (SciOrch, 2026-06-18). SciOrch (arxiv 2606.15872) trains a lightweight 8B model to orchestrate frontier commercial LLMs for hard scientific reasoning: it decomposes each question, delegates sub-problems to selected models via API calls, and synthesizes a final answer. It is the fourth datapoint in the page's "routing IS the policy" line after Conductor (05-11, 7B RL orchestrator over frontier workers, beats every individual model at ~3 calls/question), Maestro (05-23, 4B outcome-only RL orchestrator over a frozen model-skill registry, beats GPT-5 on multimodal), and Orchestra-o1 (06-15, 8B DA-GRPO orchestrator with decision-aligned reward, +10.3% on OmniGAIA). What SciOrch adds is a constraint the prior three sidestepped: when every routing action is a paid frontier API call, online RL rollouts are infeasible. Its answer is to decouple exploration from optimization. MCTS (Monte Carlo Tree Search, which explores decision branches by sampling rollouts and backing up their values) mines diverse orchestration trajectories once, the per-node single-turn samples become a reusable offline dataset, and GRPO-style training optimizes the orchestrator on that dataset without further live calls. The result lands on the same side as Conductor and Maestro: the 8B router reaches 56.66% average accuracy on a 240-question SGI-Reasoning + SFE test set, beating the strongest single commercial model by 3.74 points and the strongest multi-agent baseline by 3.33 points, at under half the API cost of typical multi-agent methods. The open economic question shifts from inference cost to training cost: the MCTS mining phase still issues many paid calls, so the amortization point (how many served queries before offline-trained SciOrch beats running a multi-agent baseline online) is the production number the paper leaves unstated. → summary

Current State (as of 2026-06-16)

Capability is phase-localized: three same-day datapoints say decompose the loop and right-size the model per piece. The page's trajectory-routing axis (pick the model per step from trajectory signals) gets its sharpest production instance and two architectural confirmations on one day. Kilo's plan/implement split ran both Claude Fable 5 and GPT-5.5 on the same task: the strong model wrote the clearly better plan (rubric 9.1 vs 8.3), but when both implemented the winning plan, both passed 15/15 acceptance with identical rollout — at $6.30 (GPT-5.5) vs $16.66 (Fable 5). Plan-strong/implement-cheap produced the same service for 59% less. This adds a third finding to the Kilo Code audit (06-07, "more reasoning is not monotonically better" + "coverage is disjoint"): the expensive model earns its cost in the hard-to-verify planning phase; the cheap model suffices for the verifiable implementation phase. FastContext (06-16, Microsoft) is the subtask version — a trained 4B–30B exploration subagent finds code for a fixed solver, cutting tokens up to 60% and lifting resolution up to 5.5% — the coding-agent cousin of S2L-PO's small-model explorer. VibeThinker-3B (06-16) supplies the architecture-level reason: its Parametric Compression-Coverage Hypothesis holds that verifiable reasoning compresses into a small core while knowledge needs broad coverage — so the verifiable, plan-constrained phases (implementation, exploration) are exactly where a small/cheap model is correct, not merely tolerable. The standing tension with the learned-orchestration line (Conductor, Orchestra-o1) sharpens: a hand-specified two-phase split already captures most of the cost win, setting a bar any RL-learned router must beat.

Current State (as of 2026-06-15)

Orchestration is routing, and now it is a learned, modality-aware policy (Orchestra-o1, 2026-06-15). Orchestra-o1 (arxiv 2606.13707) extends the page's "routing IS the policy" thread (Conductor 05-11, RL orchestrator over frontier models; Maestro 05-23, RL-orchestrated model/skill ensemble) into the omnimodal regime where text, image, audio, and video coexist in one task. It does modality-aware task decomposition, online sub-agent specialization, and parallel execution, beating the second-best system by 10.3% on OmniGAIA. The technically interesting piece is DA-GRPO (decision-aligned GRPO), which rewards the orchestrator's routing/decomposition decisions rather than only final-answer correctness, training Orchestra-o1-8B to SOTA among open omnimodal agents. That decision-aligned reward is the orchestration analogue of the RL orchestrators above and joins today's broader GRPO-variant cluster (S2L-PO, AdaSR's HRPO) where three same-day papers each rebuild GRPO's advantage assignment for a different structure. The open unification the page has tracked widens again: a router that jointly emits (which model, which experts, how many tokens, which operator, which sub-agent), all priced by one signal, with the orchestration layer now a learnable policy rather than a hand-built prompt graph. → summary

Current State (as of 2026-06-09)

Input-conditioned compute allocation arrives at two new granularities the same day — and one of them ships to phones. The page has tracked routing across models (Conductor), tasks (CaRE), heads (MISA), recovery strategies (RePoT), and token budgets (CLEAR). Today adds the intra-network operator and per-prompt expert-set granularities. Chiaroscuro Attention (CHIAR-Former, arxiv 2606.08327) routes each token to one of three mixing operators (cheap DCT, RBF, or full self-attention) by its per-token spectral entropy, a complexity signal. Routing collapses to DCT+attention only (the router rejects RBF), and the two-operator variant hits 45% lower perplexity on WikiText-103 at 62.5% fewer attention FLOPs — spending the expensive operator only on hard tokens, the same instinct as sharpness-routed LIVEditor/ISA (05-07). Meanwhile Apple AFM 3 ships per-prompt adaptive compute into hundreds of millions of devices: AFM 3 Core Advanced is a 20B MoE that decides once, up front which experts (and how many active params, 1–4B) to use for the full model depth. This is the consumer-scale product proof of the routing-as-compute-allocation thesis CLEAR (06-05) and the Kilo audit (06-07) argued at the batch and model layers. The open unification widens: a router that jointly emits (which model, which experts, how many tokens, which operator), all priced by one signal. CHIAR-Former's spectral entropy is a candidate signal that could also drive KV decisions (cf. today's FlashMemory LSA).

Current State (as of 2026-06-07)

The production face of the top routing layer (which model), priced in dollars and bug coverage. The Kilo Code audit (2026-06-07) is the field datapoint behind all the research above on the model-selection layer. Same task (a webhook service with 17 planted bugs as answer key), Claude Opus 4.8 at four reasoning levels vs MiniMax M3: M3 found 13/17 for ~$0.07, the cheapest Claude run found the same 13 for $1.30, Claude xhigh/max led at 15/17 but cost 10x+ more. Two findings matter for the concept. First, more reasoning is not monotonically better (Claude medium/high caught an async-transaction bug that xhigh/max missed; max cost 67% more than xhigh for nothing better) — the production echo of the recurring "more compute is not monotonically better" result the page has tracked since the 05-29 hybrid study and CLEAR (06-05). Second, coverage is disjoint: the cheap and expensive runs that tied on count did not catch the same bugs, which argues the optimum is an orchestrated ensemble keyed to bug class (cf. Conductor, MAESTRO), not cheapest-capable selection alone. The urgency is economic: GitHub Copilot moved to usage-based billing on 2026-06-01 and Uber reportedly burned its 2026 AI-coding budget by April, so routing-as-cost-control is now a line item, not a research nicety. Open lever: coverage-aware routing (predict which model catches which issue class) rather than cost-aware routing alone. → summary

Current State (as of 2026-06-05)

A third scarce resource gets routed: the reasoning-token budget across a batch (CLEAR, 2026-06-05). Where the routing thread above is about which model or which recovery strategy, CLEAR (Constrained Latent-utility Equilibrium Allocation for Reasoning, arxiv 2606.03092) routes how many tokens each query in a batch gets to think. It models per-query compute-utility as an S-shaped curve and casts batch-level budget allocation as constrained optimization governed by a global shadow price (the marginal accuracy per token, equalized across funded queries). Its distinctive move is rational abandonment: stop spending on insolvent queries and reallocate to solvable ones near their accuracy-emergence threshold, for up to 3x global accuracy over uniform allocation under scarcity. This is the batch-level, compute-amount counterpart to the compute-location routing the wiki tracked all week (Perplexity local/cloud orchestrator 06-04, NVIDIA OpenShell 06-03) and the per-query over-thinking cut of ThoughtFold (06-04). It also reinforces the recurring "more compute is not monotonically better" finding from a fresh angle: the optimum abandons some queries entirely. The open unification is a router that emits a (model, token-budget) pair priced by one shared shadow price.

Current State (as of 2026-05-31)

Recovery strategy becomes a routed decision (2026-05-31): Adaptive RePoT. RePoT (Recoverable Program-of-Thought) recovers a failed plan from its last verified checkpoint with one extra LLM call, and ships Adaptive RePoT, a rule-based dispatcher that routes between suffix repair (resume from the verified prefix) and fresh retry based on verified-prefix length. The reason it needs a router at all is a capability-scaling pattern: checkpoint repair helps strong models (decisive win on Gemini), ties on GPT-medium/Claude, and hurts the weakest (loses on GPT-mini). This is the same trajectory-level routing as Step-level Optimization for Computer-Use Agents (05-02, which escalated to a frontier model on a Stuck signal), but the routed decision is a recovery strategy and the downside is real, picking the wrong route lowers accuracy. That makes it a sharper test of trajectory routing than the upside-only escalation cascades the wiki has collected so far, and it adds a concrete instance to the recurring "more compute is not monotonically better" finding (cf. the 05-29 hybrid cloud/device study, which reached the same verdict on the accuracy/cost/energy frontier). → summary

Current State (as of 2026-05-27)

MoE design points at both extremes (2026-05-27): MobileMoE and MiniMax-M2. Two papers fix the MoE active/total ratio to a deployment constraint rather than a fixed architecture choice. MobileMoE derives an on-device MoE scaling law under joint mobile memory and compute limits and finds a sweet spot (moderate sparsity, fine-grained + shared experts) that beats on-device dense LLMs at 2-4x fewer FLOPs and runs 1.8-3.8x faster prefill on real phones. MiniMax-M2 goes the other way: a 229.9B-total / 9.8B-active open MoE (~23:1) tuned end-to-end for agentic deployment, where the mini-activation footprint is the economically correct shape for long token-heavy agent trajectories. Both pair with MoE-muP (re-confirmed today on Kurate cs.LG #14, ai_rating 9.0): MoE-muP makes scale-up principled, MobileMoE makes scale-down principled, MiniMax-M2 pushes the active ratio to a deployment extreme. The MoE routing surface now spans pre-training scaling (MoE-muP), per-token activation (BEAM), post-training dynamic conversion (ZeDA), and deployment-constrained sizing (MobileMoE / MiniMax-M2).

Current State (as of 2026-05-19)

Latest addition (2026-05-19): ZEDA post-trained static-to-dynamic MoE conversion. Existing dynamic-MoE methods require pre-training from scratch or task-specific adaptation, which makes them inapplicable to frontier post-trained static MoEs. ZEDA (Zero-Expert Self-Distillation Adaptation) converts a post-trained static MoE into a dynamic one. It injects parameter-free zero-output experts into each MoE layer (the explicit option to skip computation) and adapts via two-stage self-distillation, using the original frozen MoE as teacher with a group-level balancing loss. On Qwen3-30B-A3B and GLM-4.7-Flash across 11 benchmarks, ZEDA eliminates >50% of expert FLOPs at marginal accuracy loss, beats the strongest dynamic-MoE baseline by 6.1 and 4.0 points, and gives ~1.20x end-to-end inference speedup. With ZEDA, the MoE routing surface now has four orthogonal directions: MoE-muP for pre-training scaling (2026-05-17), BEAM for per-token activation (2026-05-16), HodgeCover for resident-count compression (2026-05-18), ZEDA for post-training dynamic conversion. → summary

Prior State (as of 2026-05-16)

Addition (2026-05-16): BEAM dynamic expert-activation masking. Fixed Top-K MoE routing wastes computation on easy tokens and underserves hard ones. BEAM (Binary Expert Activation Masking) replaces it with token-adaptive trainable binary masks: a small gating head per layer learns which experts to activate, with a straight-through estimator carrying gradients through the binarization and an auxiliary regularization on activation count enforcing a budget. Ships with a custom vLLM kernel that exploits the binary structure to avoid the gather-scatter overhead that has historically killed dynamic-K MoE inference. >98% performance retention, up to 85% MoE-layer FLOP reduction, 2.5x faster decoding, 1.4x higher throughput. The routing surface now has seven addressable internal layers (model, adapter, expert-set-per-token, expert-router-per-task, distillation loss, decoding head, latent code) plus the orthogonal profile-design axis. → summary

Prior State (as of 2026-05-15)

Latest additions (2026-05-15): the design space opens on two new axes. Two papers reframe routing from "router mechanism" to "routing target and signal." RouteProfile treats LLM profiling (the structured description of what each candidate model is good at) as an independent design surface with four dimensions: organizational form (flat vs structured), representation type (text vs embeddings vs scalars), aggregation depth (raw vs summary vs deep abstract), learning configuration (frozen vs trainable). Across three representative routers and standard-plus-new-model-generalization settings: structured profiles beat flat ones, query-level signals beat domain-level signals, and generalization to newly introduced models benefits most from structured + trainable profiles. → summary. Dynamic Latent Routing (DLR) moves routing into the LM post-training pipeline itself: joint learning of discrete latent codes, routing policies, and model parameters in a single stage, motivated by the General Dijkstra Search theorem (globally optimal goal-reaching policies recoverable through temporal composition of sub-policies). +6.6 pp mean gain over SFT in low-data fine-tuning; mechanistic analyses show the learned codes have distinct causal roles. → summary. Together with yesterday's MinT (catalog routing) and Orthrus (cache-as-coordination), the routing surface now has six addressable layers: model, adapter (MinT), expert (CARE), distillation loss (Cliff), decoding head (Orthrus), and latent code (DLR), plus an orthogonal profile-design axis (RouteProfile).

Prior state (as of 2026-05-11)

2026-05-11 additions: Routing has now visibly moved inside the model. Three new papers on the same axis: Conductor (Sakana, ICLR 2026) trains a 7B model with RL to orchestrate frontier workers (GPT-5, Claude Sonnet 4, Gemini 2.5 Pro), beating every individual model on GPQA-D / LiveCodeBench / AIME25 at ~3 calls per question; the orchestrator decides topology and per-worker prompt content as a single RL policy, generalizes to unseen agent mixes, forms recursive topologies when allowed to self-call. CaRE (HF) introduces Bi-Level Routing MoE for continual learning: a task-router selection stage above the expert-routing stage, scales to 300+ non-overlapping tasks (where flat MoE routers collapse). MISA (HF) routes the indexer-head axis of sparse attention, treating the 64 indexer heads as an MoE pool. Three papers in 24 hours making the same architectural claim: routing is the policy, not the wrapper. The wiki's three-axis framing (query / provider / trajectory) now needs a fourth: model-internal routing. DAIR.AI's weekly top-papers email (Gmail) independently surfaced Conductor plus HeavySkill (RLVR-trained parallel-deliberation as an inner skill) and described both as "harness wins look like model wins" — same pattern from a third source.

Prior state (as of 2026-05-04)

Routing is an active research and production concern operating on three distinct axes that the May 2026 batch makes explicit:

1. Query-level routing — given a query, pick a model. TRACER (04-17) is the canonical surrogate-routing example.
2. Provider/tier routing inside a harness — pick provider, model, fallback chain, cheap-vs-primary tier per turn. Ken Huang Ch 14 (05-01) is the most concrete public read.
3. Step-level (trajectory) routing — in multi-step agentic systems, pick the model per step based on signals from the trajectory itself. Step-level Optimization for Computer-Use Agents (05-02) is the first concrete mechanism.

Stacked, the three axes form a routing surface: which provider × which tier × which model per step. The wiki has now seen at least one concrete paper / harness analysis on each axis.

Three routing paradigms remain operative:

1. Surrogate routing — cheap classifier handles easy traffic, fall back to LLM for hard cases (TRACER).
2. Capability-based routing — direct queries to different models based on task type or capability match (cheap-model heuristics in Hermes are the production form; learned variants are open).
3. Agent trajectory routing — in multi-step agentic systems, optimize the path through a sequence of model calls, not just individual ones (Step-level Optimization).

Key Papers / Posts

SciOrch (2026-06-18) — A lightweight 8B model trained to orchestrate frontier commercial LLMs for scientific reasoning: decompose the question, delegate sub-problems to selected models via API, synthesize the answer. The distinctive contribution is offline orchestrator training: because each routing action is an expensive paid API call, online RL rollouts are infeasible, so SciOrch mines orchestration trajectories with MCTS, extracts per-node single-turn samples, and optimizes with GRPO-style training on that fixed dataset. Reaches 56.66% average accuracy on a 240-question SGI-Reasoning + SFE set, beating the best single commercial model by 3.74 points and the best multi-agent baseline by 3.33 points at under half the API cost. The first paper in this wiki to treat the training cost of the orchestrator (not just its inference cost) as the binding constraint. → summary

Conductor (Sakana AI, ICLR 2026, surfaced 2026-05-11) — 7B RL-trained orchestrator over frontier LLMs (GPT-5, Claude Sonnet 4, Gemini 2.5 Pro). The model writes natural-language subtasks, assigns each to a worker, and decides which prior outputs that worker sees. Outperforms every individual worker on GPQA-D, LiveCodeBench, AIME25 at ~3 calls per question. Trained against randomized agent pools (generalizes to unseen mixes). Recursive topologies emerge when self-call is allowed. Cross-source confirmed via DAIR.AI weekly email and @burkov retweet. → summary

CaRE: Bi-Level Routing MoE (2026-05-11) — Continual learning with PTM backbones at 300+ tasks. Bi-level routing: a task-router selection stage activates relevant task-specific routers, then an expert-routing stage activates and aggregates expert outputs at every intermediate layer. Introduces OmniBenchmark-1K (the first 100-to-300+ task CIL benchmark). The first task-axis routing primitive in the wiki, structurally similar to MISA's head-axis routing in inference. → summary

TRACER (2026-04-17) — Trains lightweight ML surrogates on an LLM's own production traces (free labeled data). A parity gate activates the surrogate only when it agrees with the teacher above a confidence threshold. Achieves 100% surrogate coverage on a 150-class benchmark using Claude Sonnet 4.6 as teacher. Generates interpretability artifacts for the routing boundary. → summary

Ken Huang Ch 14 — Routing and Provider Abstraction (2026-05-01) — Cross-harness comparison of how Claude Code and Hermes implement routing. Claude Code: compile-time provider abstraction, single fallback model, signature-stripping on retry. Hermes: runtime API-mode auto-detection, ordered fallback chain, live OpenRouter context-window discovery (cached 1 h), switch_model mid-session, conservative choose_cheap_model_route. The most detailed public read of production routing engineering. → summary

Step-level Optimization for Computer-Use Agents (2026-05-02) — Event-driven cascade for GUI agents: small policy by default, escalate to frontier model when learned monitors detect a Stuck pattern (progress stalled) or a Milestone (semantically significant checkpoint). Trajectory-aware routing inside the agent. Modular, no retraining. The first concrete mechanism for the trajectory-level axis. → summary

Ken Huang Ch 15 — Structured Output (2026-05-02) — Cross-harness comparison of schema-constrained generation. Both Claude Code and Hermes converge on tool-use forcing as the portable mechanism. Claude Code: SyntheticOutputTool with Ajv compile, schema-identity caching, retries excluded from agent tool budget, child agents have it stripped. Hermes: extract_structured() with portable tool-choice forcing, plus JSONL trajectory format as infrastructure-level structured output. Schema support is a per-model capability — a sub-axis of routing the Hermes flag exposes but no router yet consumes. → summary

Xiaomi MiMo-V2.5-Pro (2026-05-03) — Open-weight long-horizon coding model claiming 40-60% fewer tokens per task than Claude Opus 4.6. The pricing axis shifts from "capability ceiling" to "tokens-per-task." Mechanism not disclosed; candidates include MoE sparsity, LenVM-style length value heads, CoPD-style distillation, RL chain-of-thought truncation. Tokens-per-task is a routing-relevant signal regardless of mechanism — a router can prefer a model that is 50% as expensive at 95% of the quality. → summary

Key Concepts

• Surrogate model: a cheap ML classifier trained to approximate a more expensive LLM's decisions on a specific task
• Parity gate: a confidence threshold that controls when the surrogate is trusted vs. when to fall back to the LLM
• Coverage: fraction of traffic the surrogate handles vs. falls back to the LLM
• Production traces: labeled input-output logs from a deployed LLM — free training data for a surrogate
• Routing boundary: the region of input space where the surrogate is reliable; interpretability artifacts describe this
• Fallback chain: ordered list of (provider, model) tuples consumed in sequence on failure (Hermes); contrast with single-fallback (Claude Code)
• API-mode auto-detection: inferring the API contract (anthropic_messages / chat_completions / codex_responses) from URL and provider name rather than explicit configuration
• Signature stripping: removing model-specific extended-thinking blocks before retrying with a different model — required for cross-provider fallback to work
• Stuck Monitor / Milestone Monitor: learned signals on agent execution traces; fire when escalation to a stronger model is warranted
• Cheap-model routing: per-turn demotion to a cheap model when the user message is short, single-line, free of code blocks/URLs, free of complexity keywords; conservative by design

Open Problems

• Routing for open-ended tasks (no ground-truth labels to train surrogates)
• Multimodal routing: routing queries across text, image, and video models — Nemotron 3 Nano Omni (05-02) multimodal token reduction is one upstream primitive
• Agent trajectory routing: optimizing multi-step tool-use sequences, not just individual calls — Step-level Optimization is the first concrete attempt; Claw-Eval-Live (05-01) provides the calibration data
• Dynamic routing that adapts as model capabilities and costs change
• Cache-aware routing. Switching models invalidates prompt cache; SemiAnalysis (05-01) showed cache hits drive blended Opus pricing to $0.99/MTok. A router that knows current cache state and routes within-cache aggressively is the obvious efficiency move; nobody has published it.
• Reasoning-mode routing. Compliance vs Sensibility (05-02) shows reasoning mode is a steerable linear direction. A router that picks both model and forced reasoning mode is the deeper control surface.
• MCP server selection as routing. Ken Huang Ch 13 (05-01) made clear that MCP server selection is an explicit routing problem; today the agent picks "whichever first."
• Schema-aware routing. Hermes's ModelCapabilities.structured_output flag (Ch 15, 05-02) is a per-model feature today; routing to a model that lacks native JSON-mode requires falling back to tool-use forcing. The router that consumes this flag does not exist.
• Tokens-per-task pricing as routing signal. Xiaomi MiMo-V2.5-Pro (05-03) reframes the cost axis: a router that knows which model is cheap on this query, rather than which model is cheap in general, is more valuable in long-horizon agents. Concept now has external open-weight competitive pressure.
• Safety-as-routing-constraint. Defense Trilemma (05-02) implies the wrapper around any single model cannot be both utility-preserving and complete. Routing across models with uncorrelated failure modes is a defense-in-depth mechanism the trilemma cannot constrain. Nobody has formalized this as a routing objective.

Cross-axis composition (to track)

• Step-level × Provider/tier: stack the 05-02 step-level cascade with Ch 14 provider routing → two-axis routing surface inside one agent.
• Step-level × Surrogate: train a TRACER-style surrogate on the cheap-tier model in the cascade; 05-02 cascade only needs the small model for routine steps.
• Trajectory routing × Claw-Eval-Live: 05-01 noted that no single frontier model crosses 70% on Claw-Eval-Live; trajectory-aware routing is the cleanest open lever to cross it. Step-level Optimization is the first candidate mechanism.
• Tokens-per-task × Trajectory routing: Xiaomi MiMo-V2.5-Pro (05-03) optimizes the per-step cost; Step-level Optimization (05-02) optimizes per-trajectory routing. Composing them — a cheaper model in routine steps + Stuck/Milestone escalation — is the multiplicative efficiency play that no paper has measured.
• Schema-aware × Provider routing: a router that picks among providers where some support native JSON mode and others require tool-use forcing should bias toward the cheaper mechanism when both succeed. Hermes's flag is the substrate; no consumer exists.

Related Pages

• Inference Efficiency
• KV Cache

Recent additions

• 2026-05-08 — Netflix Tech Blog: State of Routing in Model Serving (Nipun Kumar, Rajat Shah, Peter Chng). Title-level signal only; surfaced via Gmail Medium digest. Production-engineering routing taxonomy from Netflix. Worth a manual read. Stubbed for now, full summary pending.
• 2026-06-04 — Perplexity hybrid local/cloud orchestrator. Productized local-vs-cloud query router shipped to consumers, the third in a week after NVIDIA OpenShell (06-03) and the Cloud-Device Hybrid Agents research framing (05-29). The compute-location axis (local vs cloud) is now the dominant productized routing axis, while research still lacks a benchmark scoring router decisions on cost × latency × decision quality together. The runtime market (NVIDIA, Microsoft, Nous, Perplexity) is being decided by product polish, not measured routing quality.
• 2026-06-04 — Scaling-parametrization for sparse/efficient architectures continues: Gated Delta Networks μP extends the closed-form hyperparameter-transfer programme from MoE (MoE-μP, 05-17) to gated linear attention, and Marin's open MoE recipe reports the empirical dense→MoE speedup ledger (6.7x theoretical / 3.6x realized). "Derive the scaling rule once, never re-sweep at target scale" now spans two architecture families in three weeks.
• 2026-06-08 — Disentangling Agent Self-Evolution: a routing insight wearing a self-evolution costume. It splits self-evolution into harness-updating (an evolver model writes scaffold edits) and harness-benefit (a solver model exploits them), and finds updating is flat across model tiers (a Qwen3.5-9B edits as well as Claude Opus 4.6) while benefit is non-monotonic (mid-tier solvers gain most). The cost-optimal recipe is to route the cheap model to the editor role and reserve frontier capacity for solving, the same "no model is Pareto-dominant, assign by role" logic as the Kilo audit but applied inside a single agent's improvement loop.
• 2026-06-07 — Kilo Code audit: MiniMax M3 vs Claude Opus 4.8. The production datapoint under the whole routing thesis: same code-audit task, MiniMax M3 found 13/17 bugs for ~$0.07 while Claude Opus 4.8 cost 10x+ for the same-to-slightly-better coverage — and the cheap and expensive runs that tied did not catch the same bugs. Empirically grounds the topmost routing axis (which model) in dollars and bug coverage, and argues for coverage-aware routing or ensembling (cf. Conductor, MAESTRO) since no model is Pareto-dominant. Demand-side mirror of CLEAR: with GitHub Copilot on usage billing (06-01) and Uber's 2026 AI-coding budget gone by April, routing-as-cost-control became a line item.

The latent axis opens, and industry ships the phase split (2026-07-27)

Two things landed the same day that belong together.

Multi-Head Latent Control (07-27) is the first router in this wiki that makes an inter-model decision from intra-model evidence. Two lightweight heads read the hidden-state trajectory of a frozen LLM or VLM as it generates. A Capability Head decides whether this model can finish the instance or should defer to a stronger one; a Resolution Head picks among clarify, tool use, abstain, or direct answer. Up to 90.7% fewer large-model calls on AndroidWorld, 27 to 53% fewer on average, plus up to +158% relative gain on tool-use decision quality and 65.5% fewer missed required tool calls. Because the signal is the partial generation rather than the prompt, handoff can fire mid-answer, which no input-side router can do. This adds a fifth routing axis to the page: after model-tier, task/expert, head, and trajectory, there is now latent-trajectory routing.

Note the tension with When Is Routing Meaningful? (07-20), which found learned KNN routers collapse under paraphrase while prompted routing stays stable. MHLC is a learned router and inherits the suspicion, but it conditions on a generation rather than a query, and generation-trajectory paraphrase-invariance is plausibly higher than query-embedding invariance. Untested, and it is the decisive experiment.

Cursor's agent swarm (07-27) is the same split shipped in a product. Cursor separated a planner from workers and had the swarm rebuild SQLite in Rust from documentation alone, no source code and no internet. Every configuration of the new system reached 100% on the test suite; the old swarm, without the split, choked on merge conflicts of its own making. That failure mode is the informative part: the binding constraint in multi-agent coding is write-conflict management, not per-agent code quality, so the planner's real job is partitioning rather than reasoning depth. Nobody has tested whether a cheap partitioner works as well as an expensive one.

Role-based model assignment is now settled, at five independent datapoints in six weeks. Kilo (06-16) plan-strong/implement-cheap at 59% cheaper with identical 15/15 acceptance results; Disentangling Agent Self-Evolution (06-08) harness-updating flat across tiers so cheap models edit scaffolding as well as frontier ones; Conductor (05-11) a 7B orchestrator beating every frontier worker it directs; DSPy/Shopify (07-25) 550x from fixing the contract and searching for the cheapest passing model; and now Cursor plus MHLC. The claim: no frontier model is Pareto-dominant across the roles inside a single task, and the value is in assigning roles, not picking a winner.

The open axis these five sharpen: all of them decide by phase or by instance. None decides by cache state, which the Open Problems list above has flagged as unpublished since May and which MHLC makes more urgent, since routing more often means invalidating prefix cache more often, and discarding a partial generation to hand off means re-prefilling on the larger model.

A sixth axis: adapter-level routing over a frozen base (2026-08-11)

Macaron-V1 (08-11) adds a routing granularity this page has not carried. The five axes above are model-tier, task/expert, attention-head, trajectory, and latent-trajectory. Macaron's Mixture-of-LoRA (MoL) freezes a very large base model, composes specialist LoRA adapters on top of it, and selects exactly one adapter per user turn. The flagship pairs a 744B GLM-5.2 base with four adapters (chat, agent, coding, GenUI); a 50B Qwen3.6 variant runs the same design locally.

Why this axis is different, and it is a cost argument. Every other axis on this page pays a switch penalty. Routing across separate models means separate resident weights and, as the Open Problems list has flagged since May, prefix-cache invalidation whenever the route changes mid-conversation. Multi-Head Latent Control (07-27) sharpened this: because it can hand off mid-generation, it must re-prefill on the larger model, so its 90.7% reduction in large-model calls on AndroidWorld is bought partly with re-prefill cost that the paper does not price. MoL is the first proposal where the base weights and, in principle, the base KV cache stay put and only a small patch swaps. If that holds, it is the cheapest route-switch mechanism anyone has published, and it is the first candidate answer to the routing-by-cache-state question this page has listed as unpublished since May.

The claim is not yet evidenced. No adapter-switch latency and no cache-behaviour measurement appear in the report, which is exactly the figure that would convert this from a plausible cost story into a demonstrated one. Nor is the selection mechanism specified: whether the per-turn choice is learned or prompted, what it costs, and how often it is wrong are all unstated, in a paper whose headline is selection.

Where it sits against the page's existing tensions. At four coarse, semantically distinct roles, MoL is near the easy end of the spectrum When Is Routing Meaningful? (07-20) mapped when it found learned KNN routers collapse under paraphrase while prompted routing stays stable, so paraphrase robustness is plausible here and would not transfer to a finer adapter set. The interesting threshold is adapter count: at four this is specialist selection, and somewhere past a few dozen it becomes a genuine router with a real decision problem. MinT-scale LoRA serving (05-14) already established that serving very large adapter counts against one base is tractable, so the infrastructure exists ahead of the routing policy, which is the same ordering this page has observed repeatedly.

Role-based model assignment reaches six independent datapoints, after Kilo (06-16), Disentangling Agent Self-Evolution (06-08), Conductor (05-11), DSPy/Shopify (07-25), and Cursor plus MHLC (07-27). Macaron is the first to ship the assignment inside one served model rather than across an API boundary, which is what makes it a deployment story and not only a research one.

And the routable unit keeps growing. Macaron versions the model-and-harness together as its unit of improvement. A²E (08-11) reached the same conclusion from measurement, finding no model-harness combination wins across all task types, and Evo-Bench (08-11) found synthesized harnesses transfer across policy models. Three papers on one day treating model and scaffold as a single routable object, and still nothing in production routes over harnesses.

Training-time routing: decaying router noise (2026-08-11)

A mechanism worth recording separately because it concerns how routers are trained, not how they decide. Motif 3 (08-11) perturbs its FP32 router logits with Gaussian noise on a cosine decay schedule: large early, annealed to a floor once expert roles form. The stated rationale is that early routing decisions become self-reinforcing before experts have developed meaningful specialization, so a router can lock in a bad assignment permanently. Their Figure 6b shows maximum tokens per expert falling to the median-load regime by roughly 17B training tokens with noise, against roughly 25B without, and the report claims faster early loss decrease.

Two reasons this matters here. It is a direct treatment for premature routing lock-in, which is the training-time analogue of the load-imbalance problem that inference-time expert-budgeting work like BEAM addresses from the other end. And Hugging Face's Elie Bakouch noted he had not seen router noise defined in a large-model report before, possibly since early Google MoE work, which suggests a technique that was known, dropped, and is now being rediscovered at scale. Motif 3 pairs it with Expert-Specific PolyNorm, which holds a higher effective rank in the expert gate weights than SwiGLU at every layer, meaning a flatter singular-value spectrum and less collapsed specialization. Both are load-balancing mechanisms acting on the router rather than on the loss, which is a different lever from the auxiliary-loss balancing that dominates MoE practice.

The field gets a common formalism, and a cost metric it was measuring wrong (2026-08-14)

Two things landed today that address the two structural weaknesses this page has carried since April: no shared vocabulary, and a cost axis nobody had validated.

LLMRouter (08-14) proposes a single formulation of LLM routing as a sequential decision process with five components: context encoders, model encoders, scoring functions, decision rules, and learning signals. Single-turn, multi-turn, and personalized routing all become instances of it. It ships xRouteBench (generic LLM, memory-augmented, vision, time-series, and personalized routing tasks) plus an open-source library with 16+ representative routers behind one interface, and evaluates jointly on response quality and inference cost rather than quality alone. Three results: learned routers beat the strongest fixed-model baseline by 14.6% relative; lightweight routers get more competitive as the cost constraint tightens, because a heavy router's own cost eats the savings it produces; and user-conditioned routing consistently improves personalization, so identity is a real routing feature.

The immediate value to this page is commensurability. The six axes catalogued above (model-tier, task/expert, attention-head, trajectory, latent-trajectory, adapter) are, under LLMRouter's decomposition, different scoring functions over different context encoders. That is the first frame the wiki has had that makes them comparable rather than merely adjacent. What the formulation does not cover is routing inside one model, where DLR (05-15) and MISA live, and it has no time-of-day term, which DeepSeek's new peak/off-peak pricing (08-14) just made a live variable.

The cost metric problem is worse than this page assumed. The AlphaSense study (08-14) tested models on searching and analyzing a large financial-document corpus and found GPT-5.6 Sol and Opus 4.8 produced better answers at lower total cost than Moonshot's Kimi K3 and Z.ai's GLM-5.2, despite charging $25 and $30 per million output tokens against Kimi's $15. The mechanism is that a stronger model finishes the task in fewer tokens and fewer retries, so unit price and task cost can point in opposite directions. Artificial Analysis, ranking the same models by cost-to-accomplish, puts the Chinese models significantly cheaper, so the two measurements genuinely disagree and the disagreement is about task composition rather than arithmetic.

This is a direct problem for every routing result on this page that optimizes a token-price objective, which is most of them, and for xRouteBench specifically: a benchmark that scores cost per token will rank routers differently from one that scores cost per completed query. It also sharpens the Kilo Code audit (06-07) finding that cheap and expensive runs which tie on score do not catch the same bugs. If task cost is the metric, then routing to the cheap model is only correct when the cheap model actually finishes.

Industry moved on price in the same window. Gemini 3.7 Flash shipped three weeks after 3.6 Flash claiming Claude Sonnet 5 and GPT-5.6 Terra parity at half the price, and Cognition put it in Devin reporting Sonnet-5-level FrontierCode 1.1 performance at less than half the cost with a 50% launch discount through 2026-08-27, noting it is particularly strong on tightly scoped refactors that produce minimal diffs matching repo conventions. That last detail is a routing signal, not a marketing one: it names a task class where the cheap model is not merely acceptable but preferred. Meanwhile Fable 5's slow adoption (six percent of Anthropic tokens sold, per Ramp data) is the demand-side evidence that the frontier-price tier has a ceiling, which is the market condition under which routing stops being an optimization and becomes the default architecture.

Standing gap, unchanged and now three months old: nothing in production routes over harnesses, and no paper proposes it, even though DarwinX and AutoDesign (08-14) both showed a harness transferring across base models and agent frameworks. The routable unit is the model-harness pair and the routers are still routing over models.

Value of information becomes the formalism, and routing turns out to happen at three levels of the stack (2026-08-25)

Three results landed on one day that, read together, change how this page should be organized.

Pandora's Router (08-25) prices the one thing every result above assumed was free. Google DeepMind (Fisch, Trivedi, Huot, Cohen, Kaisers, Lapata, Larson, Eisenstein) observe that all prior routing work, including everything catalogued on this page, estimates each model's expected return on an input and picks the maximizer while treating value estimation as costless with a fixed error profile. It is not. Cheap estimators (embedding predictors) are fast and noisy; accurate ones (fine-tuned scorers with retrieval results or partial reasoning traces) cost real compute. They map this onto the classical Pandora's Box problem (Weitzman 1979, optimal search when inspection costs money) crossed with information value theory (Howard 1966), and derive a closed-form value-of-information policy under a Gaussian signal model: for each specialist and input, an expression says whether refining the estimate beats the cost of refining it. A decentralized variant, Pandora's Bidder, has each model decide independently whether to pay for self-assessment before accepting a price, which is the multi-vendor serving case where no central authority can measure everything.

This is the theory for an empirical result this page recorded five days earlier. LLMRouter (08-14) found that lightweight routers become more competitive as the cost constraint tightens, because a heavy router's own cost eats the savings it produces. That is Pandora's thesis observed experimentally, by a different group, without the formalism. Worth naming the ordering: the measurement came first and the closed form arrived after. Together they also make a meta-point this page should carry explicitly. The AlphaSense study (08-14) showed the field mismeasures the cost of the answer (a stronger model finishes in fewer tokens and retries, so unit price and task cost can point opposite ways). Pandora shows the field ignores the cost of the decision. Two independent corrections in eleven days, both saying the routing literature has been working from an incomplete ledger.

VoI routing for Mixtures of LoRA Experts (08-25) derives the same formalism independently, one level down. Surfaced here because Daniel Whitmore crossed the Kurate rising-author threshold this week; it is LLM-rated (cs.LG #4-#5 across two weeks) and never appeared on HuggingFace. Its target is uncertainty-aware MoLE routers such as CARE, which set the expert count from router concentration and expert disagreement. The named error is confounding uncertainty magnitude with uncertainty reducibility: high uncertainty can mean an unqueried expert holds complementary evidence (recoverable risk, worth paying for) or that the input is irreducibly ambiguous (residual risk, where more experts are pure waste). Existing routers spend identically in both cases. The fix is to route by expected risk reduction against query cost.

The sharpest technical point in that paper is a supervision asymmetry this page should record: because all adapters sit in a shared pool over one frozen backbone, you can observe during training what the unqueried experts would have contributed, giving genuine counterfactual supervision for learning reducibility. Across-model routing has no such signal (you would have to run every model), which means within-model expert acquisition is a fundamentally easier learning problem than the cross-model routing most of this page is about.

It also supplies the mechanism Macaron-V1 (08-11) was missing. This page's recorded critique of Macaron was that its per-turn adapter selection was unspecified: learned or prompted, at what cost, wrong how often, all unstated in a paper whose headline was selection. And it answers the threshold question raised there. Macaron at four coarse semantic adapters is specialist selection; a pool large enough that you must reason about which adapter to query is the genuine router regime. Because only a small patch swaps while base weights and in principle the base KV cache stay put, adapter-level VoI routing remains the best candidate answer to the routing-by-cache-state question this page has listed as unpublished since May.

The reorganizing observation: TileMix (08-25) makes it three levels on one day. TileMix routes numerical precision over hardware-aligned tiles of the attention score matrix, dispatching each tile group through FP16 or INT8 with both paths updating one shared online-softmax state. That is a routing decision below every axis this page has catalogued. So on 2026-08-25 the same decision structure appears at three distinct levels of the stack: across models (Pandora), across adapters inside a model (VoI-MoLE), and across regions of a matrix inside a kernel (TileMix). The six axes this page tracks (model-tier, task/expert, attention-head, trajectory, latent-trajectory, adapter) are all one of those levels. Routing is not a serving-layer pattern. It is the same allocation problem recurring wherever compute can be spent unevenly.

The open problem this creates, and it is a new one. A production serving stack today makes the model choice, the adapter choice, and the precision choice independently, each with its own cost model, none aware of the others. All three are "should I spend more compute to reduce risk on this input." Jointly they are a single constrained allocation problem, and nobody has written that objective down. That is now the highest-value unpublished item on this page.

Standing gap, unchanged and four months old, but Pandora is the first tool for it. Nothing in production routes over harnesses and no paper proposes it, even though A²E and Evo-Bench (08-11), DarwinX and AutoDesign (08-14) all showed harnesses transferring across base models and agent frameworks. Harness selection is exactly the expensive-inspection regime Pandora's formalism targets, since you cannot know whether a harness suits a task without partially running it. A Pandora-style policy whose boxes are model-harness pairs and whose inspection cost is a partial rollout is the concrete unwritten paper.