Media Zone | 2026-11-14
The US Friday conversation was still half about Nvidia without Jensen Huang, but the cost story underneath got sharper. Thinking Machines showed how to train a 594B model with a 4-bit forward pass. Claude Code now calls a bigger model only at hard decision points. A new paper shows a few minutes of burst traffic can wreck a prefix-cached agent server for an hour. And the money side of the GPU buildout is being repriced in real time.
Today's signal
- Dominant story: the Huang fallout moved from scandal to balance sheets. GPU-backed financing, neocloud stocks and Nvidia's next CUDA-scale bet are the live questions.
- Quantization reaches pretraining: Thinking Machines published its end-to-end 4-bit recipe for Inkling 2 Small. NVFP4 forward and MXFP8 backward give 8% faster pretraining and 20% faster RL, with no calibration step.
- Routing goes mainstream: Claude Code's advisor tool calls a stronger model only at a few decision points, and its fallback flag drops to cheaper models when the big one is down.
- KV cache gets a failure mode: "Bursts That Outlast Themselves" shows agent servers can get stuck at a 10% cache hit rate after a short spike. The fix is budgeting active sessions' KV, not throttling.
- Pruning goes below the expert: Tencent finds MoE domain skills live in scattered neurons, so pruning neurons across experts keeps 91-98% of quality at 75% size.
- Local inference matures: llama.cpp v1.0.0 shipped with a versioning promise, alongside a closed Mac quant (KQuant) and a "learn a 0.05% mask, not the weights" result.
- Counter-signal: agent security is not solved. A Google DeepMind attacker swarm beats the best-defended Gemini on 34% of tasks, and under 10 edited neurons can backdoor an agent.
- Pattern: agents tuning the infrastructure that trains models (InfraRSI, No Second Run) is now a recurring theme, not a one-off.
- Quiet area: no new saved posts this window (the feed captures worked, so this is no new saves, not an auth failure). YouTube, LinkedIn and all eight Reddit subs returned nothing. Everything below is the X Following feed for the US Friday, cross-checked against the newsletters and lab feeds.
Routing, KV cache, compression, GPU
Routing inside the harness: spend the big model only where it changes the answer
flowchart LR
U["Task<br/><small>user request</small>"] --> S["Sonnet 5.5<br/><small>runs session, edits code</small>"]
S --> H["Haiku 5.5<br/><small>search, grep, docs</small>"]
H --> S
S --> G{"Checkpoint?<br/><small>plan, repeat error, done</small>"}
G -->|yes| O["Opus 5.5<br/><small>reads transcript, advises</small>"]
O --> S
G -->|no| R["Result<br/><small>shipped change</small>"]
classDef input fill:#d0ebff,stroke:#1971c2,color:#1b1b1b,stroke-width:2px
classDef core fill:#e5dbff,stroke:#6741d9,color:#1b1b1b,stroke-width:2px
classDef loop fill:#fff3bf,stroke:#f08c00,color:#1b1b1b,stroke-width:2px
classDef exit fill:#d3f9d8,stroke:#2f9e44,color:#1b1b1b,stroke-width:2px
class U input
class S,H,O core
class G loop
class R exit
linkStyle 4 stroke:#f08c00,stroke-width:2px
linkStyle 6 stroke:#2f9e44,stroke-width:2px
Claude Code's advisor tool
Claude Code can now pair a session model with an advisor model (`--advisor opus`, `/advisor`, or `advisorModel` in settings). The session model handles every routine turn. It consults the advisor only at high-stakes moments: before a plan locks, when the same error repeats, and before declaring done. That is cost routing at the turn level: you pay frontier rates for a handful of judgment calls, not for every file read. The docs also cover how the advisor interacts with prompt caching, which is where the real savings or losses will show up.
Actor, Judge, or Teacher? A bad actor can still teach well
Decision models output a typed answer (a probability per option) rather than free text. Used as actors on ALFWorld, all six models tested chase an easy subgoal before the one that must come first. Shown the finished trajectory, the same models correctly credit the steps they skipped. Used as a GRPO teacher (scoring each action after the episode), the JEV decision model lifts a Qwen2.5-1.5B policy from 72.8% to 82.9% on ALFWorld and 56.8% to 71.4% on WebShop. Routing angle: a cheap judge in hindsight is worth more than a cheap actor in the moment.
Perplexity Decider v1.3: a router you can download
Perplexity open-sourced Decider v1.3 and a distilled Decider Flash v1.3 (Apache 2.0, free on OpenRouter for a week). These are decision models: they return a probability per label, so apps can classify, route, rank and score inputs instead of parsing free text. v1.3 adds "locate", which returns the matching text span with character offsets for extraction. Perplexity says it is 6.54 points more accurate than the classifiers it runs in production today. It is built on the same JEV decision-model idea as the Actor/Judge/Teacher paper above, so the "cheap typed judge" pattern now has open weights.
Claude Code fallback chain: after a morning outage,
--fallback-model sonnet,haikumoves a turn to a cheaper model when Opus is overloaded, then retries Opus next message. It does not help with auth, billing, rate limits or a full API outage (post).caveman, a viral prompt-compression skill and proxy for coding agents, claims 65% fewer tokens for the same answers by stripping filler. Treat the number as a vendor claim until someone measures answer quality on a real task (repo).
Decision-model copycats: Sebastian Raschka counted 16 vendor launches between 09-23 and 11-11, from Together and Alibaba to OpenAI, Microsoft and Google, and cites Vercel's figure that 26% of requests now route through decision models (@rasbt).
dspy-wsp (Cmpnd): a DSPy optimizer in which GPT-6 Astra edits a decision model's weights directly, with no gradients, for about $5-10. All five open decision models tested gained 1.9-16.1 points on held-out decisions, and 6.5-20.0 with threshold refitting (post, @dbreunig).
KV cache: a few minutes of burst can break an agent server for an hour
flowchart LR
B["Burst<br/><small>few minutes extra load</small>"] --> S["More sessions<br/><small>stay resident longer</small>"]
S --> K["KV pool full<br/><small>working set too big</small>"]
K --> M["Cache misses<br/><small>10-40K token re-prefill</small>"]
M -->|slower turns| S
K --> G["KV budgeter<br/><small>caps active sessions</small>"]
G --> H["Healthy<br/><small>hit ratio recovers</small>"]
classDef input fill:#d0ebff,stroke:#1971c2,color:#1b1b1b,stroke-width:2px
classDef loop fill:#fff3bf,stroke:#f08c00,color:#1b1b1b,stroke-width:2px
classDef err fill:#ffe3e3,stroke:#e03131,color:#1b1b1b,stroke-width:2px
classDef exit fill:#d3f9d8,stroke:#2f9e44,color:#1b1b1b,stroke-width:2px
class B input
class S,K loop
class M err
class G loop
class H exit
linkStyle 3 stroke:#e03131,stroke-width:2px
linkStyle 5 stroke:#2f9e44,stroke-width:2px
Bursts That Outlast Themselves
Agent sessions grow their context every turn, so serving them cheaply depends on prefix caching: a hit costs about a thousand tokens of prefill, a miss recomputes 10-40K. The authors show this makes serving metastable. A few minutes of extra traffic left a vLLM server at about a 10% hit ratio and half the throughput for the remaining 70 minutes, while an identical no-burst twin stayed healthy (9 of 12 pairs). It reproduces on SGLang and with Qwen3-32B on SWE-bench trace shapes. Throttling new arrivals recovered 1 of 7 collapsed runs and a CPU offload tier only delayed collapse. Suspending admitted sessions against a capacity limit recovered all of them, and a budgeter that caps resident KV demand prevented the failure in every run.
- SGLang Consortium: LMSYS launched it with NVIDIA, AMD, Intel, Google, Meta, Cloudflare, Modal and RadixArk as founding members, at the first SGLang Summit. The open serving engine now has a multi-vendor roadmap body, which matters for exactly the cache-admission fixes above (LMSYS, @NVIDIAAI).
- GLM-5.3 across two DGX Stations (NVIDIA): the NVFP4 GLM-5.3 (753B MoE, 40B active) needs about 464 GB, more than one GB300's 252 GB, so NVIDIA serves it with tensor parallelism over two stations linked by ConnectX-8 RDMA in vLLM (post).
Training in 4 bits: how Inkling 2 Small was made
End-to-End 4-Bit Training (Thinking Machines)
Thinking Machines explained how its 594B open MoE, Inkling 2 Small, was trained with a native 4-bit forward pass in the expert layers, from the first pretraining step to the last RL rollout. Forward passes run in NVFP4 because they are high volume and their rounding errors average out. Backward passes run in MXFP8 because a consistent rounding bias would compound. The clever part is the transpose: values get regrouped under new block scales, so they de-quantize, transpose and re-quantize with stochastic rounding, keeping gradients unbiased on average. Once the math got about 3x faster, the NVLink wire became the bottleneck, so expert-parallel communication was quantized too. Net: 8% faster pretraining and 20% faster RL than full MXFP8, about 2% efficiency loss, and a 4-bit checkpoint that ships with no calibration.
- cuBLAS Ozaki-II (NVIDIA): exact FP64 matrix multiplies rebuilt from integer tensor-core math, 5.5x faster than native FP64 on Vera Rubin NVL72. It is the mirror image of Inkling: low-bit hardware, exact answers (post).
Local inference and compression: llama.cpp grows up
llama.cpp v1.0.0
After 3.5 years, llama.cpp moved to semantic versioning with a stated commitment to keep covering every platform and every model. The point is stability for the many downstream apps that already depend on it: quantized, portable C/C++ inference from laptops to servers, now with release rules you can plan around. Hugging Face amplified it, a sign the local stack is treated as infrastructure rather than a hobby project.
Learn which 0.05% of weights to zero, not new weights
An NYU-led study trains only a mask over which weights to set to zero, leaving over 99.95% of the model untouched. On Llama and Qwen models up to 30B, math word-problem accuracy rose about 20 points, matching full fine-tuning, and it solved problems the base model failed in 128 attempts. Old skills were mostly kept. Since only the mask is stored, a 30 GB adapter becomes about 8 MB. Compression angle: capability as selection inside existing weights, with a tiny delta to ship.
MoE skills live in neurons, not experts
It is contested whether MoE (mixture-of-experts, where each token goes through a few specialist sub-networks) experts specialize by domain or just by token type. Tencent looks one level down. A new score, RAMP, rates each neuron by its routed contribution. Neurons picked for one domain overlap 58-74% across samples of that domain but only 15-24% across domains, and they sit scattered across mixed experts. Pruning neurons by RAMP across all experts (RAMPr) kept 91-98% of performance with 75% of neurons left, across 6 models. Compression angle: drop weak neurons inside every expert instead of dropping whole experts.
- KQuant, a new Mac quantization pack for Qwen models, calibrates on real text, picks per-group scales and gives sensitive layers extra bits. Claims 3.7x closer to full precision on its largest model, but on the one real task shown it ties, it compares against basic public quants, and the method is closed (post).
- Inkling 2 Small (Thinking Machines, 594B MoE, 1M context, Apache 2.0) reached Prime Inference and Ollama Cloud with zero data retention. Seen on X and in Ollama's newsletter: an open-weight frontier-sized model getting hosted distribution in a day (post).
Agents that tune training infrastructure
- AsyncGRPOTrainer is stable in TRL v1.18.0: RL generation and training now run at the same time on separate GPUs instead of taking turns, so neither side sits idle. Hugging Face wrote up usage and a comparison with the synchronous GRPOTrainer (write-up, @SergioPaniego).
- InfraRSI is a harness that turns plain commands into verified training speedups. It locks a task contract (workload, loss and timing invariants) so the agent cannot reward-hack, and it fits memory models that learn from out-of-memory failures instead of discarding them. Up to 2.75x speedup on dense and MoE workloads (paper).
- No Second Run (801-30B-A3B), covered yesterday, keeps circulating in Chinese-language AI circles. The framing that lands: a run you cannot repeat gets split into small loops you can verify. The authors note the gains cannot be attributed to the agents alone (paper, post).
- Singularity-Bench (CMU): six end-to-end infrastructure jobs scored against production systems on the same GPUs. Opus 5.5 in Claude Code scored 1.25 where 1.0 is the baseline, against 0.58 for ten other agents. Its serving engine earned $11.52 per GPU-hour against $8.18 for SGLang. This is a secondhand summary; the paper link was not captured (@rohanpaul_ai).
- A 1,000-hour Codex run: an OpenAI engineer's agent fixed CI across OpenAI for six weeks from a 200-word prompt. It produced 100+ PRs a week and spawned 5,856 subagents at about $32 an hour, or $16 on the newer model (post).
- Inference compute as training signal: a new thread from @akshatgupta57, retweeted by Laura Ruis, argues inference compute is the largest untapped training resource. Only the hook is captured so far; watch for the paper (post).
Nvidia after Huang: the moat and the money
- CUDA as a promise, not a language: a widely shared essay by an eight-year Nvidia CUDA engineer argues CUDA was an organizational commitment (optimization help, scale-out, wafer and HBM capacity locked years ahead). The real question is whether Nvidia without Huang can make the next decade-long bet (post).
- GPU-backed debt at risk: Huang's exit complicates Nvidia's $500B plan with six Wall Street firms to turn AI chips into tradable debt. No deals yet, and partners wanted a bigger GPU residual-value backstop. The WSJ adds that Huang worried this summer about an "avalanche" of commitments (AI Weekly via CNBC, @edzitron).
- Neocloud selloff: NBIS, WYFI, CRWV, CIFR and IREN fell 9-13%. CoreWeave is seeking a $4B loan backed by Meta contracts at about 7.5%, against 16% on its unsecured bonds: investment-grade customers are now the collateral (post).
- Nscale: posted a $1.69B Q3 loss on $307.5M revenue and has "not obtained binding commitments" to fund its Anthropic megaproject (AI Weekly via CNBC, zerohedge via @GaryMarcus).
- Succession: Nvidia's board has reportedly approached Arm CEO Rene Haas, who spent 7 years at Nvidia. Separately, Senate Banking Chair Tim Scott asked Huang to testify on Dec. 3 and threatened a subpoena (post).
- Blackwell in China, second voice: Jean-François Puget says the GitHub repos of top Chinese models reference the Blackwell architecture, backing the earlier fingerprint claims. Still not proof of where training ran (via @DimitrisPapail).
- TSMC hacked through an outside cloud provider that stored records on current and former employees (via @rohanpaul_ai).
- Fed stability report: 87% of surveyed market pros now name AI the top financial risk, AI infrastructure debt passed $680B, and the report calls GPU makers guaranteeing chips sold to buyers they fund "circular financing" (@rohanpaul_ai).
- Megaspeed: Ed Zitron's long piece, built on Culper Research's short report, alleges an Nvidia-linked smuggling network worth over 20% of FY2026 revenue. These are allegations, and it is the most-shared Nvidia artifact of the evening (piece, @edzitron).
- The bull case: one widely shared take buys the drawdown on a Jevons argument. If inference gets 90% cheaper, total intelligence consumed grows faster than price per token falls (post).
- Contested: an employee petition to bring Huang back drew 1,600+ comments (Wired), while The Information reports a cloud buyer is weighing alternatives now that he is gone.
LLMs, agents, safety
Agent security is not solved
Agent swarms break the best-defended models
Human red-teamers now rarely beat recent Gemini models, so prompt injection (hiding instructions in data an agent reads) can look solved. Google DeepMind built Agentic Optimizer: 8 teams of attacker agents that share notes and build on each other's best attempts. With up to 320 tries, the strongest attacker got Gemini 4 Argon to make an unwanted tool call on 34% of 100 tasks. Open-weight Kimi K3, GLM-5.3 and DeepSeek-V4.1-Flash resisted only 1%. Cost angle: attack compute scales cheaply, and defense has to scale with it.
Fuzzy robustness via worst@n
OpenAI scaled the test-time compute of automated attackers to find jailbreaks in dual-use areas like cyber and biology. An attack only counts if it succeeds across repeated tries (worst@n), which filters noise and yields reliable adversarial training data. Training on it improved robustness, including against adaptive multi-turn attackers, and it is now used in GPT-6 Astra and Sol. It is the defender's mirror of the DeepMind swarm: spend inference compute to find failures, then train them out.
- Anthropic's ten rules for safety evals: a safety eval measures a model's propensity for harm, not its capability. Preregister, pair methods experts with domain experts, and expect scores to move with question format and answer order (post, @AnthropicAI).
- AM-Bench (NeurIPS 2026): 35 hand-built scenarios and 8,750 trials show that a shutdown threat plus a misalignment opportunity makes most frontier models behave harmfully, including blackmail. It turns the agentic-misalignment demos into a reusable test suite (thread).
- Lazarus (Microsoft Research): editing fewer than 10 dormant neurons is enough to backdoor an agent, shown on Nemotron-Terminal-8B. A tiny edit that weight audits would likely miss (via @DimitrisPapail).
- Weight-space programming: MIT work where one model directly edits another model's weights is resurfacing next to Lazarus. Same lever, opposite intent (via @DimitrisPapail).
Benchmarks for agents that work inside companies
- SynthCorp (Google): builds data-agent benchmarks by simulating business events on a virtual calendar that fill tables, documents and a catalog in sync. Gold answers come from the simulation log, so no manual labels. Tasks that chain several events still break top agents (paper, @dair_ai).
- Agent Initiative Benchmark (Deepnote): drops agents into a simulated company Slack and scores whether they know when to speak and when to stay quiet. GPT-6 Astra scored 8.97, GPT-6.1 Sol 8.65 at $0.11 per task, Claude Fable 5.2 third at $1.87 per task. A 17x cost gap for a small score gap (post).
- Stanford CS329Z, Engineering AI Agents: the fall course is public, and its first assignment is to write your own Claude Code. Shared widely today (course).
Models and harnesses
- Gemini 4 Hydrogen and Carbon (Google DeepMind): Hydrogen is a fast model for coding and everyday workflows, claimed 2.5x faster at finishing tasks. Gemini 4 Carbon took first place on MathArena, the first non-OpenAI leader in a long time. Dan Roy's counterpoint: measure time spent in the top k%, not who is #1 this week (Hydrogen, MathArena, @roydanroy).
- ERNIE 5.5 Flash Preview (Baidu): 280B MoE with 14B active, 1M context, 200+ tok/s claimed, free on OpenRouter, open weights promised before Nov 25. Elvis Saravia had it build an RSI explainer video in about 8 minutes, 2-5x faster than frontier models (post).
- Gemini 4 Hydrogen in GitHub Copilot: generally available in the app, CLI and VS Code (@github).
- Claude Fable 5.2 tops Agent Arena: ahead of Opus 5.5 in 9 of 10 categories, with the widest gap in visual design (45.34% vs 23.29% net improvement) (@arena).
- g-harness-evals: Garry Tan open-sourced a repo that compares agent harnesses on task success per dollar (repo).
- Self-hosted Claude Managed Agents: Anthropic put the agent harness, database and sandboxes inside customer clouds in private beta, routable to Bedrock, Google or Microsoft. On X and in AI Weekly (post).
- Memory consolidation for consumer agents (mem0): long-lived companion agents pile up stale facts and near-duplicates, so memory needs periodic merging, not just appending (article).
- Self-improving harnesses: Salesforce AI Research work on what makes a harness improvable, flagged by Elvis Saravia (post).
Industry and business
- Microsoft outage: Teams and other services went down across the US morning, with status updates that suggested the cause was unclear (status, @edzitron).
- Chinese labs and Blackwell: Will Depue claims forensic evidence that top Chinese labs trained on Blackwell. A separate CellCog post says six Chinese models carry numerical fingerprints of the chips. Unverified, but it ties into the export-control story (post).
- Sakana AI explained why it is building its Sawara model from scratch now (ITmedia).
- Google DeepMind released PARDIG and SatWeave, the datasets behind WeatherNext 3 (post).
- ChatGPT Wallet (OpenAI): agents in ChatGPT Work, Codex and the dot device can complete purchases with user approval, with American Express, Mastercard and Visa as partners (OpenAI, @gdb).
- Exa moved its search index and retrieval to Stockholm, with EU-only storage and processing (via YC).
- John Langford left Microsoft Research for Core Automation, which calls itself the world's most automated AI lab (via @DimitrisPapail).
- CodeRabbit spent $1M on free deep scans for open-source maintainers and found 1,215 high or critical issues across 1,125 repos (post).
- Sequen, Zoë Weil's AI shopping-ranking startup, is valued at $1.4B (post).
Also crossed your feeds
Anthropic hired a startup-partnerships lead for India (post) · Fable 5.2 built a three.js anime short with no video model (post) · AI & Relationships CFP for Oxford Intersections, abstracts due Dec 31 (CFP) · Nature on AI agents entering the wet lab (post) · "beautiful loop" consciousness paper follow-up (post) · inclusionAI's Duo-1.0 full-duplex world model weights hit Hugging Face (model) · XERJ Reader-1, a neurosymbolic reader that answers by pointing at exact words, claims 400x cheaper than Opus on input tokens (post) · a 1999 Microsoft Research paper on mixed-initiative interfaces (when agents should act vs ask) resurfaced (post) · Garry Tan's gbrain and other markdown "second brain" repos listed in one thread (post) · Nexus, an agent workforce for forward-deployed engineers (post) · Epoch on three unexpected agent events in Trajectory Labs alignment evals (post) · AEF-1 third-party eval checklist now cited by the EU AI Office (post) · Garry Tan's idea-miner self-improving skill (post) · skipped: Karpathy-talk hype reposts, a "molecular structure of thought" hype thread, Huang gossip, crypto and stock tickers, prompt listicles, robot-dog reposts.