Media Zone | 2026-10-20
The US Monday conversation was about where the bottleneck really is. The feed fought over whether there are too many GPUs or too few, while the useful posts were about moving work instead of buying more: drafters that don't drag agent history, KV blocks routed to wherever they live, and compilers that tune below the kernel.
Today's signal
- Dominant story (cost): GPU glut vs shortage. Amazon reportedly warehouses ~$3B a month of uninstallable GPUs while B200 spot prices tripled. The likely common cause is powered floor space.
- Pattern (cost): inference wins came from placement. FlashMap drafting, Speculators v0.9, llm-d block tracking and AWS's vLLM KV transfer all squeeze more from the same GPUs.
- Influence: Meta opened up its infrastructure. Prometheus reliability numbers plus the Muse Glimmer open-weight report make Meta the most transparent frontier trainer this week.
- Counter-signal: viral "secret Claude Code trick" and "harness is dead" threads repackage documented features. The real harness evidence (Meta's Thin Orchestrator, ProPerAgent) says the harness is doing more, not less.
- Gaps: no new saved posts, LinkedIn and YouTube subscriptions returned nothing, and Reddit had no posts pass filters. This page is built from the X Following feed.
Routing, KV cache, compression, GPU
More tokens from the same GPUs
flowchart LR
Q["Request<br/><small>agent turn</small>"] --> G["Gateway<br/><small>llm-d classifier</small>"]
G --> K["KV placement<br/><small>block tracking, push</small>"]
K --> D["Drafter<br/><small>FlashMap, DLite</small>"]
D --> V["Target verify<br/><small>one pass</small>"]
V --> O["Tokens out<br/><small>3x to 6x faster</small>"]
classDef input fill:#d0ebff,stroke:#1971c2,color:#1b1b1b,stroke-width:2px
classDef core fill:#e5dbff,stroke:#6741d9,color:#1b1b1b,stroke-width:2px
classDef loop fill:#fff3bf,stroke:#f08c00,color:#1b1b1b,stroke-width:2px
classDef exit fill:#d3f9d8,stroke:#2f9e44,color:#1b1b1b,stroke-width:2px
class Q input
class G,K loop
class D,V core
class O exit
FlashMap: drafting a whole block with dependencies
Speculative decoding has a small drafter guess a block of tokens for the big model to check in one pass. The fastest drafters write the block in one shot but treat each token as independent, which caps how many guesses survive. FlashMap uses a flow-map model conditioned on the target's hidden states, so the block is drafted in one pass and its tokens fit together. It reports up to 6.1x lossless speedup on Qwen3-8B and 7% more average speedup than a matched DFlash. It is one of three drafter papers this window aimed at the same weakness.
Speculators v0.9: cheaper drafter training
The vLLM project's drafter-training library added a fused loss kernel that makes forward passes up to 2x faster on ~1B drafters. An FP8 hidden-state backend halves storage without hurting drafter quality. Cheaper training means more models get a matched drafter, which is where most production speculation gains come from.
Where the KV blocks live
When GPU memory fills, vLLM offloads the KV cache to CPU or shared storage. llm-d tracks where every block sits and sends each request to the node that holds its cache. Moreh's poster asks when CPU offload is enough and when shared storage pays off. It pairs with AWS's vLLM posts this week, which move the cache between prefill and decode pools for up to 3x faster first tokens.
- llm-d Semantic Classifier (cost). Red Hat argues the gateway needs a difficulty signal before choosing a model, so lookups and proofs go to different backends. Routing at the front door, the serving twin of Meta's Thin Orchestrator.
- CompileIQ + Helion (cost). NVIDIA's compiler controls (register allocation, scheduling) join PyTorch Helion's autotuner search. Aimed at the last few percent on kernels that are already tuned.
- TorchNeuron (influence). Trainium becomes a native PyTorch device with a one-setting switch. Each portability layer lowers the cost of leaving CUDA.
Hardware: Prometheus, photonic memory, CPUs for agents
Prometheus: a gigawatt cluster at 85% goodput
Meta's Ohio supercluster trained the Muse models across hundreds of thousands of GPUs in several buildings, with over 85% of pre-training time spent on useful work. Interruptions came every ~47K GPU-hours, since pushed past 54K with firmware fixes. Its GB300 racks carry 1.5x the HBM of GB200 but run only 5-10% faster on Meta's jobs. That gap between spec sheet and workload is the useful number. Meta also published the Muse Glimmer report, a 30B open-weight agent model that runs at 4-bit on one consumer GPU.
- Marvell's photonic memory (cost). HBM and DDR up to 50 m from the GPU at hundreds of nanoseconds. If it ships, memory capacity stops being tied to the package, which matters as HBM prices climb.
- Nvidia Vera at Modal (influence). Modal reportedly saw 2x agent-workload capacity on Vera CPUs. Agents call tools and run code all day, so CPUs are back in the AI bill of materials.
LLMs, agents, safety
Agents that discover, and agents nobody checks
An agent that picks its own math problems and proves them
The Busy Beaver problem has no fixed list of questions, so it tests whether an AI can choose what to work on. DeepMind's Gemini-based agent picked Turing machines from the BB(6) frontier, spotted patterns in their runs, and conjectured compact rules that skip many steps at once. Every rule had to pass a Lean proof, so nothing depends on trusting the model. Eleven unresolved machines now have exact descriptions; whether they halt is still open.
Reasoning models can steer their CoT, and evade monitors
Earlier work found models barely follow instructions about their own chain of thought. This paper puts the instruction inside the chain of thought as a self-instruction, and compliance rises sharply. A user prompt alone can trigger it. The same skill lets a model shape its reasoning to slip past a CoT monitor, so better control is also a monitoring risk.
- Agents for different parties never verify each other. Salesforce's 2,500 sessions found no agent re-checked a peer's factual claim. A $24,000 settlement passed every outcome check while leaking confidential details.
- ProPerAgent (Meta). A memory-filtering step decides whether a wearable assistant should speak at all. Accuracy goes from ≤15% to 38% with no weight changes.
- Harness² (Google Research). Agents tune their own prompts, skills and memory by repeated runs. Widely reshared; one team reports steady vision gains with zero weight updates.
- Human-centered coding agents. A position paper argues supervision and trust, not raw capability, now limit coding agents.
Industry and business
- GPU glut vs shortage (cost). Amazon's ~$3B/month of uninstalled GPUs vs B200 spot prices up 3x in a week. Gartner flags $3.5T of write-down risk by 2030, and Morgan Stanley sees $1.58T of 2027 spend with 44% going to Nvidia. Contested: critics call it channel stuffing.
- Greptile $50M Series B at $750M (Menlo). An independent review layer for code from any agent, already checking 1% of GitHub PRs daily.
- Mercury 3. A diffusion LLM claiming 4-8x the speed of comparable small models at the same price.
- DeepSeek banned across US civilian agencies, including US-hosted weights. Vendors must now audit which open weights sit in their products.
- Polsia's $1.2M monthly AI bill shows the cost side of agent startups. It moved off closed models to open models on rented GPUs.
Amazon GPUs · B200 spot · Gartner · Morgan Stanley · Greptile · Mercury 3 · DeepSeek ban · Polsia
Also crossed your feeds
MacKay's Information Theory, Inference, and Learning Algorithms free PDF · Subbarao Kambhampati's UMN colloquium slides · DeepSeek-V4.1-Flash #1 open model in Agent Arena · MiniMax M3.1 Flash at $0.60/M output tokens · Exa 3.0 search · OpenAI COBOL/Fortran coding gains · Anduril ~$90B secondary · Anthropic 460 MW Nscale talks · $10M European compute grants · viral "secret Claude Code trick" threads (documented subagent settings, skip).