AI research wiki
October 19, 2026 · daily digest

cere-bro | 2026-10-19

cere-bro | 2026-10-19

The cheapest compute is the compute you place well. Cohere gets within 1.4 points of an all-frontier coding team at 70% lower cost by giving cheap models only the planning. Meta gets 1.5x more training throughput from AMD chips by redesigning the model instead of porting it. Memorilla answers from 16 memory tokens instead of whole documents. Meanwhile the supply side splits: SemiAnalysis says 2027 adds +50GW, while Microsoft walks away from a neocloud site and Nvidia tightens its grip on HBM.

🎯 Today's 5 for you
  1. ReadSemiAnalysis: Datacenters 2027, +50GW in one year. About $1T of binding leases signed in 2026 to date says the build is already funded. It hints memory and logic become the real limit in 2027-28. Your semiconductor and compute-supply thread ([essay](https://newsletter.semianalysis.com/p/datacenters-2027-the-year-of-acceleration)).
  2. ReadMeta trains on AMD MI350X at 1.5x the ported speed. The first hyperscaler report on AMD training in production. The gain comes from reshaping the model around 1.6x more HBM, analytical kernel picking and agentic CUDA porting. GPU optimization in production ([paper](https://arxiv.org/abs/2610.22230)).
  3. ReadCohere: cheap models plan, the strong model builds. A strong lead with 16 cheap planners matches an all-strong team within 1.4 points at 70% lower cost. Cheap coders hurt. Routing by role, the next rung after yesterday's JevAgentHarness ([paper](https://arxiv.org/abs/2610.17471)).
  4. SkimMemorilla: a document set in 16 tokens. A frozen LLM beats its own full-context score on FactKG with 108x fewer source tokens. Context compression without touching the KV cache. Check for exact-recall tests before trusting it ([paper](https://arxiv.org/abs/2610.21375)).
  5. TrackNvidia wants the HBM base die on TSMC 3nm. Standardizing the logic layer lets Nvidia swap memory suppliers as HBM prices climb. Your memory supply-chain theme. Skip the "Anthropic published 400 agents" and "End of Harness Engineering" hype threads ([The Information](https://www.theinformation.com/briefings/nvidia-tells-suppliers-tap-tsmc-process-next-generation-memory-chips)).

TL;DR

Deep Dives

SemiAnalysis: Datacenters 2027, the year of acceleration

The whole world's datacenter capacity up to 2024 gets built again in a single year. SemiAnalysis says the leases that pay for it are already signed.

Source: SemiAnalysis (Gmail + RSS) Links: Essay · Wiki summary

Leases signed today are the GW delivered next year
SemiAnalysis reads the build two steps ahead: contracts fund debt, debt funds construction.
flowchart LR
  L["Pre-leases<br/><small>15-20 yr, $30-50B/GW</small>"] --> D["Debt raises<br/><small>funded against leases</small>"]
  D --> C["Construction<br/><small>self-build starts parabolic</small>"]
  C --> G["+43-50 GW<br/><small>delivered in 2027</small>"]
  M["Modular design<br/><small>under 10-month builds</small>"] --> C
  P["Behind-the-meter power<br/><small>skips grid queue</small>"] --> C
  G --> X["Next constraint<br/><small>memory and logic</small>"]
  classDef input fill:#d0ebff,stroke:#1971c2,color:#1b1b1b,stroke-width:2px
  classDef core fill:#e5dbff,stroke:#6741d9,color:#1b1b1b,stroke-width:2px
  classDef loop fill:#fff3bf,stroke:#f08c00,color:#1b1b1b,stroke-width:2px
  classDef exit fill:#d3f9d8,stroke:#2f9e44,color:#1b1b1b,stroke-width:2px
  classDef err fill:#ffe3e3,stroke:#e03131,color:#1b1b1b,stroke-width:2px
  class L input
  class D,C core
  class M,P loop
  class G exit
  class X err
  linkStyle 2 stroke:#2f9e44,stroke-width:2px
Blue is the contract signal, purple is the financing and build chain, amber is what shortened the build, green is delivered capacity, red is the next bottleneck.

What is it about? A SemiAnalysis report defending its forecast that 2027 adds roughly +43 to +50GW of datacenter capacity. It answers Brad Gerstner of Altimeter, who said on the All-In podcast that only about 25GW would get built.

What problem does it solve? Investors keep pricing a slowdown on "hunches" about power, labor and capital. The report replaces the hunch with two leading indicators that move a year or more before capacity goes live.

What's the core novelty? It tracks the chain that precedes construction. Developers spend only 5-10% of a project's capital before a customer signs a pre-lease (a 15-20 year contract worth $30-50B per GW). Leases unlock debt, and debt pays for the build. In 2026 so far, slightly under $1T of binding, non-cancellable leases have been signed. Hyperscaler self-build construction starts are, in its words, parabolic.

Key takeaways

Gaps in the study The oversupply section and the starts chart are paywalled. The $1T lease figure is proprietary and cannot be checked. Leases do not tell you whether chips and memory arrive on time to fill the halls.

Industrial implication If the leases convert, 2027 compute gets cheaper per token and capex numbers get "unthinkable." SemiAnalysis expects rates to rise to fund it. The bottleneck then moves to HBM and leading-edge logic, which is exactly where Nvidia moved today (see the HBM Deep Dive below).

→ Full summary

Meta: co-design for AMD MI350X beats porting by 1.5x

Moving a model from NVIDIA to AMD is not a port. It is a redesign, and the redesign is where the 1.5x comes from.

Source: X (arXivBangers) Links: Paper · Wiki summary

The port is the baseline, the redesign is the win
Three steps turn an NVIDIA-shaped model into an AMD-shaped one.
flowchart LR
  N["NVIDIA-tuned model<br/><small>years of CUDA tricks</small>"] --> P["Direct port<br/><small>1.0x baseline</small>"]
  N --> C["Co-design<br/><small>use 1.6x HBM capacity</small>"]
  C --> K["Analytical kernel pick<br/><small>no warm-up tuning</small>"]
  K --> A["Agentic porting<br/><small>shims, arch-aware kernels</small>"]
  A --> R["MI350X training<br/><small>up to 1.5x throughput</small>"]
  classDef input fill:#d0ebff,stroke:#1971c2,color:#1b1b1b,stroke-width:2px
  classDef core fill:#e5dbff,stroke:#6741d9,color:#1b1b1b,stroke-width:2px
  classDef loop fill:#fff3bf,stroke:#f08c00,color:#1b1b1b,stroke-width:2px
  classDef exit fill:#d3f9d8,stroke:#2f9e44,color:#1b1b1b,stroke-width:2px
  classDef err fill:#ffe3e3,stroke:#e03131,color:#1b1b1b,stroke-width:2px
  class N input
  class P err
  class C,K loop
  class A core
  class R exit
  linkStyle 1 stroke:#2f9e44,stroke-width:2px
  linkStyle 2 stroke:#2f9e44,stroke-width:2px
  linkStyle 3 stroke:#2f9e44,stroke-width:2px
  linkStyle 4 stroke:#2f9e44,stroke-width:2px
Blue is the starting model, red is the naive port, amber is the design choices, purple is the agent step, green is the result.

What is it about? Meta runs NVIDIA GPUs, AMD GPUs and its own MTIA chips side by side. This paper explains how it made recommendation-model training run well on AMD Instinct MI350X, compared against NVIDIA B200.

What problem does it solve? Years of NVIDIA-specific tuning live inside Meta's models. Move a model to AMD as-is and much of that tuning stops helping. Until now, hyperscalers had published little on whether AMD training works at production scale.

What's the core novelty? Three reusable rules. First, change the model to fit the chip: MI350X carries 1.6x the HBM (the stacked memory on the accelerator) of B200, so the model and its recipe are reshaped to use it. Second, an analytical performance model picks the best kernel for each datatype and matrix shape with zero warm-up runs, instead of slow autotuning. Third, agentic workflows translate CUDA code, extended to write microarchitecture-aware kernels and custom shims (thin glue layers that replace NVIDIA-only libraries).

Key takeaways

Gaps in the study Recommendation models only, not LLM pretraining. The baseline is Meta's own direct port, not a tuned B200 run, so this does not show MI350X beats B200. No cost-per-step or performance-per-watt numbers in the abstract.

Industrial implication Multi-vendor training stops being a negotiating bluff. If the recipe holds for LLMs, AMD's bigger HBM becomes a design advantage rather than a spec line. This lands the same day Nvidia moved to control the HBM base die, which looks like Nvidia defending the exact memory edge Meta is exploiting.

→ Full summary

Where agent compute pays: Cohere's coding teams, plus a token audit of security agents

More agents cost 3 to 7 times more. The fix is not fewer agents. It is giving cheap models the one job they cannot ruin.

Source: X (Rohan Paul) Links: Cohere paper · Token audit paper · Wiki summary

Cheap models plan, the strong model builds
Workers help most at the planning stage, and that is the only stage where a cheap model does not hurt.
flowchart LR
  T["Task<br/><small>rebuild a program</small>"] --> L["Strong lead<br/><small>GPT-5.6 Sol</small>"]
  L -->|fan out| P["16 cheap planners<br/><small>Luna, research only</small>"]
  P -->|plans| L
  L --> I["Lead implements<br/><small>and verifies itself</small>"]
  I --> R["Result<br/><small>-1.4 pts, -70% cost</small>"]
  X["Cheap implementers<br/><small>lower the reward</small>"]
  classDef input fill:#d0ebff,stroke:#1971c2,color:#1b1b1b,stroke-width:2px
  classDef core fill:#e5dbff,stroke:#6741d9,color:#1b1b1b,stroke-width:2px
  classDef loop fill:#fff3bf,stroke:#f08c00,color:#1b1b1b,stroke-width:2px
  classDef exit fill:#d3f9d8,stroke:#2f9e44,color:#1b1b1b,stroke-width:2px
  classDef err fill:#ffe3e3,stroke:#e03131,color:#1b1b1b,stroke-width:2px
  class T input
  class L,I core
  class P loop
  class R exit
  class X err
  linkStyle 1 stroke:#f08c00,stroke-width:2px
  linkStyle 2 stroke:#f08c00,stroke-width:2px
Blue is the task, purple is the strong model, amber is the cheap planning fan-out, green is the result, red is the design that fails.

What is it about? Cohere treats a team of coding agents as a budget problem. A lead agent hands subtasks to workers. The authors change one thing at a time: which stage workers help with (planning, implementing or verifying), which model the workers use, and how many there are. The test is ProgramBench, where agents rebuild a whole program from only its executable and docs. A second paper audits where security agents spend their tokens.

What problem does it solve? Multi-agent teams promise a new way to scale inference, but in practice they duplicate work and cost 3 to 7 times a solo agent. Nobody had mapped which extra agent buys how much reward.

What's the core novelty? It finds the reward-cost Pareto frontier (the set of designs where you cannot get more reward without paying more) by role. Planning is the job where extra workers help most, and it is the only job where a cheap model is safe. The security-agent audit adds the matching lesson for tokens: measure where they go before cutting.

Key takeaways

Gaps in the study One benchmark and one model pair. No wall-clock latency for a 17-agent team. AVRI is tested on 20 tasks.

Industrial implication Coding-agent products should ship mixed-model teams by default: strong model for the lead and the code, cheap models for research fan-out. This is the third routing paper in three days to move the routing decision inside the agent. Expect "subagent model" settings to become a standard knob.

→ Full summary

Memorilla: a whole document collection in 16 memory tokens

A frozen LLM scored higher reading 16 learned tokens than reading the documents themselves.

Source: X (Elvis Saravia), DAIR Academy pick Links: Paper · DAIR page · Wiki summary

Encode once, read 16 tokens per query
The frozen LLM never sees the documents, only a query-specific 16-token summary in its own embedding space.
flowchart LR
  D["Documents<br/><small>encoded once, reused</small>"] --> E["Memory bank<br/><small>stored embeddings</small>"]
  Q["Query<br/><small>user or agent step</small>"] --> W["Writer module<br/><small>gathers across bank</small>"]
  E --> W
  W --> M["16 memory tokens<br/><small>108x fewer than docs</small>"]
  M --> L["Frozen LLM<br/><small>weights untouched</small>"]
  L --> A["Answer<br/><small>84.3% vs 80.4% full</small>"]
  classDef input fill:#d0ebff,stroke:#1971c2,color:#1b1b1b,stroke-width:2px
  classDef core fill:#e5dbff,stroke:#6741d9,color:#1b1b1b,stroke-width:2px
  classDef loop fill:#fff3bf,stroke:#f08c00,color:#1b1b1b,stroke-width:2px
  classDef exit fill:#d3f9d8,stroke:#2f9e44,color:#1b1b1b,stroke-width:2px
  class D,Q,E input
  class W loop
  class M,A exit
  class L core
  linkStyle 3 stroke:#2f9e44,stroke-width:2px
Blue is stored and incoming data, amber is the learned writer, purple is the frozen model, green is the compressed memory and the answer.

What is it about? A memory system from Stanford and Toronto. Documents are encoded once into stored embeddings. For each question, a learned module reads across the whole collection and writes what matters into 16 "memory tokens" that sit in the LLM's input like ordinary tokens.

What problem does it solve? Long context is paid for twice: once in prefill (reading the prompt) and again in the KV cache (the stored attention keys and values for every token in context). RAG and long-context models pay that cost on every query. Memorilla pays it for 16 positions.

What's the core novelty? The compression is query-specific and lands in the frozen model's own embedding space. No weights change and no cache is evicted. Only the writer module is trained.

Key takeaways

Gaps in the study No exact-recall or needle-in-a-haystack results in what we saw, and that is where soft-prompt compression usually fails. No writer cost or latency per query. Every result is a QA score, and Amazon's FEB-30 (below) shows QA scores can hide a broken memory.

Industrial implication For fact-checking and agent memory, a small writer in front of a frozen model could replace most RAG context. That shrinks serving cost far more than any KV eviction scheme. It sits next to the two routes this wiki already tracks, eviction and putting context into weights through predicted adapters, as a third, cheaper one.

→ Full summary

Nvidia moves to control the HBM base die

Nvidia wants every HBM supplier to build the same TSMC 3nm logic layer. Whoever stacks the DRAM on top becomes swappable.

Source: The Information (via Seoul Economic Daily), X feed Links: The Information · HBM price forecast · Wiki: memory hierarchy

One base die, three interchangeable memory makers
Standardizing the logic layer moves bargaining power from the memory makers to Nvidia and TSMC.
flowchart LR
  S["Samsung<br/><small>DRAM stack</small>"] --> B["Base die<br/><small>TSMC 3nm, Nvidia spec</small>"]
  H["SK hynix<br/><small>DRAM stack</small>"] --> B
  M["Micron<br/><small>DRAM stack</small>"] --> B
  B --> G["Custom HBM<br/><small>next Nvidia GPU</small>"]
  T["TSMC<br/><small>wins every order</small>"] --> B
  classDef input fill:#d0ebff,stroke:#1971c2,color:#1b1b1b,stroke-width:2px
  classDef core fill:#e5dbff,stroke:#6741d9,color:#1b1b1b,stroke-width:2px
  classDef loop fill:#fff3bf,stroke:#f08c00,color:#1b1b1b,stroke-width:2px
  classDef exit fill:#d3f9d8,stroke:#2f9e44,color:#1b1b1b,stroke-width:2px
  class S,H,M input
  class B loop
  class T core
  class G exit
Blue is the memory makers, amber is the standardized base die, purple is the foundry, green is the product.

What is it about? Nvidia reportedly asked Samsung, SK hynix and Micron to build the base die of its next custom HBM on TSMC's 3nm process. The base die is the logic chip at the bottom of an HBM stack that talks to the GPU.

What problem does it solve? HBM is getting more expensive while volume grows. A widely shared investor forecast puts HBM near $35/GB by 2030, more than double today, with a possible $640B market. If each supplier's base die differs, Nvidia is locked to whoever qualifies first.

What's the core novelty? A common, Nvidia-specified logic layer at one foundry. It makes the DRAM layers the commodity part, so Nvidia can shift volume between suppliers as prices move.

Key takeaways

Gaps in the study A single press report, and the HBM price numbers come from an investor account, not filings. No word on whether the memory makers agreed.

Industrial implication Memory makers lose pricing power on Nvidia's custom parts just as SemiAnalysis says memory becomes the 2027 bottleneck. Watch whether AMD and the custom-ASIC buyers, which HSBC put at 57% of HBM demand by 2028 (10-18), follow with their own standard.

Agent shorts: audit the memory store, and build the data world first

Two agent papers that fix how we measure and train: one shows QA scores hide broken memory, one makes open-ended data tasks gradable.

Source: X (Rohan Paul, Elvis Saravia), DAIR Academy pick, The Decoder Links: FEB-30 · Data World · Taste-Bench · Wiki summary

What is it about? Amazon's FEB-30 grades an agent's memory store directly instead of only its answers. Data World generates data-analysis environments for RL from a written-down generating mechanism.

What problem does it solve? Memory benchmarks mix storage, retrieval and answer writing into one score. Data agents are hard to train because environments are costly and open-ended findings are hard to grade.

What's the core novelty? FEB-30 scripts 30 fact histories into 147 documents, so the right memory state is known after every document. Data World specifies the world before generating the files, then audits that each answer can be recovered from them.

Key takeaways

Gaps in the study FEB-30 tests one memory architecture. Data World's held-out benchmark is only 100 tasks.

Industrial implication Teams shipping agent memory should test stored facts after each update, not just end-to-end QA. That applies directly to Memorilla above and to Tencent's Hy-Memory (10-17, a 3B-active memory writer that matched Claude Opus 5).

→ Full summary

Industry Pulse

Hardware and semiconductors

Funding, valuations, and compute deals

Global View

The compute shortage is real at the top and fragile at the edges. SemiAnalysis (today, arguing about $1T of signed leases funds +50GW in 2027) and Amazon's CEO (10-18, reportedly saying compute is sold out beyond 2028) describe a market where even SpaceX says it cannot rent inference capacity. Yet the same window saw Microsoft drop Nscale's UK site over power, Firmus pull its IPO, and Nebius carry 5 GW of contracts against 0.6 GW live. The split is by builder: hyperscaler self-builds with behind-the-meter power are on schedule, while neoclouds with debt and no track record absorb the delays. On the research side, MAI-Base-2 (10-17, 21% average MFU, meaning only a fifth of peak FLOPs did useful work on 16,000 GB200s) says the cheapest new capacity is still better utilization of what exists.

Leverage is moving into memory, and both sides of the GPU market are playing for it. Nvidia's push to standardize the HBM base die at TSMC makes memory makers swappable just as HBM prices are forecast to double by 2030. On the same day, Meta showed that AMD's 1.6x larger HBM is worth up to 1.5x training throughput once the model is redesigned around it, and HSBC (10-18) put custom chips at 57% of HBM demand by 2028. Research keeps pricing memory as the scarce input too: Memorilla answers from 16 tokens instead of a full KV cache, and the 10-17 remote-KV paper found that on B300s fetching a cached prefix only beats recomputing it above about 17K tokens. If memory is the 2027 gate, as SemiAnalysis hints, then context compression stops being a research nicety and becomes a capacity plan.

Routing has moved inside the agent, and the open question is who checks the cheap model. In three days the wiki logged three versions: SpecAgent, EdgeWise and LEASH (10-17, weak model drafts steps, strong model reviews), JevAgentHarness (10-18, a 4B model takes single tool choices behind a confidence threshold fitted on the agent's own runs), and today Cohere (cheap models only plan, strong model writes the code, 70% cheaper). Price data says the prize is large: DeepSeek's 1017 Flash now leads open models on Code Arena at $0.3/$1.2 per million tokens, and a circulated comparison puts Opus 5 at $2.34 per agent task against about $0.40 for Gemini. Anthropic's Project Deal is the warning: Haiku agents lost about $2.50 a trade to Opus agents and their users never noticed. Every working design this week keeps a strong model or a fitted threshold between the cheap model and the outcome.

Looking Ahead