Summary
KV cache economics won the evening. The best post is a long thread on DeepSeek open-sourcing DeepInfer, the stack that serves cache hits from SSDs instead of GPU memory. A second post revisits cache engineering for agents (the AKV serving engine), so that makes a cluster of 2 on the same idea: keep the prefix stable and the cache becomes the cheapest compute you have. The money side is a cluster of 3 on GPU rents and financing: GB300 rents near $14/hr, IREN borrowing $363M against its GPUs, and Vertiv beating its numbers. Separately, a cluster of 2 says the coding interview is dead now that agents ace the puzzles. Most of the rest is truncated reposts, stock tickers and hashtag spam.
Posts
- DeepSeek open-sources DeepInfer: KV cache served from SSD (@_avichawla). MLA (multi-head latent attention, which stores one small compressed vector per token instead of full per-head keys and values) shrinks the cache enough that reading it from 3FS, DeepSeek's file system spread across thousands of SSDs, beats recomputing prefill. A 128K-token cache hit gets its first token in 500ms instead of 13s, and 56% of input tokens were once served from disk. The practical rule: put stable prompt parts first, because a hit costs 50x less than a miss. See KV cache and the earlier DeepSeek KV cache economics.
- Cache engineering instead of context engineering (@TheTuringPost). The Tsinghua/CUHK AKV engine adds "drop" and "repos" operations, so agents can prune old tool outputs without recomputing the KV cache. It reports 27.7% lower cost and 1.51x P95 latency. We already covered it in cache engineering for agents (AKV).
- GPU rents and infrastructure money (cluster of 3) (@StockSavvyShay · IREN · Vertiv). Deutsche Bank puts GB300 rents near $14/hr against $3.50 for H100, or about $48B a year per GW. IREN raised $363M of debt backed by GPUs already on contract, and Vertiv beat Q3 with $4.3B revenue. GPUs are now collateral, which supports the rising-rent thread in GPU unlock and rising GPU rents.
- Google TPUs on Oracle's cloud (@rohanpaul_ai). This is the first time TPUs run on a hyperscale cloud outside Google. It was already covered today in TPUs leave Google.
- Photonics and 40km of GPU disaggregation (@Ronald_vanLoon). IOWN claims a 1.9% penalty at 40km between data and GPUs, which would let compute sit wherever the power is. It is tagged #AD, so treat it as vendor framing until there is a paper.
- Liquid AI LFM3 (@rohanpaul_ai). A new open-weight family in which the 1.7B text model beats its 2.6B predecessor. It is worth a look for small-model efficiency. Click through to read.
- The coding interview is dead (cluster of 2) (@omarsar0 · @VaibhavSisinty). Anthropic's Felix Rieseberg dropped coding puzzles because Claude beats the candidates at them. Omar argues for hiring on agency and judgment instead. It is a strong opinion, but there is no data behind it.
- Agent tool selection and reranking (cluster of 2) (@rohanpaul_ai · @omarsar0). One says tool choice now matters more than which model writes the answer. The other is a paper comparing decision models with fine-tuned rerankers. Both are truncated reposts, so click through to read. Related: tool calling.
- Truncated reposts, click through to read (@rohanpaul_ai on agent swarms in verified math · @TheTuringPost papers of the week · @thegautamkamath on how research teams use coding agents · @rohanpaul_ai on Pichai's agents · @TheTuringPost on Oracle visual agent memory). The interviews about coding agents in research teams look like the most substantive of these.
- Funding and deals (Physical Intelligence · Meta shares). Physical Intelligence is in talks to raise $1B or more at about $20B, nearly double its last valuation. Nvidia, AMD and Broadcom have expressed nonbinding interest in buying up to $5B of Meta stock.
- Anthropic IPO discourse (@GaryMarcus · @rohanpaul_ai on the $40T TAM claim · @GaryMarcus on a unilateral pause). The S-1 is drawing criticism for sizing its market as all knowledge-worker output, plus a debate over Anthropic simply stopping. Mostly heat, with little new information.
- Chasing the Karpathy peak (@IntuitMachine · essay). A Substack essay linked without commentary. The rest of IntuitMachine's posts tonight are one-line musings on tacit knowledge. Skim the essay only.
- Genie 4 (@rohanpaul_ai). DeepMind's real-time world model now generates native audio. It is a multimodal story with no efficiency angle.
- Event, ticker and hashtag noise (EMNLP tutorial posts from @sarahookr, the Sakana office visit, $SKHY/$OSS tickers, Ronald_vanLoon's robot and solar reposts, a Dwelly fundraise). Skip.