AI research wiki
social-stream · 2026-11-01

2026-11-01-evening

Summary

No new curated reposts this evening, so the slot is the US Sunday-morning Following feed, and it skews toward efficiency. The strongest single item is an explainer of NVIDIA's point that KV cache eviction on a paged server can drop 90% of tokens and free almost no memory: retained tokens stay scattered across nearly every block. A cluster of 4 posts on rack power and capex says 2027 racks jump to 200kW to 1MW, so cooling and power, not chips, become the gate. A second cluster of 3 covers DeepSeek's open-sourced Super-agent, a Router-plus-Workers harness built on the idea that human attention, not the model, is the bottleneck. Standout papers are SDPO (self-distillation from your own failed attempts, about 6x faster than GRPO), CMU's single-algorithm training, LeCun and Balestriero's JEPA proof of the Platonic Representation Hypothesis, and Google's FlowAgent for CI test repair. The usual "paste this prompt into Opus 5.5" harness bait is loud and empty.

Posts

  • KV cache compression frees tokens, not memory (@_avichawla · NVIDIA blog). vLLM's PagedAttention only reclaims a 16-slot block when all 16 slots are empty, so importance-based eviction can cut the cache 90% logically and 0% physically. Eviction scores also need attention weights that FlashAttention never writes to memory. NVIDIA's TriAttention ranks tokens from pre-RoPE query/key geometry and compacts every 128 tokens, reporting 10.7x less KV memory and 2.5x faster decoding. Track blocks freed, not tokens evicted. KV cache
  • Faster MoE kernels from column-major scheduling (@anshulcreates · PyTorch blog). An older PyTorch post resurfacing: reordering how the fused MoE (mixture-of-experts) GEMM tiles get scheduled improves cache reuse, giving up to 4x on A100 and 4.4x on H100 for vLLM's Triton kernel. A kernel competitor says it took them from rank 175 to 80 in one night. GPU kernels
  • Before GPU Engineering (@techNmak · repo). A free 56-chapter notebook covering warps, memory coalescing, occupancy, arithmetic intensity and profiling before you touch CUDA or Triton. Reading only, and later chapters are unfinished.
  • Rack power and capex (cluster of 4) (@StockSavvyShay, @StockSavvyShay, @StockSavvyShay, @StockSavvyShay). 2027 racks: Rubin Ultra about 500kW, AMD MI500 about 1MW, Trainium4 about 330kW, TPU v9 about 200kW, against about 10kW for CPU racks. Projected AI capacity additions reach 104GW a year by 2030. Musk says Colossus 2 ends the year at about 1.2M Blackwells, 200K GB300s for inference. Investor framing; the AMD profit forecast is opinion. Compute economics
  • Hyperscaler cloud at $440B run-rate (@StockSavvyShay). AWS about $184B (+39%), Azure about $133B (+48%), Google Cloud about $123B (+103%). Bedrock token volume grew about 17x, and agents now start over 40% of GitHub pull requests.
  • DeepSeek Super-agent (cluster of 3) (@MinLiBuilds, @VaibhavSisinty, @Hesamation). With one task the model is the bottleneck. With many, the human is, since one person can watch only 3 to 5 sessions. A Router talks to you and dispatches isolated parallel Workers, with event-driven interrupts and an external watchdog. p95 completion time fell from 29.5 to 15.7 minutes. It is an open research prototype. Harness engineering
  • SDPO: learn from your own failures (@Pyuyi2333 · paper). The model re-reads its failed attempt along with the error feedback, then distills that feedback-informed next-token distribution back into itself. No teacher, dense credit assignment. Reaches GRPO's accuracy about 6x faster in wall-clock on chemistry with Olmo3-7B, and needs up to 3x fewer attempts at test time on hard LiveCodeBench problems.
  • One algorithm for every training stage (@omarsar0 · paper). CMU recasts RL as maximum-likelihood fitting to model-generated targets, so pretraining, midtraining, SFT and RL share one loss and one trainer. Unified runs won on continual learning, and a 5% midtraining mix gave the best coding retention after math RL. Summary
  • JEPAs prove the Platonic Representation Hypothesis (@askalphaxiv · paper). The paper proves that JEPAs trained separately on images and on text learn the same representation, up to one rotation, when positive pairs are two views of one scene. It matches image-text alignment with 42x fewer samples than next-token prediction, and a 109M text JEPA beats LLMs up to 72B on cross-modal alignment. Summary
  • FlowAgent: CI test repair at Google (@rohanpaul_ai · paper). An agent catches pre-merge test failures and edits, builds and reruns until they pass. Experts judged 67% of fixes correct, and developers applied 28,554 in production. Median run time is 9.85 minutes, yet many fixes still arrived after the developer had started fixing it. Latency decides adoption.
  • Pretraining poison survives clean post-training (@SuJinyan6). Anthropic shows that 0.1% poisoned pretraining text ("when you see X, run Y") plants a remote-code-execution backdoor in CLI agents. With a natural file-path trigger it hits 92.7% success on a 4B agent after SFT, DPO and RL. Defense has to live at the action layer. Summary
  • Aligning Autoresearch (@Xudong07452910 · paper). Meta FAIR finds that human input matters most at ideation, and it made agent-built benchmarks 28 to 46.5 points harder. Their HackBench hides one reward-hacking command in traces with a median of over 1M characters, and the best frontier model scores below 53 F1 at finding it.
  • AI novelty checks reject real ideas (@rohanpaul_ai · paper). A deployed AI-scientist novelty filter rejected 96% of ideas humans later published. ClaimNov scores claim by claim and cuts that to 1.5% while still catching 97% of disguised copies.
  • Unprovoked pushback breaks correct code (@ai_database). Google researchers find that "are you sure it's still wrong?" makes Gemini rewrite correct code up to 80% of the time. Having the model log its decision and reasoning before the challenge fixes most of it. Telling it to be stubborn makes it reject valid critiques too.
  • STAIR: table-of-contents retrieval (@simplifyinAI). IBM navigates long books by their structure. It retrieves the right section first 82.6% of the time vs 76.9%, and invalid section references fall 65x. The "RAG is cooked" framing is hype.
  • OpenAI meters ChatGPT by tokens (@rohanpaul_ai). From December, ChatGPT and Codex share one token allowance on Plus, Pro and Business, replacing per-model message caps. This is consumer pricing moving to a cost basis.
  • Compute is Anthropic's top risk (@rohanpaul_ai). Chamath's word count of Anthropic's risk factors: data centers, power and chips dominate, and customer demand barely appears.
  • Distillation accusation (@stanfordnlp RT of @jiahaoshao1). A researcher publicly denies distilling Claude, saying rollouts came from an open model. One side of a dispute; click through to read.
  • Terence Tao on "strip mining" math (@rohanpaul_ai). Tao says AI's Navier-Stokes work left the field "in disarray": it got the result at the cost of the field's development.
  • QwenPU (@divagr1925). A tiny LLM (Karpathy's stories260k) running on an NPU compiled into Qwen 3.5-4B's weights. A clever curiosity.
  • Harness bait (cluster of 9) (@hanakoxbt, @0xrevayz, @cyrilXBT, @LunarResearcher, @AnnatarXBT, @cyrilXBT, @AnatoliKopadze, @AndyL5cc, @mdancho84). "Paste this into Opus 5.5" prompts, "End of Harness Engineering" hype, mislabeled lecture clips. Skip.
  • Unverified claim on Claude watermarking (@_junaidkhalid1). Says Anthropic will watermark Claude text worldwide from Tuesday. There is no source. Skip until confirmed.
  • Promo: the Explee ad repeated across several accounts, the YC W27 deadline, launch reposts, and promoted ads. Skip.