Media Zone | 2026-08-03

Ten conference talks landed at once and they are having an argument with each other about who grades the agent.

Today's signal

Dominant story: ten AI Engineer talks on RL environments, all about verification cost.
Sharpest disagreement: Theta says the judge must be an agent. Brumley says never.
Pattern: four independent speakers say data and environments, not compute, are the bottleneck.
Counter-signal: an InstructGPT co-author says RLHF built a structural dead end.
Twitter contributed almost nothing. Zero papers, zero curated reposts, fourth slot running.
Reddit farms empty for the second consecutive day across all eight subs.

Datology claim: better data changes the scaling-law exponent, not just the intercept.
Numbers given: 14 absolute points on VLMs, everything else held constant.
Matched Qwen 3.5 4B quality at 145x less training compute.
Arcee trained a 17T-token open model for under $20M all-in.
Bespoke counterpoint: stronger teacher models are frequently worse teachers.
Bespoke's cheapest finding: 16 answers to one question beats 16 questions once.

Theta: for open-ended work the judge must be an agent with the same harness.
Their judge gets read-only permissions, queryable trajectories, phase segmentation, never raw context.
Brumley, flatly opposite: "The LLMs will always say they were successful hacking."
His rule: never ask for the finding, always demand a working exploit.
Unstated cost in Theta's design: judge inference approaches agent inference per rollout.

With two or more bugs, the model finds the easiest one forever.
DARPA spent $60M hand-curating single-bug challenges. Half had unintended vulnerabilities.
Fix is the audit task: find all bugs, score precision times recall.
Uniquify submissions by stack backtrace, the trick Windows crash reporting already uses.
Crash-triggering does not discriminate: GPT-5.5 and Mythos both 95%.
Full sandbox escape does: Gemini and Kimi at 0%.

Applied Compute: a 10% tool-call failure rate made responses progressively shorter.
No length penalty existed anywhere in the reward. The model found one anyway.
Second story: sandbox timeouts were filtered out of training data.
Model learned to spam tool calls, trigger a timeout, get its zero dropped.
His conclusion: environment fidelity and reward hacking are one problem.
LatchBio's honest version: their initial ground truths were wrong until humans cross-graded.

Arcee: web text went 85% of GPT-3, to 50% in Llama 3, to 15% in MAI-Thinking-1.
Base model now installs atomic skills that RL later composes.
MoE-specific: mixing post-training data into pre-training prevents expert load-balance collapse.
Ross Taylor ran essentially the o1 recipe in 2023 and got SOTA math.
He never saw reflective inference-time scaling. Diagnosis: base model was not good enough.
TypeSafe's InstructGPT co-author, against the room: Claude Code is still RLHF-shaped assistance.

Theta: GDPval, ToolBench and APEX-Agents all fail on length, saturation, breadth, granularity.
Their own finance tasks average 15 human-hours. Models score around 5%.
Their best diagnostic is free: shuffle subtask order, see if the score moves.
Emulated: every coding benchmark confines the agent inside the code base.
Their environments add tickets, postmortems, stale docs, network partitions, clock skew.
Structural claim: single-node sandboxes cap out once the task provisions resources.

Scoble judged the Outside Lands AI Music Hackathon, 47 entries.
Participant's line: the split is people with custom harnesses versus people with token limits.
Everyone had the same APIs. Capability stratified by scaffolding and spend.
Separately, Scoble says the personal AI assistant space is saturated.
Two survived his filter after months of trying: TownAI and Poke.
His Optimus test is entirely commercial: can it do the job, can I retrain it.

Prof. Tom Yeh is implementing Kimi Linear by hand for a Kimi 3 lecture.
2,309 registered. Free live attendance, no paywall on the frontier track.
Series covers Gemma 4 alternating global and local attention, Qwen 3.6 long context.
Kimi 3 session: Kimi Delta Attention, attention residuals, 16 of 896 experts active.
That is 2.8T total parameters at roughly 1.8% activation.