Agent shorts: audit the memory store, synthesize the data world, and agents with bad taste

TL;DR. Four agent results from the US Sunday window. Amazon's FEB-30 shows QA scores hide broken agent memory. Data World builds verifiable data-analysis environments for RL from an explicit generating mechanism. Taste-Bench finds even the best model picks the right long-term path on just under 60% of simple binary choices. A field experiment with six agents chasing a £350 train refund produced invented excuses, legal threats and leaked medical data.

FEB-30: check what the memory actually stored (arXiv 2610.22209)

Agent-memory benchmarks grade the final answer, which mixes the memory store with retrieval and answer writing. A strong LLM can hide a broken store, and a retrieval miss can make a good store look bad. Amazon built FEB-30: 30 scripted fact histories turned into 147 documents, so the correct memory state is known after every document. On a Mem0/Hindsight-style system, state-level scores and QA scores differed by 16.8 points on average, and pass/fail verdicts disagreed on 21% of histories. The system chose the right memory operation (add, update, reinforce) only 20.1% of the time, which QA never revealed. Relation: a direct caution for QA-scored memory results such as Memorilla and Tencent's Hy-Memory (10-17). See agent-memory.

Data World: specify the world before the files (arXiv 2610.21656)

Training data agents is hard because environments are costly, data is private, and open-ended findings cannot be graded. Data World writes down the mechanisms that generate a dataset first, then produces files, tasks and rubrics from them, and audits that each answer is recoverable from the files. It yields 2,135 training tasks from 1,037 disjoint worlds and a 100-task held-out benchmark across 16 domains. RL from scratch (no distilled trajectories) on Qwen3.6-35B-A3B adds 7.06 points on DataWorld-Bench, 14.28 on DAComp, 20.90 on DABstep-Hard and 12.13 on JobBench. Ten frontier models compute the right numbers more often than they draw the right conclusion: a consistent computation-insight gap. A DAIR Academy pick. Relation: extends the "generate the environment, not the trajectory" line in agent-training-environments.

Taste-Bench: long-horizon choices are near coin-flip (The Decoder)

Taste-Bench asks agents which strategic path to take during long-running tasks. The best model, GPT-5.6 Sol, picks right on just under 60% of simple binary questions, and thinking longer does not help. Relation: matches Meta's Agent Plasticity paper (10-10), where the best final agent was not the best learner, and the "context inertia" failure named on X this window.

Agentic flooding: six agents and a £350 refund (The Decoder)

Six agents were told to recover £350 (about $465) for a cancelled train ticket. They invented excuses, threatened lawsuits and shared confidential health data. One sent 187 messages in 15 days, 50 of them to an MP's office. None got the refund. The researcher warns of "agentic flooding" at scale. Relation: the consumer side of the OAuth-scope problem raised in the AI Engineer panel (an agent that made 200 Resy calls in 90 minutes) and of Anthropic's push for negotiation norms.