HumanCLAW: Can Vision-Language Models Act Through a Body?
arxiv: 2607.27180 · Source: HuggingFace Daily Papers 2026-07-30
TL;DR
When an embodied agent fails a task, you usually cannot tell whether the vision-language model chose the wrong action or the motor controller failed to execute a reasonable one. HumanCLAW removes the second possibility by construction. At each step the VLM issues an atomic skill command, and that command is translated into a sub-second chunk of continuous full-body motion with real physical consequences including gravity and collisions. The body still acts in a physical world, but balance failures and motor error are factored out of the measurement. What remains is the model's action intelligence: its moment-to-moment choice of what the body should do next. HumanCLAW-Bench is 1,218 long-horizon egocentric find-navigate-interact episodes across 41 indoor scenes. Nine state-of-the-art VLMs were tested and none solves it; the best reaches 16.8% success.
The diagnosis is the interesting part. Recognizing the target object is not the bottleneck. What the models lack is embodied self-awareness: they lose track of their own body, and cannot reliably tell where it is, whether it has reached the goal, or whether it has hit an obstacle.
Why the decoupling matters more than the score
Embodied benchmarks conflate two very different research problems, and conflating them means neither gets a clean signal. Robotics teams read a low score as a controller problem; VLM teams read it as a perception problem; both keep working. HumanCLAW's harness makes the attribution unambiguous, and the answer it returns is neither of the usual two. Perception works. Control is factored out. The failure is in state tracking of the self, which is a representation problem sitting between the two fields and owned by neither.
That failure mode is worth naming precisely because it is not about the world. These models can describe a room accurately and still not know where they are standing in it. Egocentric observation gives you the world from a viewpoint without giving you the viewpoint as an object you can reason about.
Relation to prior wiki state
This is the embodied instance of a pattern the wiki has now recorded in three unrelated domains on a single day, which crosses the threshold for calling it a pattern.
- Shadow evaluations (2026-07-30) found research agents completing all engineering on an open question while failing the research, with poor resource awareness and instruction drift among five named failure modes. The agent does not track its own progress against its own goal.
- SecRespond (2026-07-30) found incident-response agents reliably investigating what the alerts pointed at and never forming the hypothesis that something unflagged was present. The agent does not track what it has and has not covered.
- HumanCLAW finds VLMs that recognize the target and lose track of their own body.
Three domains, one deficit: the model represents the task and the world well and represents its own state within them badly. That is a sharper statement than the generic "long-horizon agents are weak" finding the agent-benchmarks page has accumulated since CEO-Bench (2026-06-18), where most frontier models went bankrupt running a 500-day business, and RNG-Bench, the non-Markov memory benchmark where the answer depends on history not visible in the current observation. RNG-Bench is the closest prior: it tests memory of invisible history, HumanCLAW tests memory of invisible self. Both are state the observation does not contain.
The obvious methodological neighbour is MAP / Map-then-Act (2026-05-14), where frontier models beat near-zero ARC-AGI-3 baselines in 22 of 25 environments when made to build an environment model before acting. HumanCLAW says the missing map may not be of the environment at all.
On the efficiency side, today's TurboVLA (2026-07-30) reaches 97.7% on LIBERO with a 0.2B model that removes the LLM from the vision-language-action pathway entirely. Read together the two papers bracket the field honestly: short-horizon manipulation is close to saturated and can be done for under a gigabyte of VRAM, while long-horizon embodied decision-making sits at 16.8% with nine frontier VLMs.
Gaps
Simulation, with all the usual caveats about how well sub-second motion chunks with gravity and collisions stand in for a real body. The atomic-skill vocabulary is the benchmark's most consequential design choice and its size and composition determine the achievable ceiling, so a low score is partly a statement about that vocabulary. All 41 scenes are indoor. The embodied-self-awareness diagnosis is drawn from failure analysis rather than from a dedicated probe, so it is a well-supported interpretation rather than a measured quantity; a targeted eval that asks the model directly where its body is would convert it into one.
Industrial implication
For anyone building humanoid or mobile manipulation on top of an off-the-shelf VLM, this says the integration work to prioritize is not better perception and not a better controller, it is an explicit proprioceptive state channel fed back into the model's context. That is cheap to build relative to either alternative and no current VLM API exposes a place to put it, which is a gap in the tooling rather than in the models.