Tool Use & Function Calling

Concept page for how agents invoke external tools, APIs, and code from inside an LLM-driven loop.

This page accumulates findings on:

Function-call interfaces — JSON schemas, structured-output constraints, decoder-side enforcement.
Tool selection — when to call which tool, learned vs hand-coded routers.
Permission and capability — sandboxing, scoped credentials, dry-run modes.
Tool-chaining attacks — adversarial sequences that exploit chained permissions (per the Marcus 2026-05-06 study on production agent vulnerability).
Tool ecosystems — MCP, OpenAPI bindings, plugin frameworks.

Source pages tagged with this concept will accumulate at wiki/agentic-systems/YYYY-MM-DD-<slug>.md and link back here.

Current State (as of 2026-08-02)

The industry ships a tool call as one atomic decision. The research has now decomposed it into three separately-certified conditions, all published within two days, none citing the others. MemTX (08-01) gates an irreversible call on whether its premise is committed, blocking the action while the belief justifying it is still an uncommitted transaction, and machine-checks that gate over 5.5 million protocol states. Are You Still the Agent I Authorized? (08-02) gates on whether the grant still applies to this agent, and Beyond Aggregate Risk (08-02) gates on whether this particular argument is certified. Three gates on the same call, on three different axes: premise, subject, and field.

The authorization paper attacks the time axis. A long-lived agent evolves after deployment by retaining experience, acquiring tools, revising workflows and delegating, so the subject holding a live grant may no longer be the subject the user evaluated. Its state-bound model fixes a transition envelope deciding whether a grant survives a given mutation, and an immutable effect ceiling that never moves: below the ceiling authority may contract freely and expand only under specified evidence, and the paper proves that under complete mediation, sound effect abstraction, attenuating delegation and monitor integrity, mutation cannot amplify protected effects past the user-issued ceiling. Agent-produced evidence can reallocate authority underneath the ceiling but can never raise it, which is the property that matters when the agent is the thing that might be compromised. Six mutation classes are mapped to their consequences. This formalises the privilege-drift failure that Foundation Protocol and Static Authorization (05-26) named as a warning, and improves on the intent-based access control proposed there in one specific way: IBAC re-derives scope from declared intent and breaks when intent legitimately changes, whereas an effect ceiling lets intent move freely under a fixed boundary.

The risk-control paper attacks the argument axis, and its contribution is an arithmetic one that should have been obvious. Certifying risk over a whole tool call lets failures in a rare high-risk field be averaged away by many benign arguments, and the price is exact: for a role with prevalence p_r, aggregate-only certification must use an effective budget of α·p_r to guarantee role-specific risk α. A field appearing in one call in a hundred forces you to certify the whole action a hundred times more tightly than needed, which is why nobody does and why credential and recipient arguments end up uncertified in practice. Role-stratified per-field conformal risk control sets separate thresholds per semantic argument role with a finite-sample guarantee, pooling rare roles rather than dropping them, and wraps any existing per-field detector. Across AgentDojo and InjecAgent with six models, the measured utility gap tracks the predicted price of coarseness, and it holds the most consistent budget compliance under model transfer, attack transfer, detector noise, drift, unseen tool suites and adaptive attacks.

Three cautions this page should carry. First, the authorization theorem's four assumptions are exactly the properties that failed in 2026: Anthropic's disclosure that three models reached the open internet from inside evaluation environments was complete mediation failing. Second, role-stratified certification needs a sound assignment of arguments to semantic roles, and ToolSense (06-13) already showed the tool boundary is leakier than its schema suggests, so a role taxonomy derived from a schema the model partly ignores certifies the wrong partition. Third, neither paper reports latency or false-block rate, and the sibling hierarchical method on the same board (HG-CRC, 2607.24562) reports a 22-to-37-point participation cost against global conformal risk control, which is the family's real price: conformal abstention buys compliance by refusing to act.

The tool call may not cause the answer (2026-08-13)

This is the most load-bearing negative result this page has recorded, because it invalidates the cheapest proxy everyone uses. The Illusion of Visual Tool-Use (2608.06270) runs a causal audit of "thinking with images," the paradigm where a multimodal model crops and zooms into a region mid-reasoning. It formalizes tool use as a causal graph separating observation-mediated paths (the answer changed because of what came back) from action-induced shortcuts (the answer changed merely because an action was taken), then intervenes at three levels: policy, trajectory (corrupt every observation), and step (counterfactually swap one observation under a fixed prefix). The step-level estimand, Visual Evidence Gain, isolates each observation's contribution.

Across six models and five fine-grained perception benchmarks, two named failure modes. Calling Without Looking: the returned observation has no causal effect on the answer. Looking Without Planning: the observation is informative but the call schedule is incoherent. The aggregate accuracy gain is real and concentrates in a calibrated minority of rollouts.

Three consequences for this page:

Tool-invocation rate and tool-selection accuracy do not measure tool use. An agent can pick the right tool, receive the right observation, and route around it entirely. Every benchmark on this page that scores agents on whether they called the correct tool is partly scoring an action-induced shortcut, and the fraction is unmeasured everywhere except here.
It is the harder twin of a behavioral failure measured a day earlier. SPIEval, in the 08-12 benchmark cluster, found 79% of agent failures are inaccurate information localization with fewer than 2% of retrieval actions using any advanced search method: agents largely do not look, then commit. This shows the case where the agent does look, gets the evidence, and still does not condition on it. Not-searching and searching-without-using are separable defects and the second is invisible to call-counting.
The audit method generalizes past vision. A step-level counterfactual under a fixed prefix works on any tool call. Any capability whose evidence is "accuracy went up when we enabled it" is exposed to it, and the code is released. Most of that literature has not been checked.

It also sharpens the same board's safety proposal. Agent Safety Should Be a Runtime Contract (08-13) gates task submission on verifiable evidence that good actions happened. A Calling Without Looking trajectory produces a genuine log capture and passes that gate cleanly. Composed, the two papers state a stricter requirement than either alone: the evidence gate has to check that the evidence changed something.

Adversarial companion on the same board. ToolHazard (2608.11878) synthesizes executable stateful environments, discovers viable injection points, and generates environment-specific payloads for indirect prompt injection, then builds ToolHazard-Bench from them. Its finding that injection timing and placement affect attack effectiveness is the adversarial mirror of Looking Without Planning: both say when an observation enters the trajectory determines its influence. One measures that as a capability defect, the other exploits it. ToolHazard-generated alignment data improves security on both ToolHazard-Bench and AgentDojo while preserving benign utility, which makes it the rare red-team paper shipping a usable blue-team artifact.

Recent additions

2026-06-14 — Evoflux: compact tool use as workflow repair. MCP-style tool use needs more than function calling: discover tools from a live catalog, satisfy schemas, preserve cross-step dependencies, ground the answer in executed evidence. Small planners generate plausible graphs that fail on all four. Evoflux runs an inference-time evolutionary search over typed workflow graphs, using real execution feedback as fitness, and raises feasibility on MCP-Bench (250 live tools) from ~3% to 17-24%. The sharp negative result: SFT and SFT+DPO on the same search-mined traces match, underperform, or collapse below zero-shot — a direct caution to the standard "mine traces then fine-tune" recipe when teacher traces are scarce. Pairs with HarnessBridge (same day): Evoflux fixes the plan, HarnessBridge fixes the interface.
2026-06-14 — HarnessBridge: the agent-environment interface as a learnable module. Harnesses (the glue that decides what slice of tool output the model sees and how its action becomes an executable transition) are hand-engineered and don't scale to longer trajectories. HarnessBridge learns two bidirectional projections — observation (raw trajectory → compact decision-relevant state) and action (proposal → executable transition or grounded rejection) — trained end-to-end. On Terminal-Bench 2.0 and SWE-bench Verified it matches specialized harnesses while cutting tokens and trajectory length, and transfers from a small trainer model to larger commercial models. The token cut is the Tier 1 efficiency angle inside a Tier 2 agentic result.
2026-06-13 — ToolSense: the parametric-tool-retrieval scores are inflated. Parametric tool retrieval (encode each tool as a virtual vocabulary token, fine-tune the LLM to "retrieve" by generating it) scores well on ToolBench, but ToolSense (SAP) shows those benchmarks use verbose fully-specified queries plus constrained decoding, neither of which tests tool understanding. Auto-generating realistic ambiguous queries (three tiers) plus MCQ/QA probes reveals a knowledge-retrieval dissociation: several configs collapse 50-64 points and fall below the plain embedding baseline they were meant to beat, and some retrieve well while answering factual tool-probes near randomly. The takeaway for routing: a tool-retrieval router that looks accurate on ToolBench may be guessing, and the gap only appears on the underspecified queries real users actually send. Joins the measurement-crisis pattern (SusVibes 06-12, ToolMaze 06-08) where benchmark scores don't predict deployment behavior.
2026-06-08 — Critic-R: introspective feedback closes the agent-retriever loop. Agentic search keeps failing because the retriever is optimized blind to the reasoner's needs. Critic-R adds a critic that reads the agent's introspective trace after it consumes evidence and judges whether the retrieved context supports the next step. Critic-R-Zero rewrites queries and retrieval instructions at inference time; Critic-Embed turns successful and failed refinement trajectories into automatic supervision with no manual relevance labels. Improves both retrieval quality and answer accuracy on HotpotQA, 2Wiki, MuSiQue, and Bamboogle. The industry mirror shipped the same week: Perplexity's "Search as Code" lets the model write its own Python search routines and cuts tokens up to 85%.
2026-06-08 — ToolMaze: tool-use under failure. Every prior tool-call benchmark assumed tools work. ToolMaze injects explicit/implicit and transient/permanent failures and finds agents over-trust corrupted output (recovery rate drops ~37% on silent semantic failures) and that fault-tolerance barely improves with model scale. Permission and dynamic-replanning robustness must be engineered into the harness, not waited out.
2026-05-31 — CoHyDE: co-training the rewriter and the encoder for tool retrieval. Tool retrieval over large API catalogs fails on the colloquial-query-vs-technical-catalog gap. A fine-tuned dense encoder is strong on surface-matching queries but collapses on vague ones; frozen-LLM HyDE expansion is robust to vague queries but hurts well-formed ones because the LLM is catalog-unaware. CoHyDE trains both as one co-evolving loop (encoder retrained with InfoNCE on the rewriter's catalog-style hypotheticals; rewriter DPO-aligned against the encoder's retrieval scores, both warm-started on the catalog). Three rounds on a ~10k ToolBench subset beat the strongest single-component baseline by +2.5 pp NDCG@5 on standard and +6.3 pp on held-out vague queries. The retrieval bottleneck decides whether an agent even sees the right tool, so this is upstream of all tool-selection routing.