ToolChoiceConfusion: Causal Minimal Tool Filtering for Reliable LLM Agents
About
Large language model agents increasingly rely on external tools, but larger tool menus can reduce reliability and efficiency by increasing wrong-tool calls, premature actions, and token cost. Existing tool-selection methods often optimize semantic relevance, exposing tools whose names or descriptions match the user request. We argue that relevance is insufficient: a tool may be related to the task while still being unnecessary or premature at the current step. We propose Causal Minimal Tool Filtering (CMTF), a training-free method that selects tools by causal sufficiency. CMTF uses lightweight precondition-effect contracts to expose only the minimal next-step tool frontier needed to advance from the current state toward the user goal. Across multi-step tool-use tasks, we compare CMTF with all-tools exposure, keyword retrieval, state-aware filtering, and causal-path ablations, measuring task success, wrong-tool calls, premature actions, tool exposure, and token cost. In the main benchmark with 102 tasks, 100 tools, four LLM backends, and 2448 task-method-model runs, CMTF matches the strongest causal baseline in aggregate success while reducing visible tools from 100 to one per step and reducing token usage by about 90% relative to all-tools exposure.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Tool-use safety validation | safety-stress set 80-task | Harm Rate (Call)25 | 14 | |
| Goal-conditioned tool use | Aggregate Downstream Tasks (test) | Success Rate99.5 | 6 | |
| Entity Binding in Tool Use | 60-task entity binding diagnostic testbed 1.0 (aggregate) | Task Success Rate74.3 | 6 | |
| Tool Use Filtering | Tool Use Filtering Main Benchmark 102 tasks (test) | Success Rate99 | 6 | |
| Tool Use Security | RiskGate deterministic agent track | Success Rate100 | 6 | |
| Agentic Tool-use | Multi-step tool-calling agent tasks Aggregate over 4 models: Nova 2 Lite, Nova 2 Pro, Claude 3.5 Haiku, Claude Sonnet 4.6 (test) | Success Rate99 | 5 |