Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

ToolChoiceConfusion: Causal Minimal Tool Filtering for Reliable LLM Agents

About

Large language model agents increasingly rely on external tools, but larger tool menus can reduce reliability and efficiency by increasing wrong-tool calls, premature actions, and token cost. Existing tool-selection methods often optimize semantic relevance, exposing tools whose names or descriptions match the user request. We argue that relevance is insufficient: a tool may be related to the task while still being unnecessary or premature at the current step. We propose Causal Minimal Tool Filtering (CMTF), a training-free method that selects tools by causal sufficiency. CMTF uses lightweight precondition-effect contracts to expose only the minimal next-step tool frontier needed to advance from the current state toward the user goal. Across multi-step tool-use tasks, we compare CMTF with all-tools exposure, keyword retrieval, state-aware filtering, and causal-path ablations, measuring task success, wrong-tool calls, premature actions, tool exposure, and token cost. In the main benchmark with 102 tasks, 100 tools, four LLM backends, and 2448 task-method-model runs, CMTF matches the strongest causal baseline in aggregate success while reducing visible tools from 100 to one per step and reducing token usage by about 90% relative to all-tools exposure.

Rahul Suresh Babu, Laxmipriya Ganesh Iyer• 2026

Related benchmarks

TaskDatasetResultRank
Tool-use safety validationsafety-stress set 80-task
Harm Rate (Call)25
14
Goal-conditioned tool useAggregate Downstream Tasks (test)
Success Rate99.5
6
Entity Binding in Tool Use60-task entity binding diagnostic testbed 1.0 (aggregate)
Task Success Rate74.3
6
Tool Use FilteringTool Use Filtering Main Benchmark 102 tasks (test)
Success Rate99
6
Tool Use SecurityRiskGate deterministic agent track
Success Rate100
6
Agentic Tool-useMulti-step tool-calling agent tasks Aggregate over 4 models: Nova 2 Lite, Nova 2 Pro, Claude 3.5 Haiku, Claude Sonnet 4.6 (test)
Success Rate99
5
Showing 6 of 6 rows

Other info

Follow for update