Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Uncertainty-Aware Clarification in LLM Agents with Information Gain

About

Large Language Model (LLM) agents often operate under underspecified user instructions, where latent uncertainty over user intent leads to erroneous tool actions. To address this challenge, we propose a goal-oriented clarification framework that aligns clarification behavior with ambiguity resolution. Central to our approach is the Information Gain Reward, a metric that quantifies the utility of clarification questions by measuring the Bayesian belief update towards the ground-truth goal induced by the clarification exchange. We train the clarifier (LLM) using this reward to optimize for high information gain, ensuring that clarifications effectively reduce uncertainty and improve task completion within the agent-tool-user environment. We validate our framework within a clarification-enhanced $\tau$-Bench environment, conducting cross-agent evaluations across five heterogeneous backbones. Empirical results demonstrate that our method consistently improves the success rate by 3.7\% over the no-clarification baseline, while adding only 0.3 total interaction steps on average.

Mengyi Deng, Zhiwei Li, Xin Li, Tingyu Zhu, Ying Zhao, Zhijiang Guo, Wei Wang• 2026

Related benchmarks

TaskDatasetResultRank
Agent Task Completionτ-Bench Retail
Success Rate37.4
31
Agentic Task Completionτ2-bench Airline
Success Rate46
22
Agentic Task Completionτ-bench Average
Success Rate40
18
Agent Task Completionτ-Bench Airline
Success Rate17.3
8
LLM Agent Task Completionτ-Bench Average across Retail and Airline
Success Rate (%)17.8
8
Showing 5 of 5 rows

Other info

Follow for update