Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Knowing When to Ask: Self-Gated Clarification for Hierarchical Language Agents

About

In hierarchical reasoning, failures often originate at intermediate decision points where the agent commits to a wrong branch without recognizing that it lacks critical information. Rather than treating clarification as an external uncertainty trigger, we propose ACTION-RATING, a formulation that places it inside the agent's action space on a shared ordinal scale with navigation, so that asking competes directly with acting at every decision point and help-seeking becomes observable at intermediate states. Two structurally distinct information-seeking modes emerge from the agent's own ratings: mandatory (no viable branch) and opportunistic (residual uncertainty despite a leading candidate). On Harmonized Tariff Schedule classification (30,000-node taxonomy, three benchmarks, 9~LLMs across 4 families), we observe a regime shift from mandatory to opportunistic clarification, with Information-Seeking Effectiveness (ISE), a local diagnostic defined as the fraction of help interactions followed by a correct next navigation step (not a final-task metric), rising from 50% to 74%. Three diagnostic contrasts fail to reproduce this structure. A separability test shows that the information-seeking pattern (mode split, ISE ranking) persists when answer quality is degraded (-18.8% accuracy), supporting an empirical separation between where an agent seeks help and the quality of the help it receives. Under the controlled answer channel, accuracy gains reach +16.2% at 10-digit; we read this as an upper bound on what better localization could unlock, not a deployment estimate.

Aijing Gao, Yiming Kang, Mengdie Flora Wang, Jae Oh Woo• 2026

Related benchmarks

TaskDatasetResultRank
Hierarchical classificationHTS CBP-NY (N=1,181)
Success Rate97.8
12
HTS classificationHSCodeComp
6-digit Hierarchical Accuracy76.6
4
HTS classificationATLAS (test)
Accuracy (6-digit)79.5
2
Showing 3 of 3 rows

Other info

Follow for update