Reasoning about Intent for Ambiguous Requests
About
Large language models often respond to ambiguous requests by implicitly committing to one interpretation, frustrating users and creating safety risks when that interpretation is wrong. We propose generating a single structured response that enumerates the different ways an ambiguous request can be interpreted, each coupled with a corresponding answer. Our models are trained with reinforcement learning using a dual reward objective: recall on ambiguous inputs to maximise coverage of valid interpretations, and precision on unambiguous ones to suppress spurious alternatives. Training requires only multiple valid answers per input as supervision, no clarification questions or explicit interpretations are needed. Experiments on conversational question answering and semantic parsing demonstrate that our method achieves higher coverage of valid answers than baseline approaches. Human evaluation confirms that predicted interpretations are meaningful and explain their corresponding answers. Our approach promotes transparency with explicit interpretations, achieves efficiency by requiring only one generation step, and supports downstream applications through its structured output format.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Text-to-SQL Parsing | Ambrosia Ambiguous subset (test) | Recall82.4 | 11 | |
| Text-to-SQL Parsing | Ambrosia Unambiguous (test) | Recall88.7 | 11 | |
| Conversational Question Answering | Abg-CoQA Ambiguous | Overlap F172.9 | 10 | |
| Conversational Question Answering | Abg-CoQA Unambiguous | Overlap F184.4 | 10 | |
| Semantic Similarity | Abg-CoQA | Similarity83 | 2 | |
| Text-to-SQL ambiguity resolution | Ambrosia Ambiguous | Recall82.4 | 2 | |
| Text-to-SQL ambiguity resolution | AmbiQT | Recall66.9 | 2 | |
| Interpretation Alignment | Abg-CoQA (sampled 30 ambiguous examples) | Alignment90 | 1 | |
| Interpretation Alignment | Ambrosia sampled 30 ambiguous examples | Alignment Score0.917 | 1 |