Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Reasoning about Intent for Ambiguous Requests

About

Large language models often respond to ambiguous requests by implicitly committing to one interpretation, frustrating users and creating safety risks when that interpretation is wrong. We propose generating a single structured response that enumerates the different ways an ambiguous request can be interpreted, each coupled with a corresponding answer. Our models are trained with reinforcement learning using a dual reward objective: recall on ambiguous inputs to maximise coverage of valid interpretations, and precision on unambiguous ones to suppress spurious alternatives. Training requires only multiple valid answers per input as supervision, no clarification questions or explicit interpretations are needed. Experiments on conversational question answering and semantic parsing demonstrate that our method achieves higher coverage of valid answers than baseline approaches. Human evaluation confirms that predicted interpretations are meaningful and explain their corresponding answers. Our approach promotes transparency with explicit interpretations, achieves efficiency by requiring only one generation step, and supports downstream applications through its structured output format.

Irina Saparina, Mirella Lapata• 2025

Related benchmarks

TaskDatasetResultRank
Text-to-SQL ParsingAmbrosia Ambiguous subset (test)
Recall82.4
11
Text-to-SQL ParsingAmbrosia Unambiguous (test)
Recall88.7
11
Conversational Question AnsweringAbg-CoQA Ambiguous
Overlap F172.9
10
Conversational Question AnsweringAbg-CoQA Unambiguous
Overlap F184.4
10
Semantic SimilarityAbg-CoQA
Similarity83
2
Text-to-SQL ambiguity resolutionAmbrosia Ambiguous
Recall82.4
2
Text-to-SQL ambiguity resolutionAmbiQT
Recall66.9
2
Interpretation AlignmentAbg-CoQA (sampled 30 ambiguous examples)
Alignment90
1
Interpretation AlignmentAmbrosia sampled 30 ambiguous examples
Alignment Score0.917
1
Showing 9 of 9 rows

Other info

Follow for update