Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

MARCH: Evaluating the Intersection of Ambiguity Interpretation and Multi-hop Inference

About

Real-world multi-hop QA is naturally linked with ambiguity, where a single query can trigger multiple reasoning paths that require independent resolution. Since ambiguity can occur at any stage, models must navigate layered uncertainty throughout the entire reasoning chain. Despite its prevalence in real-world user queries, previous benchmarks have primarily focused on single-hop ambiguity, leaving the complex interaction between multi-step inference and layered ambiguity underexplored. In this paper, we introduce MARCH, a benchmark for their intersection, with 2,209 multi-hop ambiguous questions curated via multi-LLM verification and validated by human annotation with strong agreement. Our experiments reveal that even state-of-the-art models struggle with MARCH, confirming that combining ambiguity resolution with multi-step reasoning is a significant challenge. To address this, we propose CLARION, a two-stage agentic framework that explicitly decouples ambiguity planning from evidence-driven reasoning, significantly outperforms existing approaches, and paves the way for robust reasoning systems.

Jeonghyun Park, Ingeol Baek, Seunghyun Yoon, Haeun Jang, Aparna Garimella, Akriti Jain, Nedim Lipka, Hwanhee Lee• 2025

Related benchmarks

TaskDatasetResultRank
Question AnsweringASQA (test)
Correctness EM Recall92.94
50
Multi-hop Question AnsweringMARCH
Exact Match (EM)38.73
28
Long-form Question AnsweringASQA
str-em91.18
19
Showing 3 of 3 rows

Other info

Follow for update