Ask When It Pays: Cost-Aware Open-Ended Interaction for Instance Goal Navigation
About
Instance Goal Navigation (IGN) requires an embodied agent to find a specific object instance among distractors from an under-specified natural-language description. Such ambiguity often cannot be resolved from perception and language alone, making interaction with an oracle a natural mechanism for disambiguation. Prior interactive methods allow oracle queries but treat lightweight clarification and route-level guidance alike, letting agents boost success rate through repeated high-information questions rather than by resolving the underlying ambiguity efficiently. We recast interactive IGN as a cost-sensitive uncertainty-reduction problem, where the agent should ask the question whose answer provides the largest reduction in navigation uncertainty relative to its penalty. To this end, we apply an information-gain analysis on existing navigation corpora to identify which cues reduce navigation uncertainty, yielding a compact set of question types and data-derived weights. However, existing interactive navigation benchmarks do not model the cost of different question types or evaluate how efficiently agents use interaction, making them unsuitable for studying cost-sensitive interaction. Based on this taxonomy, we construct a benchmark for diagnosing interaction behavior and efficiency, together with a Weighted Success Rate metric that penalizes each query by its derived cost. We further propose a zero-shot MLLM navigator that selectively queries at each decision step only when the expected uncertainty reduction justifies the interaction cost.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Navigation | REVERIE Unseen (test) | SR63.1 | 51 | |
| Navigation | REVERIE (val unseen) | Success Rate (SR)62.4 | 42 | |
| Vision-and-Language Navigation | SOON (val unseen) | SPL33.5 | 37 | |
| Vision-and-Language Navigation | CVDN (test) | Goal Progress (GP)8.5 | 20 | |
| Vision-and-Language Navigation | SOON Unseen (test) | SR44.8 | 18 | |
| Interactive Graph Navigation | IGN Qwen3.5-8B backbone (evaluation) | SR@1.5 (Easy)39 | 12 | |
| Cooperative Vision-and-Dialog Navigation | CVDN (val) | Goal Progress8.15 | 10 |