Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

To Isolate or to Score? Model-Adaptive Assessment for Cost-Efficient Multi-Agent RAG

About

Multi-agent document assessment for retrieval-augmented generation is computationally expensive, driving practitioners toward smaller, deployable models whose assessment mechanisms remain poorly understood. We conduct a controlled study of training-free interventions on 7B-9B instruction-tuned models across diverse QA benchmarks, revealing a sharp dichotomy in how models benefit from assessment. For weaker baselines, the dominant mechanism is per-document isolation. Astoundingly, assessment-free isolation matches full multi-agent assessment, demonstrating that resolving multi-document context confusion, rather than scoring quality, drives outsized gains of up to 50 percentage points. Conversely, for strong baselines where scoring quality matters, we introduce Reasoning-Score Coupling, a label-free perturbation probe that classifies scoring behavior. Integrating these findings, we propose MADARA, a model-adaptive routing architecture. Crucially, MADARA's diagnostic thresholds derived from a single pilot model generalize zero-shot to four unseen model families, providing a robust, lightweight pipeline to eliminate computational overhead.

Jungseob Lee, Chanjun Park, Heuiseok Lim• 2026

Related benchmarks

TaskDatasetResultRank
Fact VerificationFEVER
Exact Match (EM)93.6
30
Knowledge Conflict ResolutionCONFLICTS
Exact Match (EM)65.4
30
Question AnsweringCONFLICTS
Exact Match62.9
16
Fact VerificationCONFLICTS
Oracle Gap (pp)0.00e+0
12
Fact VerificationFEVER
Oracle Gap (pp)0.00e+0
12
Multi-hop Question AnsweringMuSiQue--
6
Showing 6 of 6 rows

Other info

Follow for update