To Isolate or to Score? Model-Adaptive Assessment for Cost-Efficient Multi-Agent RAG
About
Multi-agent document assessment for retrieval-augmented generation is computationally expensive, driving practitioners toward smaller, deployable models whose assessment mechanisms remain poorly understood. We conduct a controlled study of training-free interventions on 7B-9B instruction-tuned models across diverse QA benchmarks, revealing a sharp dichotomy in how models benefit from assessment. For weaker baselines, the dominant mechanism is per-document isolation. Astoundingly, assessment-free isolation matches full multi-agent assessment, demonstrating that resolving multi-document context confusion, rather than scoring quality, drives outsized gains of up to 50 percentage points. Conversely, for strong baselines where scoring quality matters, we introduce Reasoning-Score Coupling, a label-free perturbation probe that classifies scoring behavior. Integrating these findings, we propose MADARA, a model-adaptive routing architecture. Crucially, MADARA's diagnostic thresholds derived from a single pilot model generalize zero-shot to four unseen model families, providing a robust, lightweight pipeline to eliminate computational overhead.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Fact Verification | FEVER | Exact Match (EM)93.6 | 30 | |
| Knowledge Conflict Resolution | CONFLICTS | Exact Match (EM)65.4 | 30 | |
| Question Answering | CONFLICTS | Exact Match62.9 | 16 | |
| Fact Verification | CONFLICTS | Oracle Gap (pp)0.00e+0 | 12 | |
| Fact Verification | FEVER | Oracle Gap (pp)0.00e+0 | 12 | |
| Multi-hop Question Answering | MuSiQue | -- | 6 |