Ranking Free RAG: Replacing Re-ranking with Selection in RAG for Sensitive Domains
About
Retrieval-Augmented Generation (RAG) systems deployed in sensitive domains must provide interpretable evidence selection and robust safeguards against data poisoning, yet current approaches rely on opaque similarity-based retrieval with arbitrary top-k cutoffs that offer no explanation for their selections and remain vulnerable to adversarial manipulation. METEORA replaces re-ranking with rationale-driven selection via three components: a DPO-tuned LLM that generates explicit retrieval rationales, an Evidence Chunk Selection Engine (ECSE) that uses those rationales with statistical elbow detection for adaptive cutoff determination, and a Verifier LLM that filters poisoned evidence using the same rationales. Across six datasets, METEORA achieves 13.41% higher recall, 21.05% higher precision (without expansion), an 80% reduction in evidence volume, a 33.34% improvement in answer accuracy, and a 4.4x improvement in adversarial robustness. Human evaluation confirms genuine interpretability (3.64/5 confidence; 86% ground-truth agreement), demonstrating that interpretability, efficiency, and robustness are synergistic rather than competing objectives. The code is available in the GitHub repository https://github.com/YashSaxena21/METEORA
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Context Prioritization | Qasper | Recall@8100 | 12 | |
| Context Prioritization | Contract-NLI | Recall@3100 | 12 | |
| Context Prioritization | PrivacyQA | Recall@6100 | 12 | |
| Context Prioritization | CUAD | R@1294 | 12 | |
| Context Prioritization | MAUD | R@3375 | 12 | |
| Context Prioritization | Average QASPER, Contract-NLI, FinQA, PrivacyQA, CUAD, MAUD | Recall94 | 12 | |
| Context Pruning | FinQA | P@1312 | 12 | |
| Context Prioritization | FinQA | Recall@1097 | 12 | |
| Corpus poisoning detection | PrivacyQA | Precision60 | 6 | |
| Corpus poisoning detection | CUAD | Precision25 | 6 |