RAGFort: Dual-Path Defense Against Proprietary Knowledge Base Extraction in Retrieval-Augmented Generation
About
Retrieval-Augmented Generation (RAG) systems deployed over proprietary knowledge bases face growing threats from reconstruction attacks that aggregate model responses to replicate knowledge bases. Such attacks exploit both intra-class and inter-class paths, progressively extracting fine-grained knowledge within topics and diffusing it across semantically related ones, thereby enabling comprehensive extraction of the original knowledge base. However, existing defenses target only one path, leaving the other unprotected. We conduct a systematic exploration to assess the impact of protecting each path independently and find that joint protection is essential for effective defense. Based on this, we propose RAGFort, a structure-aware dual-module defense combining "contrastive reindexing" for inter-class isolation and "constrained cascade generation" for intra-class protection. Experiments across security, performance, and robustness confirm that RAGFort significantly reduces reconstruction success while preserving answer quality, offering comprehensive defense against knowledge base extraction attacks.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Knowledge Base Extraction | Mini-Wikipedia RAG-Thief attack | CRR37 | 5 | |
| Knowledge Base Extraction | Mini-BioASQ Pirates attack | CRR53.8 | 5 | |
| Knowledge Base Extraction | Mini-BioASQ RAG-Thief attack | CRR38 | 5 | |
| Knowledge Base Extraction | ChatDoctor Pirates attack | CRR50.2 | 5 | |
| Knowledge Base Extraction | ChatDoctor RAG-Thief attack | CRR37.8 | 5 | |
| Knowledge Base Extraction | Mini-Wikipedia (Pirates attack) | CRR42.4 | 5 | |
| Knowledge Base Extraction | Multi-Agent Aggregate ChatDoctor Mini-Wikipedia Mini-BioASQ | Relative Mean CRR0.6 | 5 | |
| Privacy-Utility Trade-off Evaluation | ChatDoctor worst-case attack setting | CRR50.2 | 5 |