WikiMIA
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Membership Inference Attack | WikiMIA length 64 | AUC0.9556 | 84 | |
| Membership Inference | WikiMIA 32 tokens 1.0 | ROC AUC99.7 | 66 | |
| Membership Inference Attack | WikiMIA length 128 | AUC98.84 | 56 | |
| Membership Inference Attack | WikiMIA length 32 | TPR @ 5% FPR0.5964 | 54 | |
| Membership Inference Attack | WikiMIA length 128 (test) | TPR @ 5% FPR96.72 | 49 | |
| Membership Inference Attack | WikiMIA length 64 (test) | TPR@5%FPR73.77 | 49 | |
| Pre-training data detection | WikiMIA (test) | AUROC96 | 30 | |
| Membership Inference Attack | WikiMIA length 128 | AUC0.874 | 28 | |
| Membership Inference Attack | WikiMIA length 256 | AUC (MIA)0.8 | 24 | |
| Membership Inference Attack | WikiMIA | AUC71 | 24 | |
| Contamination Detection | WikiMIA | AUC74.3 | 23 | |
| Membership Inference | WikiMIA (test) | AUC0.9998 | 21 | |
| Pre-training Data Detection | WikiMIA Original Length 32 | AUROC85.9 | 21 | |
| Membership Inference Attack | WikiMIA | TPR@5%FPR19.6 | 16 | |
| Training Data Identification | WikiMIA alpha=0.3 1.0 (test) | Perplexity2.9 | 15 | |
| Training Data Identification | WikiMIA alpha=0.2 1.0 (test) | Power (Perplexity)1.04 | 15 | |
| Training Data Identification | WikiMIA alpha=0.1 1.0 (test) | Power Perplexity0.21 | 15 | |
| Membership Inference | WikiMIA 128 tokens 1.0 | ROC AUC94.8 | 11 | |
| Membership Inference | WikiMIA 64 tokens 1.0 | ROC AUC97.9 | 11 | |
| Membership Inference Attack | WikiMIA Hard (Paraphrased) 2024 v1 (test) | AUROC58.4 | 7 | |
| Membership Inference Attack | WikiMIA Hard Original 2024 v1 (test) | AUROC60.6 | 7 | |
| Pre-training Data Detection | WikiMIA Paraphrased Length 128 | AUROC73.3 | 2 | |
| Pre-training Data Detection | WikiMIA Original Length 128 | AUROC75.9 | 2 | |
| Pre-training Data Detection | WikiMIA Paraphrased Length 64 | AUROC0.706 | 2 | |
| Pre-training Data Detection | WikiMIA Original Length 64 | AUROC74.8 | 2 |
Showing 25 of 26 rows