Trec
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Question Classification | TREC | Accuracy98.07 | 205 | |
| Text Classification | TREC | Accuracy98 | 179 | |
| Question Classification | TREC (test) | Accuracy97.53 | 124 | |
| Text Classification | TREC (test) | Accuracy97.2 | 113 | |
| Reranking | TREC 2020 (test) | NDCG@1070.9 | 55 | |
| Reranking | TREC | NDCG@5 (DL19)74.45 | 35 | |
| Text Classification | Trec synthetic noise (test) | Accuracy97.2 | 34 | |
| Text Classification | TREC (val) | Top-1 Acc93.54 | 30 | |
| Question Classification | TREC | Spearman's rho (x100)78.72 | 23 | |
| 6-way question classification | TREC 6-class (test) | Accuracy96.1 | 23 | |
| End-to-end Open-Domain Question Answering | TREC (test) | Exact Match (EM)63.1 | 21 | |
| Backdoor Defense | TREC | AUC0.99 | 20 | |
| Information Retrieval | TREC Title queries 1-3 | MAP0.2873 | 19 | |
| Passage retrieval | TREC (test) | Top-20 Accuracy95.5 | 17 | |
| Question Classification | TREC 50 (test) | Accuracy97.2 | 17 | |
| Open-domain QA | Curated TREC | QA-F141.8 | 16 | |
| PII Extraction (Personal Names) | TREC (train) | Unique PII Items Extracted8,071 | 14 | |
| PII Extraction (Phone Numbers) | TREC (train) | Unique PII Count981 | 14 | |
| PII Extraction (Email Addresses) | TREC (train) | Unique PII Count3,840 | 14 | |
| Question Classification | TREC | Attack Success Rate (ASR)0.938 | 13 | |
| Information Retrieval | TREC DL | NDCG@1065.14 | 13 | |
| OOD Detection | TREC-10 | AUROC99.1 | 12 | |
| Open-set selective classification | TREC-10 (test) | AUAC96.6 | 12 | |
| Topic Classification | TREC (test) | Accuracy97.8 | 11 | |
| Patient-to-trial matching | TREC 2021 (test) | Precision@10.71 | 11 |
Showing 25 of 58 rows