MRCR
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Long Context | MRCR | Score89.7 | 25 | |
| Long Context Understanding | MRCR | Accuracy75.3 | 15 | |
| Long-context reasoning | MRCR OOD 8-128K | Accuracy (8-16K)96.6 | 10 | |
| Long-context reasoning | MRCR ID 4-8K | Accuracy99.8 | 10 | |
| Multi-needle Retrieval | MRCR | MRCR 2-Needle Success38 | 9 | |
| Long-context & Multi-turn Dialogue | MRCR 16K-256K | Score80.37 | 8 | |
| Multi-round coreference resolution | MRCR | 2-Needle Score81.6 | 8 | |
| Long Context Retrieval | MRCR v2 | Accuracy66.4 | 6 | |
| Complex retrieval and positional sorting | MRCR 512K~1M | Score46.88 | 6 | |
| Complex retrieval and positional sorting | MRCR 128K~512K | Score53.98 | 6 | |
| Multi-Round Chat Retrieval | MRCR | Accuracy92.3 | 6 | |
| Long Context Performance | MRCR v2 | Score89.4 | 5 | |
| Reasoning | MRCR v2 | Score94.8 | 5 | |
| Multi-round Code Repair | MRCR | pass@10.77 | 5 | |
| Long Context Performance | MRCR 8-needle v2 | Score79 | 4 | |
| Multi-needle retrieval | MRCR-8needle 100-example | Sequence Match Accuracy27.9 | 4 |
Showing 16 of 16 rows