FRAMES
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Multi-hop Question Answering | FRAMES | Accuracy86 | 46 | |
| Error Detection | FRAMES (test) | Precision97 | 36 | |
| Error Detection | FRAMES | F1 Score95 | 36 | |
| Open-domain long-horizon evaluation | FRAMES (test) | Accuracy23.91 | 35 | |
| Question Answering | FRAMES (test) | Unreliability Rate41.1 | 28 | |
| Long-context reasoning | FRAMES | Score84.7 | 27 | |
| Multi-hop Question Answering | Frames | ACCE41.38 | 24 | |
| Long-context Question Answering | FRAMES | Avg@4 Score73.54 | 22 | |
| Deep search QA | Frames | Accuracy46.42 | 16 | |
| Agentic Search | Frames | String-F136.6 | 14 | |
| Deep Research | FRAMES | Accuracy56 | 14 | |
| Question Answering | FRAMES | Accuracy82.5 | 14 | |
| Document-level retrieval | FRAMES (test) | Recall73.3 | 13 | |
| Document Question Answering | FRAMES | EM10.5 | 13 | |
| Multi-hop Reasoning and Fact-checking | FRAMES | Average @390.6 | 13 | |
| Complex Reasoning | Frames | Accuracy90.6 | 13 | |
| Information Retrieval | FRAMES | Recall81.5 | 11 | |
| Question Answering | FRAMES out-domain (test) | LasJ31.31 | 11 | |
| Multi-hop QA | FRAMES Wiki18 index (test) | Pass@437.5 | 10 | |
| Long-context Question Answering | FRAMES C=12k | Accuracy34.95 | 10 | |
| Multi-hop Factual Reasoning | FRAMES | Accuracy82.3 | 10 | |
| Task-oriented Dialogue | Frames | Success Rate (SR)50.57 | 9 | |
| Fact Retrieval and Analysis | FRAMES | Accuracy90.6 | 9 | |
| Agentic Reasoning | FRAMES n=50 (full) | Accuracy77.31 | 8 | |
| Multi-step Reasoning and Factuality | FRAMES | Pass@190.6 | 7 |
Showing 25 of 44 rows