MuSR
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Multistep Reasoning | MuSR | Accuracy73.33 | 53 | |
| Reasoning | MuSR 0-shot | Reasoning Score (0-shot)48.82 | 46 | |
| Multistep Soft Reasoning | MuSR | Accuracy69 | 41 | |
| Multistep Reasoning | MUSR | Accuracy61.67 | 31 | |
| Multistep Soft Reasoning | MUSR | Accuracy (Multi-choice)50.77 | 27 | |
| Math & Logic | MUSR | MUSR Performance42.12 | 24 | |
| Multi-step Narrative Reasoning | MUSR | Accuracy65.86 | 22 | |
| Reasoning | MuSR | Accuracy71.89 | 20 | |
| Reasoning | MuSR (test) | Accuracy73.9 | 17 | |
| Multi-hop Reasoning | MuSR | Accuracy43.12 | 10 | |
| Reasoning | MuSR | MuSR Score37.14 | 9 | |
| Self-doubt detection | MuSR 90-trace | AUROC (Self-doubt)83.66 | 7 | |
| Adding Mistake | MuSR | AOC0.731 | 7 | |
| Truncated CoT Answering | MuSR | AOC33.6 | 7 | |
| Multistep Reasoning | MUSR-fr | Average Score33.79 | 6 | |
| Multi-step Reasoning | MuSR | Murder Mystery Score57.36 | 4 | |
| Multi-step Reasoning | MUSR Teams | Accuracy61 | 3 | |
| Multi-step Reasoning | MUSR Objects | Accuracy58 | 3 | |
| Multi-step Reasoning | MUSR Murder | Accuracy68 | 3 | |
| Multi-step reasoning and knowledge retrieval | MuSR (test) | Accuracy0.7867 | 1 |
Showing 20 of 20 rows