StrategyQA
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Commonsense Reasoning | StrategyQA | Accuracy95.7 | 174 | |
| Question Answering | StrategyQA | Accuracy94.4 | 114 | |
| Commonsense Reasoning | StrategyQA (test) | Accuracy83.49 | 81 | |
| Logical Reasoning | StrategyQA | Accuracy89 | 58 | |
| Reasoning | StrategyQA | Accuracy83.5 | 40 | |
| Multi-hop Reasoning | StrategyQA | Accuracy95.6 | 36 | |
| Question Answering | StrategyQA | Exact Match (EM)85.59 | 35 | |
| Question Answering | StrategyQA | EM80.1 | 35 | |
| Reasoning | StrategyQA (test) | Factuality Acc100 | 28 | |
| Question Answering | StrategyQA (test) | Task Accuracy83 | 28 | |
| Question Answering | StrategyQA | Accuracy90.4 | 26 | |
| Multi-hop Question Answering | StrategyQA (test) | Accuracy77.12 | 26 | |
| Calibration | StrategyQA | ECE0.285 | 24 | |
| Question Answering | STRATEGYQA | Accuracy61.8 | 24 | |
| Knowledge-intensive QA | StrategyQA | ACC66.7 | 24 | |
| Question Answering | StrategyQA | EM89.34 | 21 | |
| Multi-hop QA | StrategyQA (SQA) | Cover-EM76.95 | 20 | |
| Question Answering | StrategyQA | Exact Match (EM)82 | 16 | |
| Strategy-based Question Answering | StrategyQA | Verifiability69.11 | 16 | |
| Multiple Choice Classification | StrategyQA | Accuracy83.4 | 16 | |
| Reasoning quality evaluation | STRATEGYQA | Somers' D0.2735 | 15 | |
| Reasoning Question Answering | StrategyQA | Accuracy80 | 14 | |
| Strategic Question Answering | StrategyQA | Reusability Score51.57 | 12 | |
| Question Answering | StrategyQA | Prefilling Speedup Ratio3.39 | 12 | |
| Follow-up Questioning Consistency | StrategyQA (unseen) | Average Success Count (M.)42.65 | 12 |
Showing 25 of 46 rows