HLE
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Knowledge-Intensive Reasoning | HLE | Avg Score85 | 75 | |
| Math Reasoning | HLE Math-100 | Pass@135.84 | 68 | |
| Reasoning | HLE | Score64.7 | 65 | |
| Reasoning | HLE | Accuracy (HLE Reasoning)40.8 | 63 | |
| Logical Reasoning | HLE | Accuracy0.7226 | 62 | |
| Long-horizon agentic task | HLE | Performance60 | 41 | |
| Multimodal Reasoning | HLE | Accuracy48.8 | 33 | |
| Mathematical Reasoning | HLE Maths | Accuracy38.2 | 31 | |
| Scientific Reasoning | HLE (test) | Pass@149 | 25 | |
| High-Level Expert Knowledge Evaluation | HLE Gold 149 | Accuracy (Bio)80.5 | 25 | |
| HLE | HLE | Accuracy67.1 | 25 | |
| Hard Reasoning and Language Evaluation | HLE | Accuracy54 | 25 | |
| Humanities Question Answering | HLE | HLE Score13.37 | 24 | |
| General Reasoning | HLE | Accuracy38.4 | 21 | |
| General and STEM reasoning | HLE | Pass@18.12 | 20 | |
| Reasoning | HLE | Head-to-head Win %100 | 20 | |
| Scientific Reasoning | HLE | pass@1612 | 17 | |
| High-Level Reasoning | HLE | Average Score26.6 | 17 | |
| Knowledge Question Answering | HLE | Accuracy4.36 | 16 | |
| Reasoning | HLE | Accuracy50.2 | 16 | |
| Mathematical reasoning | HLE math | Accuracy23.3 | 16 | |
| Deep research | HLE | Accuracy51 | 16 | |
| Deep search | hle | Accuracy11.4 | 16 | |
| Textual long-horizon complex reasoning | HLE Text | Score62 | 15 | |
| Web Agent Task Completion | HLE | Accuracy37.6 | 15 |
Showing 25 of 85 rows