Open LLM Leaderboard
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Leaderboard Evaluation | Open LLM Leaderboard 2 | Overall Score51.28 | 55 | |
| General Language Understanding and Reasoning | Open LLM Leaderboard Population (Top-50) | Accuracy60.08 | 50 | |
| Language Model Evaluation | Open LLM Leaderboard v2 (test) | BBH60.84 | 47 | |
| Leaderboard Evaluation | Open LLM Leaderboard 1 | Overall Score69.28 | 46 | |
| Large Language Model Evaluation | Open LLM Leaderboard | Average Score74.2 | 41 | |
| Unified Multi-task Language Understanding and Instruction Following | Open LLM Leaderboard v1 (test) | MMLU-P Accuracy19.84 | 37 | |
| Large Language Model Evaluation | Open LLM Leaderboard v1 (test) | Average Score69.6 | 34 | |
| Language Modeling and Reasoning | Open LLM Leaderboard | ARC82.8 | 33 | |
| Open-style response generation | Open-LLM Leaderboard | Accuracy53.45 | 28 | |
| Performance Estimation | Open LLM Leaderboard subset-selection | MAE1.4 | 24 | |
| General LLM Evaluation | Open LLM Leaderboard (test) | Average Score70.1 | 21 | |
| General language understanding and reasoning | Open LLM Leaderboard v1 (test) | MMLU-Pro44.97 | 20 | |
| General Language Understanding and Reasoning | Open LLM Leaderboard Lighteval (test) | Mean Accuracy91.07 | 17 | |
| Language Modeling | Open LLM Leaderboard & General Ability Benchmarks (MMLU-P, GPQA, BBH, MATH, MuSR, IFEval, ARC, Hellaswag, PIQA, BoolQ, WinoGrande, COPA, OpenBookQA, SciQ) unified (test) | MMLU-P Accuracy12 | 16 | |
| General Language Capabilities | Open LLM Leaderboard lm-eval-harness (test) | HellaSwag Accuracy83.29 | 14 | |
| Language Modeling Evaluation | Open LLM Leaderboard | ARC70.22 | 14 | |
| Natural Language Understanding | Open LLM Leaderboard (test) | ARC57.94 | 13 | |
| General Language Understanding | Open LLM Leaderboard HuggingFace 2023a (test) | ARC-c Accuracy (25-shot)59.4 | 12 | |
| Ranking quality gain estimation | Open LLM Leaderboard v2 | Ranking Quality Gain21.04 | 9 | |
| General Language Evaluation | Open LLM Leaderboard 1 | Overall Score66.12 | 9 | |
| General Language Understanding | Open LLM Leaderboard (test) | ARC62.03 | 9 | |
| Reasoning and Language Understanding | Open LLM Leaderboard MMLU-PRO, IFEval, BBH, GPQA, MATH, GSM8K, ARC v0.4.0 (test) | MMLU-PRO28.38 | 7 | |
| General Language Understanding | Open LLM leaderboard | Average Score65.51 | 7 | |
| Downstream Language Understanding | Open LLM Leaderboard zero-shot | ARCE52.9 | 6 | |
| LLM Selection | Open LLM Leaderboard Scientific Discovery scenario K = 42 models | Main Objective64 | 4 |
Showing 25 of 29 rows