LiveCodeBench
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Code Reasoning | LiveCodeBench | Accuracy87.4 | 102 | |
| Code Generation | LiveCodeBench v6 | Accuracy100 | 91 | |
| Code Generation | LiveCodeBench | Pass@190.7 | 89 | |
| Code Generation | LiveCodeBench | Pass@195.8 | 86 | |
| Code Generation | LiveCodeBench | Accuracy88.6 | 84 | |
| Code Generation | LiveCodeBench | Pass@188.1 | 76 | |
| Reasoning | LiveCodeBench | pass@1 Accuracy55.45 | 71 | |
| Code Generation | LiveCodeBench | Average Score168 | 68 | |
| Speculative Decoding | LiveCodeBench | Speedup Factor7.16 | 66 | |
| Code Generation | LiveCodeBench | Accuracy73.2 | 64 | |
| Predicting code correctness | LiveCodeBench Python | ECE0.015 | 60 | |
| Code correctness prediction | LiveCodeBench Python | AUROC86.7 | 60 | |
| Code Correctness Prediction | LiveCodeBench Python | Brier Score0.067 | 60 | |
| Coding | LiveCodeBench v6 | Score (%)92 | 51 | |
| Code Generation | LiveCodeBench | Pass@11,784 | 51 | |
| Coding | LiveCodeBench Aug 24 – Jan 25 | Accuracy50.9 | 45 | |
| Code Generation | LiveCodeBench v6 | Score91.7 | 45 | |
| Programming | LiveCodeBench V3 V4 (test) | Accuracy61.4 | 42 | |
| Code Generation | LiveCodeBench (test) | Pass@1 Overall53.6 | 42 | |
| Coding | LiveCodeBench | Accuracy70 | 40 | |
| Code Generation | LiveCodeBench v5 | Pass@161.5 | 39 | |
| Code Generation | LiveCodeBench (LCB) | Pass@191.7 | 37 | |
| Code Generation | LiveCodeBench | Solve Rate48 | 36 | |
| Code | LiveCodeBench V5-6 | Accuracy50.8 | 33 | |
| Code | LiveCodeBench V1-4 | Accuracy47.1 | 33 |
Showing 25 of 254 rows
...