Code Generation on LiveCodeBench (test)
53.6Pass@1 Overall
o1-preview
Updated 4d ago
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
2025.01 | 53.6 | - | - | - | - | - | |
2025.01 | 51.6 | - | - | - | - | - | |
2025.01 | 34.8 | 57.7 | 32.4 | 24.5 | - | - | |
2025.02 | 33.9 | 73.5 | 24.2 | 4.8 | - | - | |
2025.01 | 33.4 | - | - | - | - | - | |
2025.01 | 33 | 61.5 | 29.7 | 20.4 | - | - | |
2025.01 | 33 | 53.8 | 29.7 | 24.5 | - | - | |
2025.01 | 33 | 57.7 | 32.4 | 20.4 | - | - | |
2025.02 | 32.5 | 70.6 | 23.9 | 3.7 | - | - | |
2025.02 | 31.2 | 69.2 | 22 | 3.4 | - | - | |
2025.02 | 31 | 68.5 | 23.3 | 1.9 | - | - | |
2025.02 | 29.5 | 67 | 20.8 | 1.5 | - | - | |
2025.01 | 26.8 | 65.4 | 18.9 | 12.2 | - | - | |
2025.02 | 26.4 | 59.5 | 17.5 | 3 | - | - | |
2025.01 | 25.9 | 61.5 | 24.3 | 8.2 | - | - | |
2025.02 | 25.3 | 58.4 | 16.3 | 2.2 | - | - | |
2025.02 | 25.1 | 58.4 | 15.7 | 2.2 | - | - | |
2025.01 | 25 | 61.5 | 16.2 | 12.2 | - | - | |
2025.02 | 24.4 | 57.7 | 15.1 | 1.5 | - | - | |
2025.01 | 24.1 | 57.7 | 16.2 | 12.2 | - | - | |
2025.01 | 24.1 | 57.7 | 24.3 | 6.1 | - | - | |
2025.02 | 24.1 | 56.7 | 15 | 1.7 | - | - | |
2025.01 | 22.3 | 42.3 | 18.9 | 14.3 | - | - | |
2025.02 | 21.3 | 51.3 | 11.8 | 1.9 | - | - | |
2025.02 | 20.9 | 50.5 | 11.8 | 1.5 | - | - | |
2025.02 | 20.1 | 49.8 | 11.2 | 0.4 | - | - | |
2025.02 | 19.8 | 49.1 | 10.9 | 0.4 | - | - | |
2025.02 | 19.7 | 47.7 | 10.6 | 1.9 | - | - | |
2025.02 | 19.3 | 48.1 | 10.7 | 0.1 | - | - | |
2025.02 | 19.2 | 47 | 10 | 1.9 | - | - | |
2025.02 | 18.4 | 46.2 | 9.4 | 0.7 | - | - | |
2025.02 | 18.3 | 46.2 | 9.4 | 0.4 | - | - | |
2025.02 | 18.1 | 45.3 | 9.1 | 0.9 | - | - | |
2025.02 | 16.9 | 42.3 | 8.5 | 1.1 | - | - | |
2025.02 | 16.6 | 41.9 | 8.2 | 0.7 | - | - | |
2025.02 | 16.4 | 41.9 | 7.9 | 0.4 | - | - | |
2025.02 | 16.1 | 41.2 | 7.9 | 0.4 | - | - | |
2025.02 | 15.5 | 39.9 | 7.4 | 0.4 | - | - | |
2026.02 | - | - | - | - | 57.2 | 69.7 | |
2026.02 | - | - | - | - | 57.1 | 69.7 | |
2026.02 | - | - | - | - | 58.9 | 71.4 | |
2026.02 | - | - | - | - | 57.2 | 72.6 | |
2026.02 | - | - | - | - | 59.4 | 72.6 | |
2026.02 | - | - | - | - | 60 | 69.1 | |
2026.02 | - | - | - | - | 61.7 | 75.4 | |
2026.02 | - | - | - | - | 62.3 | 75.4 | |
2026.02 | - | - | - | - | 62.9 | 72 | |
2026.02 | - | - | - | - | 62.9 | 74.9 | |
2026.02 | - | - | - | - | 63.4 | 74.9 | |
2026.02 | - | - | - | - | 65.1 | 78.9 | |
2026.02 | - | - | - | - | 68.9 | 79.4 | |
2026.02 | - | - | - | - | 68.6 | 79.4 | |
2026.02 | - | - | - | - | 70.3 | 80.6 |