Multiple-choice Question Answering on LongBench v2 (val)
37.8Overall Accuracy
o1-mini
Updated 4d ago
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
2024.09 | 37.8 | 38.9 | 37.1 | 48.6 | 33.3 | 28.6 | 14 | |
2024.09 | 34.4 | 38 | 32.2 | 41.7 | 30.7 | 29.6 | 15 | |
2024.09 | 31.8 | 33.3 | 30.9 | 37.8 | 28.4 | 28.7 | - | |
2024.09 | 31.6 | 32.3 | 31.2 | 41.1 | 27.4 | 24.1 | 16 | |
2024.09 | 31.2 | 31.3 | 31.2 | 39.4 | 27 | 25.9 | - | |
2024.09 | 31 | 32.8 | 29.9 | 38.3 | 27.9 | 25 | 17 | |
2024.09 | 30.8 | 33.9 | 28.9 | 37.8 | 27.4 | 25.9 | 18 | |
2024.09 | 30.2 | 30.7 | 29.9 | 33.9 | 29.8 | 25 | 19 | |
2024.09 | 30 | 30.7 | 29.6 | 35 | 27.9 | 25.9 | 20 | |
2024.09 | 30 | 30.7 | 29.6 | 40.6 | 24.2 | 24.1 | 21 | |
2024.09 | 29.8 | 34.4 | 27 | 36.7 | 27 | 24.1 | 22 | |
2024.09 | 29.3 | 31.1 | 28.2 | 31.8 | 28.6 | 26.2 | 23 | |
2024.09 | 28.2 | 30.2 | 26 | 36.8 | 24.2 | 21.5 | - | |
2024.09 | 27.8 | 30.2 | 26.4 | 36.7 | 23.7 | 21.3 | 24 | |
2024.09 | 26.6 | 29.7 | 24.8 | 37.8 | 19.5 | 22.2 | 25 |