Long-context language understanding on LongBench 1.0 (test)
61.5MultiNews
Original
Updated 4d ago
Evaluation Results
| Method | Links | |||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
2025.02 | 61.5 | - | 77.8 | 40.7 | 9.8 | - | - | - | - | - | 31.52 | 18.7 | 19.2 | 36.8 | 25.4 | 32.8 | 27.3 | 20.8 | 25.8 | 52.4 | 43.8 | |
2024.01 | 27.7 | 0.5 | 63.5 | 85.7 | 41.9 | 26 | 57.6 | 54.5 | 39.79 | 14.55 | 34.18 | - | - | - | - | - | - | - | - | - | - | |
2024.01 | 27.2 | 15.1 | 74 | 86.2 | 40.8 | 28.8 | 51 | 43.5 | 36.49 | 26.55 | 34.28 | - | - | - | - | - | - | - | - | - | - | |
2024.01 | 27.1 | 16.4 | 74 | 84.9 | 27.8 | 29.8 | 44.1 | 45.6 | 38.08 | 23.86 | 34.92 | - | - | - | - | - | - | - | - | - | - | |
2024.01 | 27 | 9.8 | 73.5 | 87.9 | 40.6 | 36 | 65.4 | 59.1 | 44.73 | 28.56 | 41.13 | - | - | - | - | - | - | - | - | - | - | |
2024.01 | 26.8 | 10.2 | 75 | 87.8 | 40.9 | 44.5 | 63.8 | 57.5 | 44.55 | 31.93 | 41.74 | - | - | - | - | - | - | - | - | - | - | |
2024.01 | 26.7 | 16 | 68 | 91.4 | 41.7 | 29.2 | 54.7 | 53.6 | 44.6 | 33.78 | 42.19 | - | - | - | - | - | - | - | - | - | - | |
2024.01 | 26.4 | 9.9 | 63.5 | 82.3 | 34.2 | 23.2 | 53 | 55.3 | 36.86 | 20.43 | 33.21 | - | - | - | - | - | - | - | - | - | - | |
2024.01 | 26.1 | 17.2 | 66 | 85.2 | 36.5 | 20.3 | 51.9 | 52.8 | 37.87 | 24.38 | 34.87 | - | - | - | - | - | - | - | - | - | - | |
2024.01 | 25.9 | 9.6 | 68.5 | 89.2 | 38.2 | 35 | 65.8 | 58.1 | 41.26 | 26.7 | 38.03 | - | - | - | - | - | - | - | - | - | - | |
2024.01 | 25.9 | 9.1 | 72.5 | 86.5 | 27.9 | 31 | 62.5 | 51.1 | 40.88 | 26.35 | 37.65 | - | - | - | - | - | - | - | - | - | - | |
2024.01 | 25.8 | 0.2 | 61.5 | 77.8 | 40.7 | 19.8 | 52.4 | 43.8 | 35.17 | 15.45 | 20.79 | - | - | - | - | - | - | - | - | - | - | |
2024.01 | 25.4 | 11.7 | 70.5 | 88.4 | 32.5 | 40 | 64.5 | 60.9 | 41.74 | 30.23 | 39.18 | - | - | - | - | - | - | - | - | - | - | |
2025.02 | 24.78 | - | 67.5 | 74.24 | 40.28 | - | 56.39 | 50.36 | - | - | 27.22 | 8.48 | 13.97 | 20.4 | 13.62 | 16.77 | 5.46 | 25.17 | 12.47 | 2.3 | 3.25 | |
2025.02 | 24.78 | - | 67.5 | 74.24 | 40.28 | - | 56.39 | 50.36 | - | - | 27.22 | 8.48 | 13.97 | 20.4 | 13.62 | 16.77 | 5.46 | 25.17 | 12.47 | 2.3 | 3.25 | |
2025.02 | 23.37 | - | 62 | 72.8 | 25.12 | - | 58.71 | 48.51 | - | - | 25.8 | 6.29 | 16.73 | 22.26 | 5.88 | 26.8 | 12.82 | 23.58 | 15.96 | 1.83 | 2.83 | |
2025.02 | 21.35 | - | 25.78 | 69.5 | 81.99 | - | 60.6 | 54.33 | - | - | 34.62 | 16.77 | 21.69 | 25.02 | 35.21 | 34.34 | 30.24 | 14.13 | 27.32 | 5.66 | 5.83 | |
2025.02 | 13.65 | - | 6.31 | 67.5 | 69.88 | - | 58.21 | 51.65 | - | - | 26.97 | 6.89 | 12.67 | 25.95 | 9.08 | 11.25 | 7.91 | 16.39 | 22.79 | 2.18 | 3.21 | |
2025.02 | 10.9 | - | 25.92 | 64.5 | 40.6 | - | 57.13 | 42.16 | - | - | 23.57 | 3.79 | 10.37 | 22.38 | 10.26 | 26.13 | 13.36 | 1.32 | 24.78 | 2.29 | 4.33 | |
2025.02 | 6.31 | - | 63.5 | 45.35 | 32.36 | - | 51.74 | 43.91 | - | - | 22.22 | 3.22 | 10.86 | 22.14 | 3.81 | 29.52 | 20.13 | 22.84 | 13.65 | 2.2 | 2.15 | |
2025.02 | 0.51 | - | 50.7 | 27.91 | 17.08 | - | 56.39 | 50.36 | - | - | 17.08 | 0.73 | 10.33 | 19.44 | 0.42 | 19.47 | 6.26 | 23.17 | 0.52 | 2.3 | 3.25 |