Long-context language understanding on RULER 64k context length
0FWE (Error)
Full Attention
Updated 5d ago
Evaluation Results
| Method | Links | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
2026.02 | 0 | 99.4 | 98.4 | 96.8 | 93.7 | 19.8 | 70.4 | 47.6 | 65.8 | - | - | - | - | - | - | - | |
2026.02 | 0 | 99.6 | 73.2 | 90.3 | 81.7 | 19.9 | 63.4 | 44.4 | 59 | - | - | - | - | - | - | - | |
2024.09 | 6.33 | - | - | - | - | 0.6 | - | - | - | - | 29.5 | 2.63 | 4.9 | - | - | - | |
2024.09 | 10.67 | - | - | - | - | 4.4 | - | - | - | - | 1 | 50.13 | 2.6 | - | - | - | |
2024.09 | 13.33 | - | - | - | - | 0.2 | - | - | - | - | 15 | 23.25 | 0.3 | - | - | - | |
2024.09 | 15.67 | - | - | - | - | 0 | - | - | - | - | 9 | 0.31 | 2.4 | - | - | - | |
2025.03 | 18.4 | 35.42 | - | 1.56 | 1.82 | 1.46 | 20.83 | 30.21 | 13.78 | - | - | - | - | 12.5 | 13.54 | 2.08 | |
2024.09 | 32.67 | - | - | - | - | 6.2 | - | - | - | - | 34.5 | 9.34 | 0.2 | - | - | - | |
| 34.17 | - | - | - | - | - | - | - | - | - | - | - | 2 | - | - | - | ||
2025.03 | 58.33 | 89.58 | - | 21.35 | 14.06 | 49.38 | 61.46 | 35.42 | 45.04 | - | - | - | - | 66.67 | 43.75 | 10.42 | |
2025.03 | 61.11 | 25 | - | 44.79 | 28.91 | 3.54 | 47.92 | 36.46 | 45.71 | - | - | - | - | 51.04 | 61.46 | 96.88 | |
2025.03 | 68.06 | 100 | - | 98.44 | 100 | 95.42 | 83.33 | 57.29 | 89.94 | - | - | - | - | 100 | 100 | 96.88 | |
2025.03 | 68.75 | 100 | - | 97.66 | 100 | 95 | 83.33 | 58.33 | 89.68 | - | - | - | - | 100 | 98.96 | 94.79 | |
2025.03 | 78.47 | 100 | - | 96.62 | 96.09 | 86.67 | 78.13 | 56.25 | 88.5 | - | - | - | - | 96.88 | 97.92 | 97.92 | |
2025.03 | 80.21 | 94.79 | - | 66.15 | 82.81 | 46.04 | 48.96 | 50 | 71.79 | - | - | - | - | 66.67 | 83.33 | 98.96 | |
2025.03 | 84.38 | 100 | - | 89.58 | 91.41 | 81.46 | 80.21 | 55.21 | 86.87 | - | - | - | - | 96.88 | 98.96 | 90.63 | |
2025.03 | 85.42 | 100 | - | 97.66 | 98.96 | 97.29 | 83.33 | 59.38 | 91.89 | - | - | - | - | 100 | 98.96 | 97.92 | |
2025.03 | 85.76 | 100 | - | 91.93 | 100 | 95 | 71.88 | 60.42 | 90.19 | - | - | - | - | 97.92 | 100 | 98.96 | |
2025.03 | 86.11 | 100 | - | 97.14 | 99.74 | 99.58 | 83.33 | 64.58 | 91.17 | - | - | - | - | 100 | 100 | 81.25 | |
2025.03 | 89.58 | 100 | - | 92.97 | 98.18 | 97.71 | 78.13 | 64.58 | 90.45 | - | - | - | - | 98.96 | 95.83 | 88.54 | |
2025.03 | 90.28 | 100 | - | 95.57 | 99.74 | 99.79 | 81.25 | 65.63 | 91.66 | - | - | - | - | 100 | 100 | 84.38 | |
2025.03 | 91.32 | 100 | - | 98.96 | 100 | 99.58 | 80.21 | 64.58 | 93.36 | - | - | - | - | 100 | 100 | 98.96 | |
2026.02 | - | - | 86.88 | 86.67 | - | - | - | - | 83.78 | 98.27 | 63.3 | - | - | - | - | - | |
2026.02 | - | - | 58.2 | 88.33 | - | - | - | - | 73.12 | 82.23 | 63.7 | - | - | - | - | - | |
2026.02 | - | - | 87.16 | 86.67 | - | - | - | - | 81.28 | 87.69 | 63.6 | - | - | - | - | - | |
2026.02 | - | - | 87.72 | 86.67 | - | - | - | - | 82.01 | 89.85 | 63.8 | - | - | - | - | - |