Long-context language understanding on RULER (4k Context)
27VT Score
LLama2-7B-chat
Updated 3d ago
Evaluation Results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
2024.09 | 27 | - | - | - | - | 74.33 | - | - | - | 95.87 | 85.6 | 63 | |
2026.02 | 17.4 | 100 | 89.6 | 87.8 | 79.2 | 10 | 79.8 | 56.4 | 63.7 | - | - | - | |
2026.02 | 17.3 | 100 | 89.4 | 88.4 | 78.9 | 10 | 80 | 56.2 | 63.7 | - | - | - | |
2024.09 | 9.8 | - | - | - | - | 20.33 | - | - | - | 44 | 15.2 | 7 | |
2024.09 | 7.2 | - | - | - | - | 2 | - | - | - | 66.22 | 32.4 | 37 | |
2024.09 | 5.2 | - | - | - | - | 4.67 | - | - | - | 27.44 | 7.6 | 14.5 | |
2024.09 | 3 | - | - | - | - | 0 | - | - | - | 8.28 | 1.5 | 35 | |
2024.09 | 1.6 | - | - | - | - | 9.33 | - | - | - | 41.07 | 16.6 | 55.5 | |
2024.09 | 1.2 | - | - | - | - | 13.67 | - | - | - | 2.45 | 6.8 | 35.5 | |
2024.09 | 0 | - | - | - | - | 24.67 | - | - | - | 0.44 | 27.7 | 32.5 |