Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
In-context Retrieval on RULER 8K
Loading...
100
S-N
Full-Attn RoPE
14.72
36.86
59
81.14
Jul 3, 2026
S-N
MK-MQ
VT
Updated 18d ago
Evaluation Results
Method
Method
Links
S-N
MK-MQ
VT
Full-Attn RoPE
Parameters=345M, Train...
2026.07
100
97
34
Full-Attn HoPE
Parameters=345M, Train...
2026.07
100
98
36
Full-Attn HoPE (extra MLP)
Parameters=345M, Train...
2026.07
100
99
39
Naive-BSA-HoPE
Parameters=345M, Train...
2026.07
100
97
23
HiLS-Attn-HoPE
Parameters=345M, Train...
2026.07
100
95
72
Dash-Attention-RoPE
Parameters=345M, Train...
2026.07
92
67
34
HSA-Ultralong
Parameters=345M, Train...
2026.07
87
79
23
InfLLM v2*
Parameters=345M, Train...
2026.07
77
39
25
NSA-RoPE
Parameters=345M, Train...
2026.07
49
17
16
SWA-RoPE
Parameters=345M, Train...
2026.07
18
8
22
Feedback
Search any
task
Search any
task