Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
In-context retrieval on RULER 16K
Loading...
100
S-N Score
Full-Attn HoPE (extra MLP)
-4
23
50
77
Jul 3, 2026
S-N Score
MK-MQ Score
VT Score
Updated 18d ago
Evaluation Results
Method
Method
Links
S-N Score
MK-MQ Score
VT Score
Full-Attn HoPE (extra MLP)
Parameters=345M, Train...
2026.07
100
97
13
HiLS-Attn-HoPE
Parameters=345M, Train...
2026.07
100
94
65
Full-Attn HoPE
Parameters=345M, Train...
2026.07
99
97
21
Naive-BSA-HoPE
Parameters=345M, Train...
2026.07
99
89
16
HSA-Ultralong
Parameters=345M, Train...
2026.07
92
75
17
InfLLM v2*
Parameters=345M, Train...
2026.07
21
0
1
NSA-RoPE
Parameters=345M, Train...
2026.07
6
5
4
SWA-RoPE
Parameters=345M, Train...
2026.07
4
1
11
Full-Attn RoPE
Parameters=345M, Train...
2026.07
0
0
0
Dash-Attention-RoPE
Parameters=345M, Train...
2026.07
0
0
0
Feedback
Search any
task
Search any
task