Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Attention Quantization Error Analysis on Flash Attention (seq=16384, head_dim=64)
Loading...
0.49
L2 Relative Error
Flash+MSD
0.4532
0.7016
0.95
1.1984
May 13, 2026
L2 Relative Error
Percentage of Elements with L2 Error > 0.1%
Percentage of Elements with L2 Error > 0.5%
Percentage of Elements with L2 Error > 1%
Percentage of Elements with L2 Error > 5%
Updated 2mo ago
Evaluation Results
Method
Method
Links
L2 Relative Error
Percentage of Elements with L2 Error > 0.1%
Percentage of Elements with L2 Error > 0.5%
Percentage of Elements with L2 Error > 1%
Percentage of Elements with L2 Error > 5%
Flash+MSD
Sequence length (seq)=...
2026.05
0.49
89.4
45.9
22.1
4.1
Flash
Sequence length (seq)=...
2026.05
1.38
97.7
87.2
66.7
8.6
Dequant
Sequence length (seq)=...
2026.05
1.41
97.7
87.4
67.5
8.9
Feedback
Search any
task
Search any
task
