Our new X account is live! Follow @wizwand_team for updates
Home
/
Benchmarks
Multi-Head Attention (MHA) on NVIDIA A100 GPU
Loading...
207.2
TFLOPS
DeepSeek-V3 + Ours
-0.28
53.585
107.45
161.315
Jun 14, 2025
TFLOPS
Speedup
Updated 4d ago
Evaluation Results
Method
Method
Links
TFLOPS
Speedup
DeepSeek-V3 + Ours
Head Dimension=128, Se...
2025.06
207.2
3.73
flash-attn v2
Head Dimension=128, Se...
2025.06
195.1
-
DeepSeek-V3 + Ours
Head Dimension=64, Seq...
2025.06
184.3
34.94
cuDNN
Head Dimension=64, Seq...
2025.06
95.3
-
DeepSeek-V3
Head Dimension=64, Seq...
2025.06
7.7
-
Feedback
Search any
task
Search any
task