MHA
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Multi-Head Attention (MHA) | MHA causal mask head dimension 128 FP8 on NVIDIA L40S GPU (test) | Performance (TFLOPS)257.9 | 6 | |
| Multi-Head Attention Operator Development | MHA (hd=64, sl=1024) on A100 GPU | TFLOPS175.6 | 2 |
Showing 2 of 2 rows