Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Training Iteration Efficiency on Qwen3-8B base (train)
Loading...
131.6
Per-Step Latency (ms)
FORGE
100.476
310.563
520.65
730.737
Jun 22, 2026
Per-Step Latency (ms)
Peak GPU Memory (GiB)
Updated 1mo ago
Evaluation Results
Method
Method
Links
Per-Step Latency (ms)
Peak GPU Memory (GiB)
FORGE
Hardware=H200, BS=1, S...
2026.06
131.6
36.7
FlashOptim
Hardware=H200, BS=1, S...
2026.06
178.8
43.5
fused AdamW
Hardware=H200, BS=1, S...
2026.06
184.8
76.6
GaLore r=128
Hardware=H200, BS=1, S...
2026.06
196.1
39
optimi.gradient_release
Hardware=H200, BS=1, S...
2026.06
230
66.4
APOLLO-Mini r=256
Hardware=H200, BS=1, S...
2026.06
240
40.1
vanilla AdamW
Hardware=H200, BS=1, S...
2026.06
243.5
91.8
bnb 8-bit AdamW
Hardware=H200, BS=1, S...
2026.06
370.8
46.3
AdaLomo
Hardware=H200, BS=1, S...
2026.06
909.7
25.2
Feedback
Search any
task
Search any
task