Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Training Iteration Efficiency on Qwen3-4B base (train)
Loading...
129.8
Per-step Latency (ms)
FORGE
102.616
286.108
469.6
653.092
Jun 22, 2026
Per-step Latency (ms)
Peak GPU Memory (GiB)
Updated 1mo ago
Evaluation Results
Method
Method
Links
Per-step Latency (ms)
Peak GPU Memory (GiB)
FORGE
Hardware=H200, BS=1, S...
2026.06
129.8
20.6
fused AdamW
Hardware=H200, BS=1, S...
2026.06
158.4
40.1
GaLore r=128
Hardware=H200, BS=1, S...
2026.06
176.9
19.1
FlashOptim
Hardware=H200, BS=1, S...
2026.06
179
22.6
vanilla AdamW
Hardware=H200, BS=1, S...
2026.06
187.6
46.2
APOLLO-Mini r=256
Hardware=H200, BS=1, S...
2026.06
212.4
20.1
optimi.gradient_release
Hardware=H200, BS=1, S...
2026.06
225.6
33.5
bnb 8-bit AdamW
Hardware=H200, BS=1, S...
2026.06
260.6
24.4
AdaLomo
Hardware=H200, BS=1, S...
2026.06
809.4
12.1
Feedback
Search any
task
Search any
task