Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Training Iteration Efficiency on Qwen3-1.7B base (train)
Loading...
101.7
Per-Step Latency (ms)
FORGE
84.668
199.634
314.6
429.566
Jun 22, 2026
Per-Step Latency (ms)
Peak GPU Memory (GiB)
Updated 1mo ago
Evaluation Results
Method
Method
Links
Per-Step Latency (ms)
Peak GPU Memory (GiB)
FORGE
Hardware=H200, BS=1, S...
2026.06
101.7
10.4
fused AdamW
Hardware=H200, BS=1, S...
2026.06
114.5
18
vanilla AdamW
Hardware=H200, BS=1, S...
2026.06
127.3
20.1
GaLore r=128
Hardware=H200, BS=1, S...
2026.06
137.4
9.5
FlashOptim
Hardware=H200, BS=1, S...
2026.06
138.1
10.4
APOLLO-Mini r=256
Hardware=H200, BS=1, S...
2026.06
160.3
9.9
bnb 8-bit AdamW
Hardware=H200, BS=1, S...
2026.06
164.8
11.1
optimi.gradient_release
Hardware=H200, BS=1, S...
2026.06
177.2
15
AdaLomo
Hardware=H200, BS=1, S...
2026.06
527.5
7
Feedback
Search any
task
Search any
task