Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Training Iteration Efficiency on Llama-3.1-8B base (train)
Loading...
99.1
Latency (ms)
FORGE
77.276
224.588
371.9
519.212
Jun 22, 2026
Latency (ms)
Peak GPU Memory (GiB)
Updated 1mo ago
Evaluation Results
Method
Method
Links
Latency (ms)
Peak GPU Memory (GiB)
FORGE
Hardware=H200, BS=1, S...
2026.06
99.1
35.4
FlashOptim
Hardware=H200, BS=1, S...
2026.06
148.5
42.3
fused AdamW
Hardware=H200, BS=1, S...
2026.06
150.9
75
GaLore r=128
Hardware=H200, BS=1, S...
2026.06
158.1
37.2
APOLLO-Mini r=256
Hardware=H200, BS=1, S...
2026.06
199.6
38.4
vanilla AdamW
Hardware=H200, BS=1, S...
2026.06
209.2
90
optimi.gradient_release
Hardware=H200, BS=1, S...
2026.06
224
64.5
bnb 8-bit AdamW
Hardware=H200, BS=1, S...
2026.06
328.8
45.4
AdaLomo
Hardware=H200, BS=1, S...
2026.06
644.7
23.6
Feedback
Search any
task
Search any
task