Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
LLM Pre-training Resource Utilization on C4
Loading...
12.55
Model Weights Share
Full params
11.9225
12.23625
12.55
12.86375
Mar 6, 2026
Model Weights Share
Gradients Share
Optimizer State Share
Other Overhead Share
Total Resource Utilization
Updated 1mo ago
Evaluation Results
Method
Method
Links
Model Weights Share
Gradients Share
Optimizer State Share
Other Overhead Share
Total Resource Utilization
Full params
Backbone=LLaMA-7B
2026.03
12.55
12.55
25.1
14.66
64.86
GaLore
Backbone=LLaMA-7B, ran...
2026.03
12.55
12.55
1.73
4.4
31.23
GoLore
Backbone=LLaMA-7B, ran...
2026.03
12.55
12.55
1.73
4.4
31.23
LISA
Backbone=LLaMA-7B, sam...
2026.03
12.55
1.24
2.48
3.29
19.56
LISA-wor
Backbone=LLaMA-7B, sam...
2026.03
12.55
1.24
2.48
3.29
19.56
Feedback
Search any
task
Search any
task