Scaling Law Fitting

Benchmarks

Dataset Name	SOTA Method	Metric
NMM interleaved (test)	PBH	MRE2.693	23	2mo ago
NMM-captions (test)	PBH	MRE0.0373	23	2mo ago
LVM highquality 2 (test)	PBH	Mean Relative Error (MRE)0.0754	23	2mo ago
LVM-highquality 1 (test)	PBH	MRE0.0763	23	2mo ago
LVM-alttext (test)	PBH	MRE3.083	23	2mo ago
LLM-wikipedia (test)	PBH	MRE0.0854	23	2mo ago
LLM-stackexchange (test)	PBH	MRE6.109	23	2mo ago
LLM-github (test)	PBH	MRE8.597	23	2mo ago
LLM-commoncrawl (test)	PBH	Mean Relative Error0.0396	23	2mo ago
LLM-c4 (test)	PBH	MRE0.0411	23	2mo ago
LLM-book (test)	PBH	Mean Relative Error (MRE)3.671	23	2mo ago
LLM-arxiv (test)	PBH	MRE6.848	23	2mo ago
NMM text (test)	PBH	MRE0.0234	14	2mo ago
LVM-synthetic (test)	PBH	MRE9.318	14	2mo ago
StarCoder	Perturbation-aware Baseline (Exponential)	Performance (1e-5)0.98	7	2mo ago
SiQA	Shannon Scaling Law	Score at Scaling Factor 1e-50.987	7	2mo ago
GSM8K	Shannon Scaling Law	Performance (Scale 1e-5)0.986	7	2mo ago
OLMo2 Suite	Shannon Scaling Law	Performance (4-bit)0.9886	7	2mo ago
Pythia Suite	Shannon Scaling Law	Performance (4-bit)99.53	7	2mo ago
Enoki (train)	AdamW	Scaling Parameter b0.405	5	5mo ago
DCLM strongly regularized baseline 20 points (Full fit)	SoftQ	RMSE0.008	4	1mo ago
Regularized-baseline points	SoftQ	RMSE0.0079	4	1mo ago
Scaling Law Datasets (Full fit)	SoftQ	RMSE0.0079	4	1mo ago
Baseline Scaling Dataset 400M (Held-out)	SoftQ	RMSE0.006	4	1mo ago
Baseline Scaling Dataset (Full fit)	SoftQ	RMSE0.008	4	1mo ago

Showing 25 of 25 rows