Our new X account is live! Follow @wizwand_team for updates

Pre-training on C4 (val)

25.08Perplexity

Full-Rank

Updated 4d ago

Evaluation Results

Method	Links
Full-Rank 2024.05		25.08	-
VeLoRA 2024.05		25.29	-
GaLore 2024.05		25.36	-
FLORA 2024.05		25.88	-
Full-Rank 2024.05		33.52	-
VeLoRA 2024.05		33.76	-
LORA 2024.05		33.92	-
FLORA 2024.05		34.35	-
GaLore 2024.05		34.88	-
LORA 2024.05		34.99	-