Share your thoughts, 1 month free Claude Pro on usSee more

Language Modeling on WikiText-2 vLLM harness (test)

8.87Perplexity (PPL)

Llama 3.1-8B-Instruct: Baseline (FP16)

Updated 1mo ago

Evaluation Results

Method	Links
Llama 3.1-8B-Instruct: Baseline (FP16) 2026.03		8.87
GGUF Q5 K S 2026.03		8.99
FP8 2026.03		9.04
GPTQ INT4 2026.03		9.3
Clustered (Orig) — no training 2026.03		9.32
AWQ INT4 2026.03		9.35
AQLM 2-bit 2026.03		11.77
Compressed 3B-Llama 2026.03		12.62
Compressed + Clustered + Fine-tuned 2026.03		13.05
Compressed + Clustered 2026.03		13.36
Compressed + Clustered + AWQ 2026.03		13.86
Compressed + Clustered + GPTQ 2026.03		14.21