Share your thoughts, 1 month free Claude Pro on usSee more

Multi-task Language Understanding on MMLU (Generic Score)

63.73MMLU Score

MONA

Updated 1mo ago

Evaluation Results

Method	Links
MONA 2026.05		63.73
Muon 2026.05		62.81
AdamW 2026.05		62.18
Llama-8B (unpruned) 2026.05		0.667
Llama-8B (unpruned) 2026.05		0.667
Qwen-3B (unpruned) 2026.05		0.644
Qwen-3B (unpruned) 2026.05		0.644
Qwen-3B (KOFF) 2026.05		0.61
Llama-8B (KOFF) 2026.05		0.603
Llama-3B (unpruned) 2026.05		0.56
Llama-3B (unpruned) 2026.05		0.56
Llama-3B (KOFF) 2026.05		0.52
Llama-3B (KOFF) 2026.05		0.484
Llama-8B (KOFF) 2026.05		0.415
Qwen-3B (KOFF) 2026.05		0.413
Llama-8B (pruning only) 2026.05		0.408
QVec 2026.06		0.396
Attacked 2026.06		0.391
Original 2026.06		0.39
Attacked 2026.06		0.388
Gaussian 2026.06		0.387
Llama-8B (pruning only) 2026.05		0.384
Original 2026.06		0.364
QVec 2026.06		0.361
Original 2026.06		0.36
Gaussian 2026.06		0.335
Attacked 2026.06		0.333
Qwen-3B (pruning only) 2026.05		0.329
Gaussian 2026.06		0.328
Qwen-3B (pruning only) 2026.05		0.326
Llama-3B (pruning only) 2026.05		0.316
QVec 2026.06		0.301
Llama-3B (pruning only) 2026.05		0.249