Share your thoughts, 1 month free Claude Pro on usSee more

Multilingual Language Understanding on M-MMLU (test)

59.6Overall Accuracy

COMPASS

Updated 1mo ago

Evaluation Results

Method	Links
COMPASS 2026.04		59.6	-	-	-	-	-	-	-	-
COMPASS-FFT 2026.04		58.1	-	-	-	-	-	-	-	-
LangRank 2026.04		56.7	-	-	-	-	-	-	-	-
CAUSAL-MULTI 2025.05		56.5	61.2	58.9	57.6	52.3	49.6	58.1	58.1	55.8
LangSim 2026.04		56.1	-	-	-	-	-	-	-	-
COMPASS-FFT 2026.04		55.5	-	-	-	-	-	-	-	-
COMPASS 2026.04		55.2	-	-	-	-	-	-	-	-
Random 2026.04		55.1	-	-	-	-	-	-	-	-
CAUSAL-MULTI 2025.05		54.7	57.4	64.7	60.9	58.4	48.2	44.1	50.6	53.1
Target 2026.04		54.6	-	-	-	-	-	-	-	-
REFLECT 2025.05		53.1	65.2	60.3	57	57.8	45.5	45	48	45.7
Pretrained (ZS) 2026.04		52.9	-	-	-	-	-	-	-	-
COMPASS 2026.04		52.4	-	-	-	-	-	-	-	-
CAUSAL-NATIVE 2025.05		52.2	57.4	47.4	53.6	53.2	45.5	48	58	54.2
LangRank 2026.04		51.8	-	-	-	-	-	-	-	-
CONFLICT 2025.05		51.4	56.8	52.6	45.7	43.5	47.6	50.4	52.7	52.7
MULTI-RELATED 2025.05		51.4	49.7	53.2	56.3	45.5	49.7	44.2	58.1	54.3
REFLECT 2025.05		51.2	41.9	51.9	57	54.6	53.1	50.4	48	52.7
CAUSAL-NATIVE 2025.05		51.2	51.6	60.8	58.9	58.4	44.1	41	44.5	50
LangSim 2026.04		51	-	-	-	-	-	-	-	-
Target 2026.04		50.8	-	-	-	-	-	-	-	-
Random 2026.04		50.6	-	-	-	-	-	-	-	-
MOREINFO 2025.05		50.5	48.4	46.8	54.3	48.7	40.7	54.3	60.1	50.4
COMPASS-FFT 2026.04		49.9	-	-	-	-	-	-	-	-
CONFLICT 2025.05		49.6	51	53.2	53.6	49.4	42.1	52.7	51.4	43.4
Pretrained (ZS) 2026.04		49.1	-	-	-	-	-	-	-	-
ASK CALI 2025.05		48.5	64.6	44.2	69.5	66.2	26.9	38.8	30.4	47.3
MOREINFO 2025.05		48.4	53.6	50.6	49	47.4	49.7	38	51.4	47.3
All 2026.04		48	-	-	-	-	-	-	-	-
MULTI-RELATED 2025.05		47.9	49.7	53.2	47.7	48.1	47.6	46.6	46	44.2
LangRank 2026.04		47.3	-	-	-	-	-	-	-	-
LangSim 2026.04		47.3	-	-	-	-	-	-	-	-
ASK CALI 2025.05		44.9	38.1	42.3	49	43.5	44.2	44.2	51.4	46.7
Random 2026.04		44.9	-	-	-	-	-	-	-	-
Target 2026.04		44.7	-	-	-	-	-	-	-	-
All 2026.04		43.8	-	-	-	-	-	-	-	-
Pretrained (ZS) 2026.04		43.5	-	-	-	-	-	-	-	-
All 2026.04		38.8	-	-	-	-	-	-	-	-