Share your thoughts, 1 month free Claude Pro on usSee more

Multilingual Knowledge Evaluation on M-MMLU

77.83Overall Accuracy

Karnak

Updated 1mo ago

Evaluation Results

Method	Links
Karnak 2026.03		77.83	-	-	-	-	-	-	-	-
CAUSAL-MULTI 2025.05		73.8	76.5	83	76	79.5	74	66	67.5	67.1
MULTI-RELATED 2025.05		72.5	77	85	74	80	72	60.5	67.5	63.9
CAUSAL-NATIVE 2025.05		71.3	80.5	75.6	76	80.5	73.7	61	60.8	62.3
Llama-3.3-70b-Instruct 2026.03		70	-	-	-	-	-	-	-	-
Qwen3-32B 2026.03		69.32	-	-	-	-	-	-	-	-
Jais-2-70B-Chat 2026.03		69.01	-	-	-	-	-	-	-	-
MOREINFO 2025.05		68.6	76.8	83.3	68.2	77.9	61.3	59.7	66.9	54.3
AceGPT-v2-70B-Chat 2026.03		68.44	-	-	-	-	-	-	-	-
Gemma-3-27B-it 2026.03		67.65	-	-	-	-	-	-	-	-
Fanar-27B 2026.03		67.4	-	-	-	-	-	-	-	-
AceGPT-v2-32B-Chat 2026.03		61.1	-	-	-	-	-	-	-	-
CONFLICT 2025.05		59.9	66.5	69.2	62.3	73.4	50.3	51.9	48.7	56.5
Fanar-1-9b-instruct 2026.03		58.3	-	-	-	-	-	-	-	-
Allam-7B-Instruct-preview-v2 2026.03		57.16	-	-	-	-	-	-	-	-
CAUSAL-MULTI 2025.05		56.7	57.5	62	52.3	59.5	57	58.1	56.8	50
CONFLICT 2025.05		55.3	52.2	53.4	52.2	53	60.8	56.2	59.9	54.9
MULTI-RELATED 2025.05		54.9	55.5	52.5	56	51	55.5	52.4	55.5	60.9
CAUSAL-MULTI 2025.05		54.5	51.2	53.8	53	53.5	57.4	54.4	54.5	57.9
ASK CALI 2025.05		53.2	55.2	53.4	59.3	54.7	58.7	42.4	47.7	53.9
CAUSAL-NATIVE 2025.05		53.2	57	59	48	60	49.3	56.5	45.5	50.3
REFLECT 2025.05		52.8	54.3	45.7	47	56.6	51.6	58.8	55	53.4
ASK CALI 2025.05		52.2	44	38.5	53.5	42.7	61.8	62.2	56.8	57.5
MULTI-RELATED 2025.05		52.1	56.5	62.5	52	57	50.5	49	48.5	40.4
CONFLICT 2025.05		50.4	47	54.7	45.6	46.1	55	49.7	55	50
CAUSAL-NATIVE 2025.05		50.4	48.2	53.8	51	47.5	55	49.7	47.5	50.7
REFLECT 2025.05		50.4	50.3	52.6	50.3	55.2	46.2	46.5	49.3	52.7
MOREINFO 2025.05		44.6	56	61.1	45.1	53.4	30	40.1	32	39
MOREINFO 2025.05		43.1	50	47.9	50	43.5	34.8	42.9	35.3	41.1
REFLECT 2025.05		42.3	47.8	47.4	46.9	46.3	37	39.5	32	41.8
ASK CALI 2025.05		34.3	20.8	20.5	29.3	20.8	46.2	41.3	41.9	53.1