Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Dialectal Arabic Evaluation on MMLU Dialectal Arabic
Loading...
69.98
Accuracy (MMLU DA)
Karnak
55.6176
59.3463
63.075
66.8037
Mar 17, 2026
Accuracy (MMLU DA)
Updated 1mo ago
Evaluation Results
Method
Method
Links
Accuracy (MMLU DA)
Karnak
Size=40B, Shot count=3...
2026.03
69.98
Llama-3.3-70b-Instruct
Size=70B, Shot count=3...
2026.03
69.63
Qwen3-32B
Size=32B, Shot count=3...
2026.03
68.96
AceGPT-v2-70B-Chat
Size=70B, Shot count=3...
2026.03
68.56
Fanar-27B
Size=27B, Shot count=3...
2026.03
67.4
Gemma-3-27B-it
Size=27B, Shot count=3...
2026.03
66.52
Jais-2-70B-Chat
Size=70B, Shot count=3...
2026.03
66.33
AceGPT-v2-32B-Chat
Size=32B, Shot count=3...
2026.03
62.82
Fanar-1-9b-instruct
Size=9B, Shot count=3-...
2026.03
59.91
Allam-7B-Instruct-preview-v2
Size=7B, Shot count=3-...
2026.03
56.17
Feedback
Search any
task
Search any
task