Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Multilingual Reasoning on MMLU ProX Lite
Loading...
79
Accuracy (en)
Full translation
76.92
77.46
78
78.54
Oct 31, 2025
Accuracy (en)
Accuracy (de)
Accuracy (es)
Accuracy (ar)
Accuracy (ja)
Accuracy (ko)
Accuracy (th)
Accuracy (bn)
Accuracy (sw)
Accuracy (te)
Average Accuracy
Translator Usage
Updated 5d ago
Evaluation Results
Method
Method
Links
Accuracy (en)
Accuracy (de)
Accuracy (es)
Accuracy (ar)
Accuracy (ja)
Accuracy (ko)
Accuracy (th)
Accuracy (bn)
Accuracy (sw)
Accuracy (te)
Average Accuracy
Translator Usage
Full translation
Model=Qwen3-4B
2025.10
79
77.8
78.3
76
77.8
77.4
76.1
76.4
71.6
74.8
76.5
100
Selective translation
Model=Qwen3-4B
2025.10
77.3
76.9
77.2
74.1
75.9
75.9
73.9
74.1
65.1
72.8
74.3
20.8
Base
Model=Qwen3-4B
2025.10
77
77.8
76.5
73.3
75
74.6
73.9
74.6
53.6
71.1
72.7
0
Feedback
Search any
task
Search any
task