Share your thoughts, 1 month free Claude Pro on usSee more

Cultural Reasoning on VIRAASAT (test)

91.45State Match

Qwen2.5-7B-Instruct

Updated 3mo ago

Evaluation Results

Method	Links
Qwen2.5-7B-Instruct 2026.02		91.45	58.01	57.54
Qwen2.5-7B-Instruct 2026.02		89.11	37.33	36.7
Qwen3-4B-Thinking-2507 2026.02		88.22	37.36	35.5
Qwen2.5-3B-Instruct 2026.02		87.87	34.73	34.26
Qwen2.5-3B-Instruct 2026.02		87.25	51.48	50.7
Phi-4-mini-reasoning 2026.02		84.19	30.08	28.53
gemini-2.0-flash 2026.02		81.71	35.19	31.32
Phi-4-mini-reasoning 2026.02		81.03	43.55	43.39
GPT4.1-mini 2026.02		77.36	34.57	32.56
Mistral-7B-Instruct-v0.3 2026.02		57.05	15.04	12.25
Llama-3.1-8B-Instruct 2026.02		55.81	11.78	10.08
Qwen2.5-7B-Instruct 2026.02		43.88	3.72	1.71
Phi-4-mini-reasoning 2026.02		35.97	8.84	6.05
Qwen2.5-3B-Instruct 2026.02		25.43	3.72	2.17
OpenHathi-7B-Hi-v0.1-Base 2026.02		25.27	4.19	2.17
gemma-2b-it 2026.02		24.34	4.96	3.26
sarvam-1 2026.02		11.78	7.13	2.17
Qwen3-4B-Thinking-2507 2026.02		7.44	1.71	0.31