Share your thoughts, 1 month free Claude Pro on usSee more

Question Answering on ARC (accuracy)

94.2Accuracy

GPT-5 nano

Updated 1mo ago

Evaluation Results

Method	Links
GPT-5 nano 2026.05		94.2
Qwen3-8B 2026.05		93
Ministral-3-8B 2026.05		92.1
gpt-oss-20b 2026.05		92.1
Qwen3-4B 2026.05		90.3
gemma-3-12b-it 2026.05		89.9
FP 2026.06		88.5
STaR-Quant 2026.06		85.12
QuaRot 2026.06		84.23
DLLMQuant++ 2026.06		84.18
DLLMQuant+ 2026.06		83.38
AWQ 2026.06		82.22
RTN 2026.06		81.77
EngGPT2-16B-A3B 2026.05		80.2
Llama-3.1-8B 2026.05		78.1
gemma-3-4b-it 2026.05		75.3
Moonlight-16B-A3B 2026.05		74.1
Velvet-14B 2026.05		70.3
Llama-3.2-3B 2026.05		69.8
LLaMAntino-3-8B 2026.05		65.6
FP 2026.06		59.8
STaR-Quant 2026.06		59.76
DLLMQuant++ 2026.06		59.43
DLLMQuant+ 2026.06		58.38
LoopQ 2026.05		58.19
QuaRot 2026.06		58.18
AWQ 2026.06		57.82
RTN 2026.06		54.34
FastwebMIIA-7B 2026.05		53.5
LoopQ 2026.05		51.45
LoopQ 2026.05		50.34
LoopQ 2026.05		48.81
deepseek-moe-16b 2026.05		47
STaR-Quant 2026.06		44.23
QuaRot 2026.06		44.2
DLLMQuant++ 2026.06		44.18
FP 2026.06		44.03
DLLMQuant+ 2026.06		43.44
AWQ 2026.06		42.92
RTN 2026.06		41.83
LoopQ 2026.05		29.61
LoopQ 2026.05		29.29
Minerva-7B 2026.05		25.9
LoopQ 2026.05		25.61
LoopQ 2026.05		23.72