Share your thoughts, 1 month free Claude Pro on usSee more

Execution-based Output Prediction on CRUXEval-O (200-sample, seed 42, test)

55Accuracy

Qwen3-14B-Base

Updated 1mo ago

Evaluation Results

Method	Links
Qwen3-14B-Base 2026.06		55
Qwen2.5-Coder-14B 2026.06		54
Qwen2.5-14B 2026.06		51
Qwen3-8B-Base 2026.06		51
Qwen2.5-Coder-7B (anchor) 2026.06		50.5
Qwen2.5-7B 2026.06		49
Qwen3-4B-Base 2026.06		45.5
DeepSeek-Coder-6.7B 2026.06		45.5
Qwen2.5-Coder-1.5B 2026.06		40
Qwen2.5-Coder-3B 2026.06		39
CodeLlama-7B 2026.06		36.5
Qwen3-1.7B-Base 2026.06		35.5
Qwen2.5-3B 2026.06		35
Qwen2.5-Coder-1.5B-Instruct 2026.06		34
Qwen2.5-1.5B 2026.06		33.5
Qwen3-0.6B-Base 2026.06		27.5
Qwen2.5-Coder-0.5B-Instruct 2026.06		27
Llama-2-7B 2026.06		17.5
Qwen2.5-0.5B 2026.06		7
Qwen2.5-Coder-0.5B 2026.06		1