Share your thoughts, 1 month free Claude Pro on usSee more

Multilingual Code Evaluation on HumanEval-X

95.45Python Score

GPT-o1-mini

Updated 1mo ago

Evaluation Results

Method	Links
GPT-o1-mini 2025.02		95.45	92.42	94.7	90.91	88.64	92.42
MCTS-Judge 2025.02		90.15	85.61	84.09	84.85	81.06	85.15
MCTS-Judge 2025.02		87.88	86.36	88.64	88.64	85.61	87.43
MCTS-Judge 2025.02		87.78	75.76	77.27	73.48	75.76	78.01
CodeJudge 2025.02		87.12	83.33	87.88	86.36	84.09	85.76
CodeJudge 2025.02		86.36	81.06	79.55	82.58	75.75	81.06
Vanilla 2025.02		86.36	82.58	85.61	86.36	84.85	85.15
CodeJudge 2025.02		85.61	69.7	68.94	71.21	66.67	72.43
ICE-Score 2025.02		84.85	78.79	86.36	83.33	85.61	83.79
GPT-o1-preview 2025.02		82.58	89.39	87.12	86.36	83.33	85.76
Vanilla 2025.02		82.58	68.18	70.45	62.88	66.67	70.15
ICE-Score 2025.02		82.58	68.18	60.61	63.64	61.36	67.27
MCTS-Judge 2025.02		80.3	77.27	80.3	78.79	82.58	79.85
MCTS-Judge 2025.02		74.24	79.55	77.27	70.45	71.97	74.7
Vanilla 2025.02		73.48	64.39	63.64	66.67	60.61	65.76
CodeJudge 2025.02		73.48	69.7	67.42	69.7	66.67	69.39
CodeJudge 2025.02		73.48	73.48	75.76	70.45	67.42	72.12
Qwen-QwQ-32B 2025.02		72.73	75	75	64.39	78.03	73.03
ICE-Score 2025.02		72.73	74.24	71.97	72.73	78.79	74.09
ICE-Score 2025.02		72.73	64.39	62.88	56.82	54.55	62.27
ICE-Score 2025.02		71.21	76.52	69.7	70.45	74.24	72.42
Vanilla 2025.02		65.91	63.64	64.39	62.12	70.45	65.3
Vanilla 2025.02		62.12	64.39	68.94	64.39	73.48	66.66
GPT-4o 2025.02		59.09	53.79	60.61	62.12	65.91	60.3