Share your thoughts, 1 month free Claude Pro on usSee more

Code Translation on HumanEval-X (test)

87.19CA@1 (C++ to Java)

CTO (Qwen2.5-Coder-7B)

Updated 2mo ago

Evaluation Results

Method	Links
CTO (Qwen2.5-Coder-7B) 2026.05		87.19	82.31	73.78	87.19	64.02	71.95
Qwen2.5-Coder-7B-SFT 2026.05		84.76	77.44	72.56	86.59	62.8	70.12
CTO (CodeLlama-7B) 2026.05		84.15	74.39	65.85	83.54	50.61	59.15
CodeLlama-7B-SFT 2026.05		82.93	68.29	59.15	82.32	50	56.71
Qwen2.5-Coder-7B 2026.05		77.44	74.39	70.12	81.71	56.71	68.29
CodeLlama-7B 2026.05		75.61	62.8	50	78.66	42.68	51.83
CTO (CodeT5) 2026.05		50	45.73	49.39	69.51	29.27	29.27
TransCoder 2026.05		48.17	38.41	38.41	42.68	20.73	26.83
CodeT5-SFT 2026.05		47.56	43.9	45.73	65.24	27.44	28.66
TransCoder-ST 2026.05		46.34	40.85	40.85	64.63	21.95	23.17
PPOCoder 2026.05		42.68	40.85	42.68	59.76	26.83	29.27