Share your thoughts, 1 month free Claude Pro on usSee more

Coding on HumanEval

98.17Pass@1

CompassMax-V3-Thinking

Updated 6d ago

Evaluation Results

Method	Links
CompassMax-V3-Thinking 2025.12		98.17
DeepSeek-R1 2025.12		96.95
SwiR 2025.10		95.73
CoT (Greedy) 2025.10		93.9
CoT 2025.10		92.68
Soft Thinking 2025.10		92.07
MT (Teacher) 2026.04		91.5
GPT-4o 2024.10		90.2
Llama-3.1-405B 2025.02		89
Qwen2.5-VL-72B 2025.02		87.8
GPT-4o mini 2024.10		87.2
Qwen2.5-72B 2025.02		86.6
Qwen2-72B-Instruct 2024.07		86
Qwen2-72B 2025.02		86
CompassMax-V3 2025.12		84.76
Gemini-1.5 Pro 2024.10		84.1
UltraMix-190k 2025.11		82.27
Llama-3-70B-Instruct 2024.07		81.7
UltraMix-187k 2025.11		81.1
Llama-3.1-70B 2025.02		80.5
TuluDPO 2025.11		80.49
HPD 2026.04		79.3
UltraMix-170k 2025.11		78.05
KD 2026.04		77.4
JSD 2026.04		77.4
RKLD 2026.04		76.8
MT (Teacher) 2026.04		76.2
Qwen1.5-110B-Chat 2024.07		74.4
Gemini-1.5 Flash 2024.10		74.3
Mixtral-8x22B-Instruct 2024.07		73.8
SFT 2026.04		73.8
Base 2026.05		73.8
Base 2026.05		73.8
Base 2026.05		73.8
ARIA 2024.10		73.2
LangMARL 2026.04		73.2
CoT2-Meta 2026.03		72.8
SFT 2025.11		72.66
Llama3.2-11B 2024.10		72.6
Pixtral-12B 2024.10		72
Qwen1.5-72B-Chat 2024.07		71.3
MS (Student) 2026.04		71.3
STM 2026.05		71.3
Self-SFT 2026.05		71.3
Iter-SFT 2026.05		71.3
Low-SFT 2026.05		71.3
Anchored Learning 2026.05		70.7
ReST-MCTS* 2026.03		70.4
Reflexion 2026.04		70.1
Anchored Learning 2026.05		70.1
Anchored Learning 2026.05		70.1
HPD 2026.04		69.5
UltraMix-190k 2025.11		69.05
TextGrad 2026.04		68.9
Self-sft 2026.05		68.9
Low-SFT 2026.05		68.9
Low-SFT 2026.05		68.9
Anchored Learning 2026.05		68.3
Vanilla ToT 2026.03		68.2
UltraMix-187k 2025.11		68.06
TuluDPO 2025.11		67.24
JSD 2026.04		67.1
GPT-4V 2024.10		67
DSPy 2026.04		66.7
Low-SFT 2026.05		66.5
UltraMix-170k 2025.11		65.61
Best-of-16 2026.03		65.4
KD 2026.04		65.2
Youtu-LLM 2025.12		64.6
STM 2026.05		64.6
Low-SFT 2026.05		64.6
Symbolic 2026.04		64.5
Iter-SFT 2026.05		64
AutoPE 2026.04		63.5
MS (Student) 2026.04		62.8
ProSeCo Sampling 2026.02		62.2
RKLD 2026.04		61.6
SFT 2026.04		61
Anchored Learning 2026.05		61
Greedy CoT 2026.03		60.5
DFT 2026.05		60.4
Agents 2026.04		59.5
CoT 2026.04		59.2
Llama3.1-Instruct 2026.02		58.54
SFT 2025.11		57.93
Iter-SFT 2026.05		57.7
Qwen3 2025.12		57.6
Anchored Learning 2026.05		56.1
Base 2026.05		54.9
Low-SFT 2026.05		53.7
STM 2026.05		53.7
Self-SFT 2026.05		53.7
ProSeCo SFT 2026.02		52.44
CodePref 2025.11		51.24
Qwen3 2025.12		49.9
SELFORG 2025.10		48.97
Vanilla SFT 2026.02		48.17
UM-190k 2025.11		48.07
UM-187k 2025.11		47.68
Phi-2 2024.07		47.6

Showing 100 of 168 rows