Share your thoughts, 1 month free Claude Pro on usSee more

Code Generation on HumanEval (HumanEval and Average metrics)

95.22HumanEval Score

SIGMA

Updated 1d ago

Evaluation Results

Method
SIGMA 2026.05	95.22	93.91
G-Designer 2026.05	94.82	93.22
FinTool-Qwen3-14B 2026.03	94.51	-
GoA 2026.05	94.25	92.86
GPTSwarm 2026.05	94.16	92.39
MoA 2026.05	94.03	89.83
CoT 2026.05	93.41	90.68
ComplexCoT 2026.05	93.14	91.01
Vanilla 2026.05	93.13	87.69
Qwen3-14B 2026.03	92.68	-
Qwen3-8B 2026.03	89.63	-
FinTool-Qwen3-8B 2026.03	89.02	-
AlphaToken 2026.06	78.88	78.88
Standard FT 2026.06	75.94	75.94
ssTOKEN ICLR 2026 2026.06	75.21	75.21
XTF ICLR 2026 2026.06	73.61	73.61
Token Cleaning ICML 2025 2026.06	73.58	73.58
LESS ICML 2024 2026.06	72.82	72.82
STM NeurIPS 2025 2026.06	71.93	71.93
LoRA ICLR 2022 2026.06	70.2	70.2
AlphaToken 2026.06	62.15	62.15
Pre-trained 2026.06	60.96	60.96
Standard FT 2026.06	58.79	58.79
ssTOKEN ICLR 2026 2026.06	57.45	57.45
LESS ICML 2024 2026.06	57.17	57.17
Token Cleaning ICML 2025 2026.06	57.12	57.12
XTF ICLR 2026 2026.06	56.64	56.64
LightMoE 2026.03	55.7	55.3
Original 2026.03	55	37.7
LoRA ICLR 2022 2026.06	54.62	54.62
LoRA 2026.03	54.2	55.5
STM NeurIPS 2025 2026.06	54.01	54.01
Replace (w/o shared) 2026.03	53.8	52.6
Replace (w. shared) 2026.03	52	52.3
MC-SMoE 2026.03	51.7	53.4
Replace (w/o shared) 2026.03	51.3	50.1
LightMoE 2026.03	51.3	53
MC-SMoE* 2026.03	50.6	50.2
Replace (w. shared) 2026.03	50.1	49.3
Full FT 2026.03	49.7	56.3
MC-SMoE 2026.03	48.4	47.3
LightMoE 2026.03	48.4	48.1
Replace (w/o shared) 2026.03	46.3	44.3
Standard FT 2026.06	44.6	44.6
Replace (w. shared) 2026.03	44.5	41.6
AlphaToken 2026.06	43.98	43.98
MC-SMoE* 2026.03	43.8	45.3
MC-SMoE 2026.03	43.1	42.5
ssTOKEN ICLR 2026 2026.06	42.93	42.93
LESS ICML 2024 2026.06	42.24	42.24
XTF ICLR 2026 2026.06	42.02	42.02
HC-SMoE 2026.03	41.7	46.8
STM NeurIPS 2025 2026.06	41.54	41.54
LoRA ICLR 2022 2026.06	41.46	41.46
MoBE 2026.03	41.3	50.4
Token Cleaning ICML 2025 2026.06	40.93	40.93
Pre-trained 2026.06	35.36	35.36
Baseline 2026.04	34.76	46.1
HC-SMoE 2026.03	33.5	38.4
Instruction Tuned 2026.04	30.49	-
DARE Ties 2026.04	30.49	-
Precise Fusion 2026.04	30.49	-
Task Arithmetic 2026.04	29.27	-
Pre-trained 2026.06	28.66	28.66
MoBE 2026.03	28.6	43
Avg Baseline 2026.04	28.05	-
Ties Merging 2026.04	28.05	-
MoBiE 2026.04	27.12	37.54
GPTQ 2026.04	27.05	37.48
DARE Task Arithmetic 2026.04	26.83	-
HC-SMoE 2026.03	26.5	34
WIDEN 2026.04	26.22	-
MoBiE 2026.04	25.66	-
WIDEN 2026.04	25	-
Baseline 2026.04	24.39	-
WIDEN 2026.04	24.39	-
ARB-LLM 2026.04	23.86	30.6
Avg Baseline 2026.04	23.17	-
Precise Fusion 2026.04	22.56	-
Precise Fusion 2026.04	22.56	-
BiLLM 2026.04	22.12	28.8
Baseline 2026.04	21.34	-
Ties Merging 2026.04	21.34	-
Avg Baseline 2026.04	20.73	-
NoWag 2026.04	20.23	26.94
Code 2026.04	19.51	-
ARB-LLM 2026.04	19.31	-
Base 2026.04	18.9	-
Task Arithmetic 2026.04	18.29	-
MoEQuant 2026.04	18.12	22.56
Precise Fusion 2026.04	17.68	-
BiLLM 2026.04	17.2	-
DARE Ties 2026.04	17.07	-
WIDEN 2026.04	17.07	-
Ties Merging 2026.04	16.46	-
DARE Task Arithmetic 2026.04	16.46	-
Avg Baseline 2026.04	15.85	-
Ties Merging 2026.04	15.85	-
MoBiE 2026.04	15.75	-
Math 2026.04	14.02	-

Showing 100 of 128 rows