Share your thoughts, 1 month free Claude Pro on usSee more

Code Generation on BigCode

46.8Accuracy

MCNIG

Updated 1mo ago

Evaluation Results

Method	Links
MCNIG 2026.03		46.8
Self-Certainty 2026.04		46.2
Lowest Centroid 2026.04		46
Tail Confidence 2026.04		45.4
Greedy Decoding 2026.04		44.9
Lowest Centroid 2026.04		44.5
Lowest Centroid 2026.04		44.5
ORM 2026.03		44.1
Bottom Window 2026.04		44.1
Bottom Window 2026.04		44
Self-Certainty 2026.04		44
Pass@1 2026.04		43.9
Bottom Window 2026.04		43.4
Greedy Decoding 2026.04		43.4
Self-Certainty 2026.04		43.3
IG 2026.03		43
Pass@1 2026.04		42.6
Tail Confidence 2026.04		42.3
Tail Confidence 2026.04		41.5
Pass@1 2026.04		39.9
Greedy Decoding 2026.04		39.8
Tail Confidence 2026.04		36.8
Greedy Decoding 2026.04		36.1
Bottom Window 2026.04		36.1
Lowest Centroid 2026.04		35.8
Pass@1 2026.04		35
OVM 2026.03		34.9
Self-Certainty 2026.04		34.9
QwenPRM 2026.03		31.5
Granite PRM v2 2026.03		31.4
MathShepherd 2026.03		29.5
ImplicitPRM 2026.03		29.3
Majority voting 2026.03		27.6
Greedy Decoding 2026.04		26.3
Lowest Centroid 2026.04		25.5
Single sampling 2026.03		23.3
Pass@1 2026.04		22.9
Tail Confidence 2026.04		19.1
Bottom Window 2026.04		19
Self-Certainty 2026.04		18