| Dataset Name | SOTA Method | Metric | Trend | ||
|---|---|---|---|---|---|
| HumanEval | MATRIX-Gen-SFT | Pass@17,927 | 1,048 | 1mo ago | |
| HumanEval (test) | Pass@1100 | 701 | 18d ago | ||
| MBPP (test) | AgentConductor | Pass@195.1 | 411 | 22d ago | |
| HumanEval+ | Pass@1100 | 393 | 1mo ago | ||
| HumanEval | Qwen3 Model | pass@196.95 | 329 | 26d ago | |
| MBPP+ | G-Memory | Accuracy85.75 | 243 | 1mo ago | |
| Code | M2CL | Accuracy98.7 | 242 | 4mo ago | |
| MBPP+ | Pass@184.39 | 238 | 1mo ago | ||
| HumanEval | Agent Q-Mix | Accuracy97.56 | 224 | 1mo ago | |
| HumanEval | SASFT | Accuracy98.27 | 212 | 1mo ago | |
| MBPP | Uno-Orchestra | Pass@192.4 | 211 | 2mo ago | |
| MBPP | Pass@189.1 | 193 | 4mo ago | ||
| HumanEval | ExPairT-LLM | Pass@194.1 | 171 | 4mo ago | |
| MBPP | GPT-4o | Accuracy79.8 | 165 | 2mo ago | |
| HumanEval | DFlash+DDTree | Speedup Factor8.22 | 147 | 1mo ago | |
| MBPP | MegaAgent | Accuracy (%)92.2 | 146 | 4mo ago | |
| HumanEval 1.0 (test) | Pass@185.4 | 145 | 4mo ago | ||
| HumanEval | SIGMA | HumanEval Score95.22 | 140 | 1mo ago | |
| HumanEval+ (test) | SKETCHVERIFY (B: sem. vote) | Pass@198.1 | 132 | 1mo ago | |
| HumanEval+ | Qwen2.5-Coder-7B-Inst | Pass@186 | 122 | 22d ago | |
| EvalPlus | EVOLVECODER-4B (r3) | Pass@189 | 118 | 1mo ago | |
| APPS | ExPairT-LLM | Pass@191.2 | 111 | 2mo ago | |
| HumanEval-ET | ThinkCoder | Pass@189.6 | 108 | 2mo ago | |
| HumanEval 0-shot | Accuracy57.93 | 100 | 29d ago | ||
| MBPP | BASTION | Average Acceptance Length (τ)8.99 | 95 | 24d ago |
