Python
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Secure Code Generation | Python (test) | Safe Code Rate94.4 | 66 | |
| Graphical API Recommendation | Python 13 | mAP83.4 | 56 | |
| Speculative Decoding | Python | MAT7.69 | 30 | |
| Code Search | Python (test) | Recall@149.4 | 25 | |
| Code Summarization Factual Consistency | Python | Pearson Correlation (rp)0.497 | 15 | |
| Adversarial Code Compliance | Python (Py) | Decoupling Probability99.4 | 9 | |
| Code Generation | Python deduplicated retrieval codebase (test) | EM1,183 | 9 | |
| Code Comment Generation | Python (test) | BLEU21.01 | 8 | |
| Code Generation | Python 10 sequential 20-step campaigns (end-of-chain) | End pass@125.9 | 5 | |
| Code Search | Python | MRR85.63 | 5 | |
| Code Generation | Python (held-out eval) | Pass@137.7 | 4 | |
| Package-level evaluation | Python Full (val) | Precision92.4 | 4 | |
| RRT-Connect Planning | Python 2D | Wall-Clock Time (ms)4.4 | 2 |
Showing 13 of 13 rows