CodeSearchNet
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Code Summarization | CodeSearchNet-Java (CSN) CodeXGLUE (test) | Smoothed BLEU-420.55 | 38 | |
| Code Generation | CodeSearchNet (test) | BLEU0.8789 | 26 | |
| NL2Code Search | CSN (CodeSearchNet) (test) | Recall (Python)86.85 | 18 | |
| Text-to-Code Retrieval | CodeSearchNet CodeXGLUE | Ruby Score78 | 9 | |
| Natural Language Code Search | CodeSearchNet | Ruby73.2 | 8 | |
| Natural Language Code Search | CodeSearchNet filtered and expanded (test) | MRR (Ruby)0.703 | 8 | |
| Code-to-Documentation generation | CodeSearchNet (test) | Ruby12.16 | 7 | |
| Extreme Code Summarization | CodeSearchNet JavaScript (test) | Precision20.34 | 5 | |
| Code Search | CodeSearchNet Python hard subset (test) | MRR51.13 | 3 |
Showing 9 of 9 rows