CodeSearchNet
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Code Summarization | CodeSearchNet-Java (CSN) CodeXGLUE (test) | Smoothed BLEU-420.55 | 38 | |
| Code Generation | CodeSearchNet (test) | BLEU0.8789 | 26 | |
| Natural Language Code Search | CodeSearchNet | Overall Score82.3 | 25 | |
| NL2Code Search | CSN (CodeSearchNet) (test) | Recall (Python)86.85 | 18 | |
| Text-to-Code Retrieval | CodeSearchNet CodeXGLUE | Ruby Score78 | 9 | |
| Natural Language Code Search | CodeSearchNet filtered and expanded (test) | MRR (Ruby)0.703 | 8 | |
| Code-to-Documentation generation | CodeSearchNet (test) | Ruby12.16 | 7 | |
| Extreme Code Summarization | CodeSearchNet JavaScript (test) | Precision20.34 | 5 | |
| Language Modeling | CodeSearchNet Python (test) | Perplexity (PPL)8.1 | 3 | |
| Code Search | CodeSearchNet Python hard subset (test) | MRR51.13 | 3 |
Showing 10 of 10 rows