CodeXGLUE
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Code Summarization | CodeXGLUE | Java Score20.83 | 38 | |
| Defect Detection | CodeXGLUE | Defect Rate0.534 | 21 | |
| Code Repair | CodeXGLUE | Repair Rate15.7 | 21 | |
| Clone Detection | CodeXGLUE | Clone Score95.7 | 21 | |
| Defect Detection | CodeXGLUE Defect Detection (test) | # Identifiers3.92 | 18 | |
| Clone Detection | CodeXGLUE Clone Detection | ASR83.19 | 18 | |
| Defect Detection | CodeXGLUE Defect Detection | ASR99.89 | 18 | |
| Vulnerability Detection | CodeXGLUE (test) | Accuracy65.8 | 18 | |
| Code Summarization | CodeXGLUE Code-Text | BLEU-413.21 | 12 | |
| Code Summarization | CodeXGLUE Java (test) | BLEU-422.95 | 11 | |
| Code Summarization | CodeXGLUE Python (test) | BLEU-421.01 | 11 | |
| Docstring Generation | CodeXGLUE Python (test) | BLEU21.99 | 11 | |
| Code Generation | CodeXGLUE Python (test) | CodeBLEU31.98 | 10 | |
| Code Refinement | CodeXGLUE Code Refinement (Medium) v1.0 (test) | BLEU-491.31 | 10 | |
| Code Refinement | CodeXGLUE Code Refinement Small v1.0 (test) | BLEU-480.02 | 10 | |
| Text-to-Code Retrieval | CodeXGLUE AdvTest | MRR44.7 | 9 | |
| Text-to-Code generation | CodeXGLUE (test) | EM22.65 | 9 | |
| Code Generation | CodeXGLUE Java (test) | CodeBLEU38.12 | 8 | |
| Code Clone Detection | CodeXGLUE Clone Detection (test) | Recall96.7 | 8 | |
| Code Translation | CodeXGLUE C# to Java v1.0 (test) | BLEU-479.87 | 8 | |
| Code Translation | CodeXGLUE Java to C# v1.0 (test) | BLEU-4 Score84.03 | 8 | |
| Code-to-Text generation | CodeXGLUE (test) | Python20.57 | 8 | |
| Code Summarization | CodeXGLUE (test) | Overall Score20.63 | 7 | |
| Code Understanding and Generation | CodeXGLUE | Code Refinement Score46.3 | 6 | |
| Code Defect Detection | CodeXGLUE Defect Detection (test) | Accuracy66.7 | 6 |
Showing 25 of 27 rows