Repository-level code completion on CrossCodeEval (test)
35.9Python EM
StarCoder (Teacher)
Updated 4d ago
Evaluation Results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
2024.07 | 35.9 | 66.1 | 41.5 | 72.9 | 38.7 | 73.7 | 56.3 | 79.3 | 43.1 | 73 | |
2024.07 | 31.7 | 62.2 | 34.6 | 69.8 | 33.2 | 69.3 | 50.9 | 76.2 | 37.6 | 69.4 | |
2024.07 | 26.5 | 53.2 | 32.4 | 61.1 | 31.6 | 64.5 | 48 | 69.8 | 34.6 | 61.2 | |
2024.07 | 24.8 | 49.3 | 31.6 | 61.5 | 30.5 | 63.7 | 47.1 | 68.4 | 33.5 | 60.7 | |
2024.07 | 20.8 | 41.5 | 25.3 | 51.4 | 25.7 | 56.2 | 40.5 | 60.5 | 28.1 | 52.4 |