Cross-file code completion on CrossCodeEval (test)
16.14Python EM
DeepSeek-Coder-Base
Updated 4d ago
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
2024.01 | 16.14 | 66.51 | 17.72 | 63.18 | 14.03 | 61.77 | 16.23 | 63.42 | |
2024.01 | 16.02 | 66.65 | 16.64 | 61.88 | 13.23 | 60.92 | 14.48 | 62.38 | |
2024.01 | 13.06 | 64.24 | 15.61 | 64.78 | 7.54 | 42.06 | 14.2 | 65.03 | |
2024.01 | 13.02 | 64.3 | 16.41 | 64.64 | 12.34 | 60.64 | 13.19 | 63.04 | |
2024.01 | 10.73 | 61.76 | 10.1 | 59.56 | 7.72 | 55.17 | 4.64 | 52.3 | |
2024.01 | 9.53 | 61.65 | 10.8 | 61.77 | 9.59 | 60.17 | 5.26 | 61.32 | |
2024.01 | 8.11 | 59.55 | 7.34 | 59.6 | 6.14 | 55.5 | 1.7 | 51.66 | |
2024.01 | 7.32 | 59.66 | 9.68 | 62.64 | 8.19 | 58.5 | 4.07 | 59.19 | |
2024.01 | 6.68 | 59.55 | 8.65 | 62.57 | 5.01 | 48.83 | 4.75 | 59.53 |