Program Synthesis on APPS 1.0 (test)
25.61Pass@5 (Introductory)
CodeRL+CodeT5
Updated 4d ago
Evaluation Results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
2022.07 | 25.61 | 17.17 | 6.78 | 4.88 | 8.48 | 9.53 | 8.91 | 12.62 | - | - | - | - | |
2022.07 | 24.52 | 22.78 | 2.64 | 3.04 | 6.75 | 3.23 | 3.08 | 7.46 | - | - | - | - | |
2022.07 | 20.36 | - | - | - | - | 9.66 | 7.75 | 11.42 | - | - | - | - | |
2022.07 | 18.18 | - | - | - | - | 8.21 | 6.65 | 9.89 | - | - | - | - | |
2022.07 | 16.37 | 7.08 | 1.86 | 0.75 | 2.69 | 4.95 | 2.84 | 6.81 | 40 | 15.67 | 17.9 | 20.98 | |
2022.07 | 14.36 | - | - | - | - | 5.63 | 4.58 | 7.17 | - | - | - | - | |
2022.07 | 9.65 | 4.14 | 0.14 | 0.02 | 0.92 | 0.51 | 0.09 | 2.25 | 25.02 | 3.7 | 3.23 | 7.87 | |
2022.07 | 9.2 | 5.6 | 1 | 0.5 | 1.82 | 1.73 | 1 | 3.08 | 35.2 | 13.15 | 13.51 | 17.63 | |
2022.07 | 5.5 | 3.9 | 0.57 | 0 | 1.12 | 0.8 | 0 | 1.58 | 27.9 | 9.83 | 11.4 | 13.76 | |
2022.07 | 3.6 | 1.3 | 0.7 | 0 | 0.68 | 1.03 | 0 | 1.34 | 25 | 9.27 | 8.8 | 12.32 | |
2022.07 | 2.7 | 1 | 0.33 | 0 | 0.4 | 0.73 | 0 | 1.02 | - | - | - | - | |
2022.07 | - | - | - | - | - | - | - | - | 17.67 | 5.24 | 7.06 | 8.09 | |
2022.07 | - | 0.2 | 0.03 | 0 | 0.06 | - | - | - | - | - | - | - |