Role Consistency on SWE dev full set (test)
8.4Total Overstepping Rate (<INFO>)
ChatDev (Joint FT)
Updated 13d ago
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
2026.04 | 8.4 | 9 | -38 | -58 | |
2026.04 | 18 | 32.4 | -28.4 | -34.6 | |
2026.04 | 40.6 | 40.8 | -5.8 | -26.2 | |
2026.04 | 46.4 | 67 | - | - |
ChatDev (Joint FT)
| Method | Links | ||||
|---|---|---|---|---|---|
2026.04 | 8.4 | 9 | -38 | -58 | |
2026.04 | 18 | 32.4 | -28.4 | -34.6 | |
2026.04 | 40.6 | 40.8 | -5.8 | -26.2 | |
2026.04 | 46.4 | 67 | - | - |