Role Consistency on SWE Dev hard (test)
6.8Overstepping Rate (<INFO>)
ChatDev (Joint FT)
Updated 13d ago
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
2026.04 | 6.8 | 7.6 | -42.9 | -60.6 | |
2026.04 | 16.8 | 30.4 | -32.9 | -37.8 | |
2026.04 | 44 | 44 | -5.7 | -24.2 | |
2026.04 | 49.7 | 68.2 | - | - |
ChatDev (Joint FT)
| Method | Links | ||||
|---|---|---|---|---|---|
2026.04 | 6.8 | 7.6 | -42.9 | -60.6 | |
2026.04 | 16.8 | 30.4 | -32.9 | -37.8 | |
2026.04 | 44 | 44 | -5.7 | -24.2 | |
2026.04 | 49.7 | 68.2 | - | - |