SWE-Gym
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Software Engineering Task Resolution | SWE-Gym (Holdout) | Task Resolution Rate19 | 12 | |
| Failure prediction | SWE-Gym (Transfer held-out) | ROC-AUC60 | 4 | |
| Failure prediction | SWE-Gym (Within corpus) | ROC-AUC85.3 | 4 | |
| Leave-one-corpus-out transfer | SWE-Gym | AUC59.9 | 3 |
Showing 4 of 4 rows