Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Task Generation on SWE OOD (held-out repositories)
Loading...
19.59
Utility
PROPEL (OOD)
9.45
12.0825
14.715
17.3475
Jun 10, 2026
Utility
Validity
Self-BLEU-3
Distinct-3
Top-Topic Coverage
Updated 1mo ago
Evaluation Results
Method
Method
Links
Utility
Validity
Self-BLEU-3
Distinct-3
Top-Topic Coverage
PROPEL (OOD)
Solver=27B
2026.06
19.59
75.13
89.8
9.3
71.8
Base (OOD)
Solver=27B
2026.06
9.84
83.56
81.1
8.5
60.5
Feedback
Search any
task
Search any
task