Terminal-bench
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Terminal task completion | Terminal-bench 2.0 | Pass@164.7 | 90 | |
| Agentic terminal task resolution | Terminal-Bench 2 | SSA Pass@1 Rate83.14 | 34 | |
| Terminal-based agent interaction | Terminal-Bench 2.0 | Success Rate (SR)59.3 | 32 | |
| Code Agent Task | Terminal-Bench 2.0 | TB 2.0 Score82 | 27 | |
| Interactive Terminal Task | Terminal-Bench 2.0 (full) | Pass@131.9 | 19 | |
| Terminal-based task execution | Terminal-Bench 2.0 | Accuracy64.7 | 19 | |
| Terminal interaction agent task | Terminal-Bench (dev) | TB-dev Score37.4 | 18 | |
| Agentic Coding | Terminal Bench 2.0 | Pass@159.1 | 18 | |
| Agentic Terminal Interaction | Terminal-Bench v1 | Success Rate47.5 | 17 | |
| Terminal task completion | Terminal-bench 1.0 | Pass@151 | 17 | |
| Software Engineering Reasoning | Terminal-Bench (TB2) 2.0 | Resolution Rate36 | 16 | |
| Terminal Task Execution | Terminal-Bench 2.0 | Success Rate59.6 | 15 | |
| End-to-end terminal tasks | Terminal-Bench 2 | Score49.6 | 13 | |
| Terminal-based problem solving | Terminal-Bench 2 (out-of-distribution) | Task Success Rate34.12 | 12 | |
| Terminal Task Execution | Terminal-Bench 1.0 | Accuracy51 | 12 | |
| Terminal Capability Evaluation | Terminal-Bench 2.0 | Accuracy27.4 | 12 | |
| Agent | Terminal-Bench | Accuracy45 | 12 | |
| Code Agent | Terminal-Bench Hard | Score57.6 | 12 | |
| Terminal task | Terminal Bench Pro | Pass@137 | 11 | |
| Terminal task | Terminal Bench 1.0 | Pass@133.44 | 11 | |
| Agentic | Terminal Bench 2.0 | Pass@148.31 | 11 | |
| Coding | Terminal-Bench 2.0 | Score59.3 | 11 | |
| Coding | Terminal-Bench 2.0 | Success Rate (SR)65.2 | 10 | |
| Agentic Terminal Tasks | Terminal-Bench (TB) (test) | Success Rate48.75 | 10 | |
| Coding | Terminal-Bench 1.1 | Resolved Rate43 | 9 |
Showing 25 of 66 rows