Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Terminal Agent Tasks on val100 (val)
Loading...
73.1
Pass@1
OT-SFT
32.644
43.147
53.65
64.153
May 23, 2026
Pass@1
Updated 2mo ago
Evaluation Results
Method
Method
Links
Pass@1
OT-SFT
Setup=ECHO
2026.05
73.1
Qwen3-14B
Setup=ECHO
2026.05
65
Qwen3-8B
Setup=ECHO
2026.05
63.7
OT-SFT
Setup=GRPO
2026.05
63.5
Qwen3-14B
Setup=GRPO
2026.05
60.3
Qwen3-8B
Setup=GRPO
2026.05
54.9
OT-SFT
Setup=SFT
2026.05
38.5
Qwen3-14B
Setup=Base
2026.05
35.3
Qwen3-8B
Setup=Base
2026.05
34.2
Feedback
Search any
task
Search any
task
