Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Large-Scale Terminal Agentic Trajectory Generation from Dockerized Environments

About

Training agentic models for terminal-based tasks critically depends on high-quality terminal trajectories that capture realistic long-horizon interactions across diverse domains. However, constructing such data at scale remains challenging due to two key requirements: \textbf{\emph{Executability}}, since each instance requires a suitable and often distinct Docker environment; and \textbf{\emph{Verifiability}}, because heterogeneous task outputs preclude unified, standardized verification. To address these challenges, we propose \textbf{TerminalTraj}, a scalable pipeline that (i) filters high-quality repositories to construct Dockerized execution environments, (ii) generates Docker-aligned task instances, and (iii) synthesizes agent trajectories with executable validation code. Using TerminalTraj, we curate 32K Docker images and generate 50,733 verified terminal trajectories across eight domains. Models trained on this data with the Qwen2.5-Coder backbone achieve consistent performance improvements on TerminalBench (TB), with gains of up to 20\% on TB~1.0 and 10\% on TB~2.0 over their respective backbones. Notably, \textbf{TerminalTraj-32B} achieves strong performance among models with fewer than 100B parameters, reaching 35.30\% on TB~1.0 and 22.00\% on TB~2.0, and demonstrates improved test-time scaling behavior. All code and data are available at https://github.com/Wusiwei0410/TerminalTraj.

Siwei Wu, Yizhi Li, Yuyang Song, Wei Zhang, Yang Wang, Riza Batista-Navarro, Xian Yang, Mingjie Tang, Bryan Dai, Jian Yang, Chenghua Lin• 2026

Related benchmarks

TaskDatasetResultRank
Terminal task completionTerminal-bench 2.0
Pass@123.88
90
Terminal-related CLI agent taskTerminalBench 1.0
Accuracy35.3
51
Terminal Agentic Trajectory GenerationTerminalBench 2.0
Score22
29
Code Agent TaskTerminal-bench 2.0
TB 2.0 Score22
27
Terminal Agent TasksTerminalBench 2.0
Success Score22
25
Terminal Agentic Trajectory GenerationTerminalBench 1.0
Score35.3
23
Terminal taskTerminal-bench 1.0
Pass@133.44
11
Terminal taskTerminal Bench Pro
Pass@130.5
11
Terminal Agentic Trajectory GenerationTerminalBench
Pass@845
4
Showing 9 of 9 rows

Other info

GitHub

Follow for update