Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

On Data Engineering for Scaling LLM Terminal Capabilities

About

Despite rapid recent progress in the terminal capabilities of large language models, the training data strategies behind state-of-the-art terminal agents remain largely undisclosed. We address this gap through a systematic study of data engineering practices for terminal agents, making two key contributions: (1) Terminal-Task-Gen, a lightweight synthetic task generation pipeline that supports seed-based and skill-based task construction, and (2) a comprehensive analysis of data and training strategies, including filtering, curriculum learning, long context training, and scaling behavior. Our pipeline yields Terminal-Corpus, a large-scale open-source dataset for terminal tasks. Using this dataset, we train Nemotron-Terminal, a family of models initialized from Qwen3(8B, 14B, 32B) that achieve substantial gains on Terminal-Bench 2.0: Nemotron-Terminal-8B improves from 2.5% to 13.0% Nemotron-Terminal-14B improves from 4.0% to 20.2%, and Nemotron-Terminal-32B improves from 3.4% to 27.4%, matching the performance of significantly larger models. To accelerate research in this domain, we open-source our model checkpoints and most of our synthetic datasets at https://huggingface.co/collections/nvidia/nemotron-terminal.

Renjie Pi, Grace Lam, Mohammad Shoeybi, Pooya Jannaty, Bryan Catanzaro, Wei Ping• 2026

Related benchmarks

TaskDatasetResultRank
Terminal task completionTerminal-bench 2.0
Pass@127.4
90
Software EngineeringSWE-bench Verified
Accuracy22.1
43
Code Agent TaskTerminal-bench 2.0
TB 2.0 Score27.4
27
Terminal Agent TasksTerminalBench 2.0
Success Score27.4
25
Software Engineering Issue SolvingSWE-bench Verified
Accuracy41.9
15
Terminal Task ExecutionTerminal-bench 2.0
Success Rate13.1
15
Financial Agent TaskFinanceAgent-Terminal
Accuracy (%)40.7
14
General AI Assistant TaskGAIA-127
Accuracy22.3
14
Function CallingBFCL Parity
Accuracy69.1
14
Terminal Capability EvaluationTerminal-bench 2.0
Accuracy27.4
12
Showing 10 of 21 rows

Other info

Follow for update