Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesis

About

Large language models (LLMs) are expected to be trained to act as agents in various real-world environments, but this process relies on rich and varied tool-interaction sandboxes. However, access to real systems is often restricted; LLM-simulated environments are prone to hallucinations and inconsistencies; and manually built sandboxes are hard to scale. In this paper, we propose EnvScaler, an automated framework for scalable tool-interaction environments via programmatic synthesis. EnvScaler comprises two components. First, SkelBuilder constructs diverse environment skeletons through topic mining, logic modeling, and quality evaluation. Then, ScenGenerator generates multiple task scenarios and rule-based trajectory validation functions for each environment. With EnvScaler, we synthesize 191 environments and about 7K scenarios, and apply them to Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) for Qwen3 series models. Results on three benchmarks show that EnvScaler significantly improves LLMs' ability to solve tasks in complex environments involving multi-turn, multi-tool interactions. We release our code and data at https://github.com/RUC-NLPIR/EnvScaler.

Xiaoshuai Song, Haofei Chang, Guanting Dong, Yutao Zhu, Ji-Rong Wen, Zhicheng Dou• 2026

Related benchmarks

TaskDatasetResultRank
Function CallingBFCL V3
Overall Accuracy54.06
104
Interactive Tool-Use Agent Performancetau2-Bench
Retail Performance Score49.1
102
Function CallingBFCL Multi-Turn v3
Overall Accuracy41.88
69
Agentic Tool-usetau2-Bench
Retail Score49.6
59
Agent PerformanceTau-Bench
Retail Accuracy55.7
55
Interactive Tool-Use Agent PerformanceVitaBench
Delivery Score25
44
Agentic Workflow Successτ2-bench
Airline Success Rate36
43
Agent PerformanceACEBench Agent
Agent Score70
36
Tool UseMCPMark
Total Success Rate5.6
26
Medical Agent Task ExecutionMedAgentBench
Success Rate56.6
24
Showing 10 of 33 rows

Other info

Follow for update