Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

TOUCAN: Synthesizing 1.5M Tool-Agentic Data from Real-World MCP Environments

About

Large Language Model (LLM) agents are rapidly emerging as powerful systems for automating tasks across domains. Yet progress in the open-source community is constrained by the lack of high quality permissively licensed tool-agentic training data. Existing datasets are often limited in diversity, realism, and complexity, particularly regarding multi-tool and multi-turn interactions. To address this gap, we introduce Toucan, the largest publicly available tool-agentic dataset to date, containing 1.5 million trajectories synthesized from nearly 500 real-world Model Context Protocols (MCPs). Unlike prior work, Toucan leverages authentic MCP environments to generate diverse, realistic, and challenging tasks with trajectories involving real tool execution. Our pipeline first produces a broad spectrum of tool-use queries using five distinct models, applies model-based quality filtering, and then generates agentic trajectories with three teacher models using two agentic frameworks. Rigorous rule-based and model-based validation ensures high-quality outputs. We also introduce three extension mechanisms to further diversify tasks and simulate multi-turn conversations. Models fine-tuned on Toucan outperform larger closed-source counterparts on the BFCL V3 benchmark and push the Pareto frontier forward on MCP-Universe Bench.

Zhangchen Xu, Adriana Meza Soria, Shawn Tan, Anurag Roy, Ashish Sunil Agrawal, Radha Poovendran, Rameswar Panda• 2025

Related benchmarks

TaskDatasetResultRank
Tool UseBFCL Multi-turn
Accuracy37.03
24
Tool-augmented ReasoningBFCL Multi-Turn v3
Overall Score22.6
14
Tool UseTau-Bench
TAU-AIR Score33.5
14
Multi-Turn Tool Callingτ2-bench
Overall Score17.77
5
Coding AgentCodeCI
Avg@237.71
5
Coding AgentRebenchT
OH-p@128.75
5
Coding AgentAggregated (RebenchT, CodeCI, Bird)
Overall Average Score29.82
5
Coding AgentBird
Pass@132.89
5
Showing 8 of 8 rows

Other info

Follow for update