Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills
About
Large Language Model (LLM) agents increasingly rely on domain-specific skills, yet manually authoring such skills does not scale, and skills generated purely from parametric knowledge often miss critical operational pitfalls. We introduce Trace2Skill, a framework that consolidates broad execution trajectories in parallel into a unified skill directory through inductive reasoning over agent experience. Trace2Skill supports both deepening existing human-written skills and creating useful skills from weak LLM-generated drafts. Experiments demonstrate the effectiveness of Trace2Skill across diverse domains, including office workflows, math reasoning, and vision QA. Importantly, the evolved skills are not merely memorized artifacts of the trajectories used to create them: they often transfer across model scales, across model families, and to out-of-distribution settings. For example, skills evolved from Qwen3.5-35B trajectories improve a Qwen3.5-122B agent by up to $57.65$ percentage points on WikiTableQuestions. Further analyses show that Trace2Skill outperforms sequential skill editing and ReasoningBank-style retrieval memories, compresses recurring failures and workarounds into standard operating procedures (SoPs), and yields portable skills that can be reused without parameter updates or test-time retrieval.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Interactive Decision-making | AlfWorld | Overall Success Rate71.3 | 398 | |
| Multi-hop Question Answering | HotpotQA (test) | -- | 334 | |
| Document Visual Question Answering | DocVQA (test) | ANLS90.6 | 292 | |
| Function Calling | BFCL Multi-Turn v3 | Overall Accuracy72.7 | 69 | |
| Mathematical Problem Solving | LiveMath held-out (test) | Score52 | 59 | |
| Office Application Question Answering | OfficeQA held-out (test) | Score (%)65.7 | 59 | |
| Spreadsheet Manipulation and Reasoning | SpreadsheetBench held-out (test) | Score53.2 | 59 | |
| Web Search Question Answering | SearchQA held-out (test) | Score (%)82.4 | 59 | |
| Embodied decision-making | ALFWorld held-out (test) | Score87.3 | 49 | |
| Question Answering | SearchQA | Accuracy68.5 | 43 |