Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instruction

About

To address intricate real-world tasks, there has been a rising interest in tool utilization in applications of large language models (LLMs). To develop LLM-based agents, it usually requires LLMs to understand many tool functions from different tool documentation. But these documentations could be diverse, redundant or incomplete, which immensely affects the capability of LLMs in using tools. To solve this, we introduce EASYTOOL, a framework transforming diverse and lengthy tool documentation into a unified and concise tool instruction for easier tool usage. EasyTool purifies essential information from extensive tool documentation of different sources, and elaborates a unified interface (i.e., tool instruction) to offer standardized tool descriptions and functionalities for LLM-based agents. Extensive experiments on multiple different tasks demonstrate that EasyTool can significantly reduce token consumption and improve the performance of tool utilization in real-world scenarios. Our code will be available at \url{https://github.com/microsoft/JARVIS/} in the future.

Siyu Yuan, Kaitao Song, Jiangjie Chen, Xu Tan, Yongliang Shen, Ren Kan, Dongsheng Li, Deqing Yang• 2024

Related benchmarks

TaskDatasetResultRank
Tool CallingAPI-Bank L-1--
46
Tool RetrievalToolBench
NDCG@1055.26
44
Tool RetrievalGorilla
NDCG@100.3598
44
Tool RetrievalToolink
NDCG@100.5378
44
Tool RetrievalAPIGen
NDCG@100.7953
44
Tool RetrievalAPIBank
NDCG@1066.29
44
Tool RetrievalMixed
NDCG@100.4668
44
Function CallingBFCL Individual Tools per Problem
Execution Accuracy76
30
Tool UseToolBench
Average Pass Rate52.97
29
Tool UseStableToolBench
I2 Category Success70.6
28
Showing 10 of 21 rows

Other info

Follow for update