ToolBench
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Tool-use | ToolBench | Average Success Rate (ASR)99.61 | 62 | |
| Tool-use | ToolBench | Average Pass Rate80.67 | 53 | |
| Tool Retrieval | ToolBench | NDCG@1058.54 | 44 | |
| Function Calling | ToolBench Average | Pass Rate75.95 | 30 | |
| Tool Retrieval | ToolBench In-domain I1 | NDCG@193.76 | 29 | |
| Tool-use Reasoning | ToolBench | API Success Rate88 | 28 | |
| Tool use | ToolBench (test) | Pass@183.7 | 28 | |
| Tool Planning | ToolBench | EM (%)42.6 | 24 | |
| Invocation attack | ToolBench | CDA98 | 24 | |
| Tool Reasoning | ToolBench (G3) | Pass Rate91.8 | 24 | |
| Tool Reasoning | ToolBench G2 | Pass Rate93 | 24 | |
| Tool Reasoning | ToolBench (G1) | Pass Rate85.5 | 24 | |
| Tool-use API Generalization | ToolBench G3 | Pass Rate71.5 | 22 | |
| Tool-use API Generalization | ToolBench G2 | Pass Rate78.2 | 22 | |
| Tool-use API Generalization | ToolBench G1 v1 | Pass Rate83.5 | 22 | |
| Tool Retrieval | ToolBench In-domain (I3) | NDCG@191.74 | 20 | |
| Tool Retrieval | ToolBench In-domain (I2) | NDCG@191.91 | 20 | |
| Tool Use | ToolBench | Energy (Wh)5.6 | 18 | |
| Throughput Efficiency | ToolBench | Throughput (tokens/s)4,602 | 18 | |
| LLM Inference | ToolBench | Goodput (req/s)3.9 | 18 | |
| Agent Task | ToolBench | Success Rate44.98 | 16 | |
| End-to-end Tool-use | ToolBench I1 v1 | SoPR56.13 | 16 | |
| Function Calling | ToolBench I3-Inst | Pass Rate52.4 | 14 | |
| Function Calling | ToolBench I2-Inst | Pass Rate71.4 | 14 | |
| Function Calling | ToolBench I1-Inst | Pass Rate57.1 | 14 |
Showing 25 of 90 rows