Tool Calling

Benchmarks

Dataset Name	SOTA Method	Metric
Confetti		AST Soft Accuracy72.7	49	19d ago
API-Bank L-1	HiTEC-KTO	F1 Name Match94.99	46	20d ago
When2Call		F1 Score76.8	42	19d ago
Supply Chain Tool Calling 1.0 (test)		Accuracy86.73	38	3mo ago
Tool-Alpaca	Llama3-70B	Tool Name Accuracy91.17	31	3mo ago
Seal-Tools Single-Tool	Llama3-70B	Name Match Score98.14	30	3mo ago
SupChain-Bench	SupChain-ReAct	Accuracy75.51	27	3mo ago
API-Bank L-2	HiTEC-KTO	Name Match F190.42	25	3mo ago
ACEBench Extended Setting		Overall Score65.17	18	2mo ago
BFCL Extended Setting		Non-Live Score85.81	18	2mo ago
ACEBench Standard Setting	ToolGT (Prompting)	Overall Score68.92	18	2mo ago
BFCL Standard Setting		Non-Live Accuracy86.46	18	2mo ago
BFCL (Berkeley Function Calling Leaderboard)	Llama-3.1-8B-Instruct	Single-Turn Non-Live Success Rate85.2	17	23d ago
DIABENCH Dynamic Evaluation 1.0	Llama-3.3-Nemotron-DiaFORGE-49B	ACC89	17	1mo ago
DIABENCH Static Evaluation 1.0	Llama-3.3-Nemotron-DiaFORGE-49B	Accuracy82	17	1mo ago
BFCL		Non-Live Success Rate90.65	17	2mo ago
BFCL Live	ParaTool (ours)	Multiple Success Rate79.01	16	5d ago
BFCL Non-live	ParaTool (ours)	Multiple Success Rate96.67	16	5d ago
Stable Toolbench I3-Inst	ParaTool	Pass Rate68.85	16	5d ago
Stable Toolbench I2-Cat	ParaTool	Pass Rate78.3	16	5d ago
Stable Toolbench I2-Inst	ParaTool	Pass Rate77.42	16	5d ago
Stable Toolbench I1-Tool	ParaTool	Pass Rate75.95	16	5d ago
Stable Toolbench I1-Cat	ParaTool	Pass Rate76.07	16	5d ago
Stable Toolbench I1-Inst	ParaTool	Pass Rate0.7909	16	5d ago
F1 Average	Llama3-70B	Tool Call Name F191.37	16	3mo ago

Showing 25 of 68 rows