Tool Calling

Benchmarks

Dataset Name	SOTA Method	Metric
Confetti		AST Soft Accuracy72.7	49	2mo ago
API-Bank L-1	HiTEC-KTO	F1 Name Match94.99	46	2mo ago
When2Call		F1 Score76.8	42	2mo ago
Supply Chain Tool Calling 1.0 (test)		Accuracy86.73	38	4mo ago
Tool-Alpaca	Llama3-70B	Tool Name Accuracy91.17	31	4mo ago
Seal-Tools Single-Tool	Llama3-70B	Name Match Score98.14	30	4mo ago
SupChain-Bench	SupChain-ReAct	Accuracy75.51	27	4mo ago
API-Bank L-2	HiTEC-KTO	Name Match F190.42	25	4mo ago
BFCL (Berkeley Function Calling Leaderboard)	Qwen3-14B	BFCL Score85.3	18	1mo ago
ACEBench Extended Setting		Overall Score65.17	18	4mo ago
BFCL Extended Setting		Non-Live Score85.81	18	4mo ago
ACEBench Standard Setting	ToolGT (Prompting)	Overall Score68.92	18	4mo ago
BFCL Standard Setting		Non-Live Accuracy86.46	18	4mo ago
DIABENCH Dynamic Evaluation 1.0	Llama-3.3-Nemotron-DiaFORGE-49B	ACC89	17	3mo ago
DIABENCH Static Evaluation 1.0	Llama-3.3-Nemotron-DiaFORGE-49B	Accuracy82	17	3mo ago
BFCL		Non-Live Success Rate90.65	17	4mo ago
BFCL Live	ParaTool (ours)	Multiple Success Rate79.01	16	1mo ago
BFCL Non-live	ParaTool (ours)	Multiple Success Rate96.67	16	1mo ago
Stable Toolbench I3-Inst	ParaTool	Pass Rate68.85	16	1mo ago
Stable Toolbench I2-Cat	ParaTool	Pass Rate78.3	16	1mo ago
Stable Toolbench I2-Inst	ParaTool	Pass Rate77.42	16	1mo ago
Stable Toolbench I1-Tool	ParaTool	Pass Rate75.95	16	1mo ago
Stable Toolbench I1-Cat	ParaTool	Pass Rate76.07	16	1mo ago
Stable Toolbench I1-Inst	ParaTool	Pass Rate0.7909	16	1mo ago
F1 Average	Llama3-70B	Tool Call Name F191.37	16	4mo ago

Showing 25 of 75 rows