Tool Calling

Benchmarks

Dataset Name	SOTA Method	Metric
API-Bank L-1	HiTEC-KTO	F1 Name Match94.99	46	1mo ago
Supply Chain Tool Calling 1.0 (test)		Accuracy86.73	38	1mo ago
Tool-Alpaca	Llama3-70B	Tool Name Accuracy91.17	31	1mo ago
Seal-Tools Single-Tool	Llama3-70B	Name Match Score98.14	30	1mo ago
SupChain-Bench	SupChain-ReAct	Accuracy75.51	27	1mo ago
API-Bank L-2	HiTEC-KTO	Name Match F190.42	25	1mo ago
ACEBench Extended Setting		Overall Score65.17	18	1mo ago
BFCL Extended Setting		Non-Live Score85.81	18	1mo ago
ACEBench Standard Setting	ToolGT (Prompting)	Overall Score68.92	18	1mo ago
BFCL Standard Setting		Non-Live Accuracy86.46	18	1mo ago
DIABENCH Dynamic Evaluation 1.0	Llama-3.3-Nemotron-DiaFORGE-49B	ACC89	17	5d ago
DIABENCH Static Evaluation 1.0	Llama-3.3-Nemotron-DiaFORGE-49B	Accuracy82	17	5d ago
BFCL		Non-Live Success Rate90.65	17	1mo ago
F1 Average	Llama3-70B	Tool Call Name F191.37	16	1mo ago
Nexus Raven	Llama3-70B	Score (Name)94.84	16	11d ago
BFCL Multiple		Accuracy92.5	12	1mo ago
Nexus Raven v1 (test)	HiTEC-KTO	F1 Name94.84	12	1mo ago
Seal-Tools Single-Tool v1 (test)	HiTEC-KTO	F1 Name98.14	12	1mo ago
Tool-Alpaca v1 (test)	HiTEC-KTO	F1 Name87.63	12	1mo ago
API-Bank L-2 v1 (test)	HiTEC-KTO	F1 Name Match88	12	1mo ago
API-Bank L-1 v1 (test)	HiTEC-KTO	F1 Score90.78	12	1mo ago
BFCL out-of-domain	TInR-U	Exact Match (EM)26	10	4d ago
In-domain (unseen)	TInR-U	Exact Match (EM)57.24	10	4d ago
In-domain seen	TInR-U	EM74.05	10	4d ago
Average across 5 benchmarks	HiTEC-ICL	F1 (Name)88.47	9	1mo ago

Showing 25 of 42 rows