Tool selection

Benchmarks

Dataset Name	SOTA Method	Metric
MetaTool	PA-Tool	Similarity80.8	39	2mo ago
ToolAlpaca	MTA	Accuracy97.42	20	4mo ago
API-Bench	MTA	Accuracy96.3	20	4mo ago
SD Single-domain	MTA	Accuracy94.04	20	4mo ago
CD Cross-domain	MTA	Accuracy97.33	20	4mo ago
APIGen sampled (test)	TokMem (w/ adapt)	Tool Selection F1 (2 calls)99.4	15	4mo ago
ScienceQA	Vicuna-7B	P(Use)99.29	14	1mo ago
BFCL	OLIVIA	F1 Score36.6	14	2mo ago
TaskBench-MM	OLIVIA	F1 Score20.7	12	2mo ago
TaskBench	CLIN	F1 Score21.1	12	2mo ago
ToolBench	OLIVIA	F1 Score58.5	12	2mo ago
Chess Skill: beginner, intermediate, advanced		Accuracy100	10	4mo ago
Chess Specialists: opening, midgame, endgame, late-endgame		Accuracy64.4	10	4mo ago
EnterpriseBench		Tool Selection Accuracy24	8	4mo ago
EnterpriseArena		Tool Selection Accuracy45	8	4mo ago
MetaTool similar choices subtask (test)	OATS-S1	Accuracy83.4	8	4mo ago
MetaTool 199 tools, 1,287 queries (30% test)	OATS-S1	R@183	7	4mo ago
ToolBench 30% 2,413 tools, 180 queries (test)		Recall@139.2	7	4mo ago
X Octopus out-of-domain v2 (test)	L_PORTS	Recall@197.5	4	17d ago
X ToolE out-of-domain (test)	L_PORTS	Recall@189.87	4	17d ago
Octopus v2 (test)	L_PORTS	Recall@197.5	4	17d ago
ToolE (test)	L_PORTS	Recall@167.35	4	17d ago
BFCL v2 (test)	L_PORTS	Recall@167.2	4	17d ago
APIBench (test)	L_PORTS	Recall@130.64	4	17d ago
API-Bank (test)	L_PORTS	Recall@159.12	4	17d ago

Showing 25 of 31 rows