Share your thoughts, 1 month free Claude Pro on usSee more

Function Calling on BFCL V4 (Custom Metrics Set)

62.3Multi-Turn Success Rate

Seed 2.0

Updated 1mo ago

Evaluation Results

Method
Seed 2.0 2026.04	62.3	92	57.8	89	82.2	76.6	75	73.4
Claude Sonnet-4.5 2026.04	61.4	81	65	88.7	81.1	68.8	86.6	73.2
Gemini-3 Pro 2026.04	60.8	80	61.7	90.7	83.1	68.8	85.6	72.5
Agent-World-14B 2026.04	53.9	53	23.9	82.3	79.3	93.8	81	55.8
4B-Thinking 2026.06	52	6	12.37	38.02	71.06	-	79.23	38.61
235B-A22B-Instruct 2026.06	50.12	44.5	27.1	90.1	83.2	-	82.38	53.54
Turn-level TRUST 2026.06	49.62	3	24.95	89.23	84.09	-	84.84	48.04
GPT-5.2 High 2026.04	48.5	75.5	45.8	81.9	70.4	75	88.7	62.9
EnvScaler-8B 2026.04	47.1	23	21.9	88.5	82.2	93.8	74.6	47.6
Qwen3-235B-A22B 2026.04	45.4	54	23.9	37.4	68.9	87.5	81.7	47.9
MAESTRO 2026.05	44.62	-	-	87.33	82.38	-	-	78.09
Agent-World-8B 2026.04	44.5	47	21.7	83.3	79.6	93.8	80.2	51.4
GPT-5.2 2026.05	43.75	-	-	78.29	67.14	-	-	68.58
Qwen3-32B 2026.04	43.3	26	15.7	90.3	82	81.3	82.4	46.7
SAGE 2026.06	41.5	3.5	13.02	86.71	80.49	-	79.23	41.16
8B-Thinking 2026.06	37.75	12	14.62	87.58	80.53	-	79.07	41.58
AWM-14B 2026.04	37.6	10	19.8	90.2	81.5	75	79.4	42.4
DeepSeek-V3.2-685B 2026.04	37.4	69.5	54.2	34.9	53.7	37.5	93.2	54.1
Qwen3-14B 2026.04	36.9	4	19.8	90	82.4	81.3	79.4	41
SAGE 2026.06	36.37	9.5	21.72	88.15	80.83	-	79.6	42.99
Gemini-2.5-Flash 2026.05	36.25	-	-	84.96	74.39	-	-	72.88
Qwen3-8B 2026.04	35.4	7	17.6	90.2	80.9	81.3	77.2	40.4
AWM-8B 2026.04	34.9	9.5	15.7	90.2	80.5	93.8	73.9	40
30B-A3B-Instruct 2026.06	30	22.5	17.63	85.77	77.94	-	79.9	41
Traj.-level TRUST 2026.06	29.88	27	30.57	82.27	75.57	-	79.33	44.48
AUQ 2026.06	23.5	3	20.43	77.08	62.4	-	81.37	35.21
Turn-level GRPO 2026.06	22.75	3	24.52	80.61	86.33	-	82.38	38.98
Traj.-level CM2 2026.06	22.5	22	21.94	80.12	74.58	-	71.46	38.15
TOUCAN-7B 2026.04	17.8	21	18.5	81	73.9	81.3	78.6	36.6
Claude-Opus-4.5 2026.05	16.12	-	-	89.65	76.02	-	-	72.14
AUQ 2026.06	10	14	15.7	84.46	75.42	-	40.14	29.08
Simulator-8B 2026.04	4.1	17.5	6	47.6	44.6	31.3	87.3	23.9