Share your thoughts, 1 month free Claude Pro on usSee more

Multi-turn Tool-use on BFCL multi-turn V3

68Average Success Rate

GLM-4.6 355B

Updated 1mo ago

Evaluation Results

Method
GLM-4.6 355B 2026.05	68	74.5	68	63	66.5
Claude-Sonnet-4.5 2026.05	61.38	69	65	52.5	59
Claude-Sonnet-4-5-20250929 2026.06	61.38	69	65	52.5	59
Gemini-3-Pro-Preview 2026.05	60.75	64.5	60	54.5	64
Gemini-3-Pro-Preview 2026.06	60.75	64.5	60	54.5	64
Grok-4-1-fast-reasoning 2026.06	58.88	70.5	59.5	43	62.5
xLAM-2-3b-fc-r 2026.05	58.38	71.5	59	57.5	45.5
xLAM-2-3b-fc-r 2026.06	58.38	71.5	59	57.5	45.5
Qwen3-4B-Instruct + RODS (ours) 2026.06	56	68	59	44	53
Claude-Haiku-4-5-20251001 2026.06	53.63	63.5	42.5	52.5	56
Nanbeige4-3B-Thinking-2511 2026.06	51.12	58.5	54	45	47
Kimi-K2-Instruct 2026.06	50.63	62	41	44.5	55
Qwen3-4B-Instruct + EnvTuning 2026.06	50.5	64	52	35	51
Qwen3-4B-Instruct + Static dataset 2026.06	50	62	51	35	52
Qwen3-32B 2026.05	47.88	56	52.5	40	43
Qwen3-32B 2026.06	47.88	56	52.5	40	43
ToolACE-2-Llama-3.1-8B + SEAL 2026.05	46.75	58	46	44	39
Grok-4-1-fast-non-reasoning 2026.06	46.75	58	39.5	37.5	52
DeepSeek-V3.2-Exp 2026.06	44.88	55	49	27	48.5
Qwen3-235B-A22B-Instruct 2026.06	44.63	54	42.5	31.5	50.5
GPT-4o-2024-11-20 2026.05	42.5	55.5	34.5	29	51
GPT-4o-2024-11-20 2026.06	42.5	55.5	34.5	29	51
Qwen2.5-7B-Instruct + SEAL 2026.05	40.25	58	36	34	33
ToolACE-MT 2026.06	40.25	57.5	31.5	34	38
Qwen2.5-7B-Instruct + RODS (ours) 2026.06	40.25	54	43.5	33.5	30
ToolACE-2-Llama-3.1-8B + Vanilla RL 2026.05	38.5	52	30	40	32
ToolACE-2-8B 2026.06	38.38	49	28	30.5	46
Qwen2.5-7B-Instruct + EnvTuning 2026.06	37.75	51.5	41	30.5	28
Qwen2.5-7B-Instruct + Static dataset 2026.06	36.92	50.33	40.33	29.33	27.67
Gemini-2.5-Flash 2026.05	36.25	41.5	36	32	35.5
ToolACE-2-Llama-3.1-8B (Original) 2026.05	32	45	26	35	22
Llama-3.1-8B-Instruct + RODS (ours) 2026.06	30.88	32	28	24.5	39
Qwen2.5-7B-Instruct + Vanilla RL 2026.05	30.75	46	27	27	23
Qwen3-30B-A3B-Thinking 2026.06	30	43.5	10.5	25	41
Gemini-2.5-Pro-Preview 2026.06	28.75	32	29	22	32
Llama-3.1-8B-Instruct + EnvTuning 2026.06	28.38	28	25.5	23	37
Llama-3.1-8B-Instruct + Static dataset 2026.06	28.25	28.2	25.85	22.15	36.8
GPT-5.2-2025-12-11 2026.06	28.13	36.5	18	27.5	30.5
Qwen2.5-14B-Instruct 2026.05	25.25	33	26	22	20
Qwen3-4B-Instruct 2026.06	22.13	26.5	21	15.5	25.5
Llama-4-Maverick 2026.06	20.25	27	22	14	18
Gemini-2.5-Flash 2026.06	16.75	14.5	16.5	17.5	18.5
Qwen2.5-3B-Instruct + SEAL 2026.05	14	19	15	12	10
Qwen2.5-7B-Instruct (Original) 2026.05	14	22	14	10	10
Qwen2.5-3B-Instruct + Vanilla RL 2026.05	9.25	16	9	6	6
Qwen2.5-7B-Instruct 2026.06	7	9.33	9.33	6.33	3
Qwen2.5-3B-Instruct (Original) 2026.05	5.75	11	6	3	3
Llama-3.1-8B-Instruct 2026.06	5.48	6.15	6.8	3.2	5.75