Robustness of Tool-use on RoTBench

78.76TS

GPT 4o

Updated 3d ago

Evaluation Results

Method
GPT 4o 2025.08	78.76	49.43	31.52
Qwen3-8B Non-Reasoning FTRL-Reinforce++ 2025.08	78.1	42.57	25.9
Qwen3-14B Non-Reasoning FTRL-Reinforce++ 2025.08	77.81	42.57	25.24
Qwen3-8B Non-Reasoning FTRL-GRPO 2025.08	77.62	44.67	27.05
Qwen3-14B Non-Reasoning 2025.08	75.9	36.38	20.67
Qwen3-8B Non-Reasoning 2025.08	75.52	36.29	22.19
Qwen2.5-32B Instruct 2025.08	75.43	36.86	20
Qwen3-14B Non-Reasoning FTRL-GRPO 2025.08	75.33	40.95	22.48
Qwen2.5-7B FTRL-GRPO 2025.08	74	30.86	18.95
Qwen3-32B Non-Reasoning 2025.08	73.24	41.71	24.86
Qwen2.5-14B FTRL-Reinforce++ 2025.08	73.05	45.24	28.19
Qwen2.5-72B Instruct 2025.08	72.19	26.29	17.14
Qwen2.5-14B Instruct 2025.08	71.81	38.29	21.33
Qwen2.5-14B FTRL-GRPO 2025.08	71.52	36.19	21.62
Qwen2.5-7B FTRL-Reinforce++ 2025.08	70.95	40.95	26.1
Qwen2.5-7B Instruct 2025.08	70.38	27.33	15.43
GPT 3.5-Turbo 2025.08	65.71	29.81	17.33
Qwen3-14B Reasoning FTRL-GRPO 2025.08	60.57	38.1	21.43
Qwen3-14B Reasoning FTRL-Reinforce++ 2025.08	58.95	37.24	21.62
Qwen3-14B Reasoning 2025.08	57.33	36.19	20.29
Qwen3-8B Reasoning FTRL-GRPO 2025.08	54.57	39.24	22.95
Qwen3-8B Reasoning 2025.08	54	34.29	20.38
Qwen3-32B Reasoning 2025.08	53.9	33.81	20.19
Gemini 2.5-Flash 2025.08	53.62	34.95	22.76
Qwen3-8B Reasoning FTRL-Reinforce++ 2025.08	53.52	37.52	22.29
Gemini 2.5-Pro 2025.08	49.05	42.38	27.52
Claude 4.0-Sonnet 2025.08	39.72	34.95	21.81