Multi-hop tool use on ToolHop

53.97Answer Correctness

Claude 4.0-Sonnet

Updated 3d ago

Evaluation Results

Method	Links
Claude 4.0-Sonnet 2025.08		53.97
Qwen3-32B Reasoning 2025.08		50.12
MatchTIR (KM) 2026.01		46.16
MatchTIR (OT) 2026.01		45.8
Gemini 2.5-Pro 2025.08		45.32
Qwen3-14B Reasoning FTRL-Reinforce++ 2025.08		44.79
FTRL-M 2026.01		43.32
MatchTIR (KM) 2026.01		42.55
ToolRL-M 2026.01		42.55
Vanilla 2026.01		42.21
MatchTIR (OT) 2026.01		41.95
Qwen3-14B Reasoning FTRL-GRPO 2025.08		41.64
FTRL-M 2026.01		41.24
Qwen3-8B Reasoning FTRL-Reinforce++ 2025.08		41.24
Qwen3-8B Reasoning 2025.08		40.7
GRPO 2026.01		40.64
Qwen3-8B Reasoning FTRL-GRPO 2025.08		40.5
Qwen3-14B Reasoning 2025.08		40.1
Gemini 2.5-Flash 2025.08		39.37
Qwen3-8B Non-Reasoning FTRL-GRPO 2025.08		38.93
FTRL-S 2026.01		38.63
Qwen3-14B Non-Reasoning FTRL-Reinforce++ 2025.08		38.32
Qwen3-14B Non-Reasoning FTRL-GRPO 2025.08		38.26
Qwen3-8B Non-Reasoning FTRL-Reinforce++ 2025.08		37.79
GRPO 2026.01		37.25
FTRL-S 2026.01		36.72
Qwen2.5-14B FTRL-Reinforce++ 2025.08		36.42
GPT 4o 2025.08		36.31
ToolRL-M 2026.01		35.68
Qwen3-32B Non-Reasoning 2025.08		35.34
ToolRL-S 2026.01		32.93
Vanilla 2026.01		31.63
Qwen2.5-72B Instruct 2025.08		30.29
ToolRL-S 2026.01		30.28
Qwen2.5-7B FTRL-GRPO 2025.08		29.68
Qwen3-8B Non-Reasoning 2025.08		28.54
Qwen2.5-14B FTRL-GRPO 2025.08		26.63
Qwen2.5-14B Instruct 2025.08		25.8
GPT 3.5-Turbo 2025.08		25.73
Qwen3-14B Non-Reasoning 2025.08		24.19
Qwen2.5-7B FTRL-Reinforce++ 2025.08		23.45
Qwen2.5-32B Instruct 2025.08		19.7
Qwen2.5-7B Instruct 2025.08		11.99