Tool-use Reasoning on Ours (test)

52.78Solve Precision (P)

Gemini 2.5-Flash

Updated 1mo ago

Evaluation Results

Method
Gemini 2.5-Flash 2025.08	52.78	23.13	23.24
Gemini 2.5-Pro 2025.08	51.77	31.96	29.98
Qwen2.5-14B FTRL-Reinforce++ 2025.08	49.51	47.2	47.57
Qwen3-14B Non-Reasoning FTRL-GRPO 2025.08	49.41	56.71	51.82
Qwen2.5-14B FTRL-GRPO 2025.08	48.67	50.56	49.2
Qwen2.5-7B FTRL-GRPO 2025.08	47.82	47.37	46.78
Qwen3-8B Non-Reasoning FTRL-GRPO 2025.08	46.44	55.84	49.54
Qwen3-14B Reasoning FTRL-Reinforce++ 2025.08	44.15	48.11	44.04
Qwen3-14B Non-Reasoning FTRL-Reinforce++ 2025.08	44.06	59.3	48.24
Claude 4.0-Sonnet 2025.08	42.33	56.07	42.59
Qwen3-8B Non-Reasoning FTRL-Reinforce++ 2025.08	41.96	47.37	43.63
Qwen3-14B Reasoning FTRL-GRPO 2025.08	41.67	47.67	43.36
Qwen2.5-7B FTRL-Reinforce++ 2025.08	41.61	41.36	40.36
Qwen3-8B Reasoning FTRL-GRPO 2025.08	40.67	40.06	38.58
Qwen3-8B Reasoning FTRL-Reinforce++ 2025.08	38.11	43.7	39.4
GPT 4o 2025.08	33.43	42.01	33.39
Qwen2.5-14B Instruct 2025.08	33.31	34.64	32.76
Qwen3-32B Non-Reasoning 2025.08	33.2	46.24	36.34
Qwen3-14B Reasoning 2025.08	31.91	40.23	32.96
Qwen2.5-72B Instruct 2025.08	31.68	43.41	35.22
Qwen2.5-32B Instruct 2025.08	30.12	37.8	32.42
Qwen3-32B Reasoning 2025.08	28.94	40.41	31.3
Qwen3-14B Non-Reasoning 2025.08	28.29	37.58	29.97
Qwen3-8B Reasoning 2025.08	27.78	38.38	29.52
Qwen2.5-7B Instruct 2025.08	27.44	27.69	25.97
GPT 3.5-Turbo 2025.08	25.77	29.29	25.66
Qwen3-8B Non-Reasoning 2025.08	21.18	30.71	23.48