Share your thoughts, 1 month free Claude Pro on usSee more

Tool-use Agent Performance on GeoDisaster

98.75Tool Usage Success Rate (Any)

MAS+SFT

Updated 1mo ago

Evaluation Results

Method	Links
MAS+SFT 2026.06		98.75	98.26	98.41	92.46	82.43	88.16	94.69	99.43	99.58	99.58	99.58	91.01
MAS+SFT+GRPO 2026.06		98.56	97.76	98.72	94.24	90.11	88.88	95.89	100	100	100	100	91.59
o4-mini 2026.06		23.26	20.93	23.26	29.12	26.64	30.59	44.19	44.19	86.97	86.97	86.97	47.1
GPT-5.5 2026.06		20.93	20.93	30.23	55.06	61.99	71.44	100	100	95.86	95.86	95.86	82.32
GPT-5 2026.06		11.99	11.99	13.7	46.27	60.48	65.37	81.51	81.51	69.71	71.56	69.71	69.27
MAS 2026.06		6.85	4.11	10.62	20.36	22.93	18.26	49.66	90.88	81.71	95.46	62.57	30.03
GPT-4o 2026.06		3.44	3.44	3.78	42.65	60.19	65.43	80.76	80.76	78.53	78.53	78.53	71.23
Qwen2.5-7B-Inst. 2026.06		2.4	0.34	4.45	3.11	2.53	4.45	5.82	5.82	42.06	77.05	36.52	6.45
Qwen3-4B-Inst. 2026.06		0	0	0	1.16	1.16	2.33	2.33	6.98	40	23.26	9.3	3.72
Llama-3.1-8B-Inst. 2026.06		0	0	0	0.15	0.25	0.25	0.25	0.25	0	0.7	0.35	0.35
Mistral-7B-Inst. 2026.06		0	0	0	0.15	0.25	0.25	0.25	0.25	0	0.7	0.35	0.35
Qwen2.5-VL-7B 2026.06		0	0	0	0.95	1.5	2.1	1.16	3.49	0	12.4	5.81	3.1