Share your thoughts, 1 month free Claude Pro on usSee more

Travel Planning on TravelPlanner (test)

87.4Final Pass Rate

Qwen3-14B

Updated 18d ago

Evaluation Results

Method
Qwen3-14B 2026.05	87.4	98.5	91.8	96.07	92.9	-
Qwen3-14B 2026.05	84.2	97.83	89.5	94.28	91.4	-
Llama-3.1-8B-Instruct 2026.05	68.7	98.81	94	84.11	71.7	-
Llama-3.1-8B-Instruct 2026.05	65.2	98.83	91.4	84.54	69.7	-
Qwen3-8B 2026.07	56.1	97.3	81.7	73.1	59.4	-
Qwen3-8B 2026.07	45	96.3	74.4	65.7	48.9	-
DeepSeek-R1 2026.05	40	92.65	55.2	74.15	62	-
DICE-PC 2026.06	15.2	-	-	-	-	100
DICE-FT 2026.06	12.7	-	-	-	-	100
DeepSeek-R1 2026.07	12.2	80.6	22.2	51.7	41.7	-
DeepSeek-R1-Distill-Llama-8B 2026.07	11.7	81.1	32.1	42.9	28.9	-
MA-Debate 2026.06	11.3	-	-	-	-	99.8
DICE-PC 2026.06	10.3	-	-	-	-	100
o1-preview 2026.07	10	79.6	15	41.9	37.8	-
MA-FT 2026.06	9.7	-	-	-	-	100
DICE 2026.06	9.3	-	-	-	-	100
Qwen3-8B 2026.05	8.3	73.84	14.6	27.95	24	-
GPT-4o 2026.07	7.8	84.7	31.1	53.6	31.1	-
DICE-FT 2026.06	7.1	-	-	-	-	100
Debate 2026.06	7.1	-	-	-	-	98.2
GPT-4o-2024-11-20 2026.05	7	84.18	25.9	49.87	26.6	-
Qwen2.5-72B-Instruct 2026.07	6.1	82.3	16.7	32.6	22.8	-
Qwen3-8B 2026.05	5.9	72.04	10.7	28	21.8	-
Direct 2026.06	5.8	-	-	-	-	100
MA-Debate 2026.06	5.7	-	-	-	-	98.9
DeepSeek-R1-Distill-Llama-8B 2026.07	5.6	80.8	25	36.2	19.4	-
DICE-PC 2026.06	5.4	-	-	-	-	99.1
GPT-4-Turbo 2026.05	4.4	80.6	15.2	44.3	23.1	-
Direct 2026.06	4.4	-	-	-	-	100
QwQ-32B 2026.07	4.4	74.9	6.1	41.4	32.8	-
MA-FT 2026.06	4.3	-	-	-	-	98.6
Debate 2026.06	3.7	-	-	-	-	97.8
Qwen3-14B 2026.05	2.9	66.62	6.13	19.03	15.5	-
DeepSeek-V3 2026.07	2.2	80.3	17.2	30.5	13.9	-
Llama-3.1-70B-Instruct 2026.07	2.2	82.8	18.9	33.1	16.1	-
Qwen3-8B 2026.07	2.2	72.7	7.8	34.8	27.8	-
Gemini Pro 2026.05	2.1	64.7	7.9	10.6	4.7	-
GPT-o3 2026.06	1.9	-	-	-	-	95.6
GPT-3.5-Turbo 2026.05	0.7	45.9	2.5	10.7	3.1	-
Mixtral-8x7B-MoE 2026.05	0.7	67	3.7	3.9	1.6	-
GPT-3.5-Turbo 2026.05	0.6	59.5	2.7	9.5	4.4	-
GPT-3.5-Turbo 2026.05	0.6	52.1	2.2	9.9	3.8	-
GPT-4-Turbo 2026.06	0.6	-	-	-	-	93.1
Llama-3.1-8B-Instruct 2026.05	0.43	57.12	1.13	6.75	3.53	-
GPT-3.5-Turbo 2026.05	0.4	64.4	2.3	9.8	3.8	-
Greedy Search 2026.05	0	72	0	52.4	31.8	-
Greedy Search 2026.06	0	-	-	-	-	100
Llama-3.1-8B-Instruct 2026.07	0	60.1	0	7.9	2.8	-
DeepSeek-R1-Distill-Llama-8B 2026.07	0	61.2	0	0	0	-