Multi-step Narrative Reasoning on MUSR (ACC, TOK, η)

65.86Accuracy

Qwen3-4B-Thinking-2507 + DiffAdapt

Updated 21d ago

Evaluation Results

Method
Qwen3-4B-Thinking-2507 + DiffAdapt 2026.05	65.86	6,841	0.909
Qwen3-4B-Thinking-2507 2026.05	64.42	6,081	1
Qwen3-4B-Thinking-2507 + BET 2026.05	63.84	1,188	5.073
Qwen3-4B-Thinking-2507 + DEER 2026.05	62.02	5,351	1.094
Qwen3-4B-Thinking-2507 + DR.SAF 2026.05	61.31	2,118	2.732
Qwen3-4B-Thinking-2507 + Length-Penalty 2026.05	57.88	2,529	2.16
Qwen3-4B-Thinking-2507 + Overthink 2026.05	57.78	4,103	1.329
Qwen3-4B-Thinking-2507 + VeriThinker 2026.05	47.68	3,906	1.152
DeepSeek-R1-Distill-Qwen-14B 2026.05	46.3	4,633	1
DeepSeek-R1-Distill-Qwen-7B 2026.05	44.44	5,173	1
Qwen3-4B-Thinking-2507 + ThinkSwitcher 2026.05	41.67	3,238	1.215
DeepSeek-R1-Distill-Qwen-14B + BET 2026.05	41.01	2,216	1.852
DeepSeek-R1-Distill-Qwen-7B + BET 2026.05	40.4	2,764	1.701
DeepSeek-R1-Distill-Qwen-7B + DR.SAF 2026.05	40.2	4,229	1.107
DeepSeek-R1-Distill-Qwen-14B + DR.SAF 2026.05	39.29	3,393	1.159
DeepSeek-R1-Distill-Qwen-7B + Length-Penalty 2026.05	39.09	4,783	0.951
DeepSeek-R1-Distill-Qwen-7B + Overthink 2026.05	38.99	4,952	0.917
DeepSeek-R1-Distill-Qwen-14B + Length-Penalty 2026.05	38.36	3,574	1.074
DeepSeek-R1-Distill-Qwen-7B + DiffAdapt 2026.05	36.16	5,716	0.736
DeepSeek-R1-Distill-Qwen-7B + DEER 2026.05	34.34	4,008	0.997
DeepSeek-R1-Distill-Qwen-7B + VeriThinker 2026.05	29.9	4,869	0.715
DeepSeek-R1-Distill-Qwen-7B + ThinkSwitcher 2026.05	23.13	4,619	0.583