Share your thoughts, 1 month free Claude Pro on usSee more

Logic Reasoning on K&K Logic Puzzles In-domain

98Accuracy (Level 3)

o3-mini-high

Updated 1mo ago

Evaluation Results

Method	Links
o3-mini-high 2025.06		98	97	95	94	89	95
MeRF 2025.06		95	92	84	78	66	83
MeRF 2025.06		95	94	92	87	71	88
RLVR 2025.06		92	89	84	79	64	82
RLVR 2025.06		90	82	78	70	55	75
MeRF 2025.06		78	73	68	62	42	65
Deepseek-R1 2025.06		73	77	78	75	88	78
RLVR 2025.06		68	67	57	43	22	51
MeRF 2025.06		68	62	48	42	20	48
GPT-4o 2025.06		57	49	32	23	21	36
MeRF 2025.06		54	53	45	36	16	41
RLVR 2025.06		52	44	36	18	12	32
o1-2024-12-17 2025.06		51	38	38	35	30	38
RLVR 2025.06		48	53	30	21	16	34
Qwen2.5-14B-Instruct 2025.06		46	31	20	9	11	23
GPT-4o-mini 2025.06		42	34	17	9	10	22
Qwen2.5-7B-Instruct-1M 2025.06		40	25	11	6	2	17
Qwen2.5-7B-Base 2025.06		34	16	9	0	0	12
DeepSeek-R1-Distill-Llama-8B 2025.06		26	22	14	6	5	15
MeRF 2025.06		25	20	12	8	4	14
Qwen2.5-7B-Instruct 2025.06		24	10	6	4	4	10
Deepseek-Math-7B 2025.06		21	8	6	2	0	7
RLVR 2025.06		20	16	16	10	4	13
NuminaMath-7B-CoT 2025.06		13	12	5	1	0	6
DeepSeek-R1-Distill-Qwen-1.5B 2025.06		8	1	0	0	0	2