Share your thoughts, 1 month free Claude Pro on usSee more

Error Attribution on Aegis-Bench

16.62Pair µF1

Qwen2.5-14B-Instruct + Aegis-SFT

Updated 15d ago

Evaluation Results

Method
Qwen2.5-14B-Instruct + Aegis-SFT 2025.09	16.62	9.99	76.53	47.97	27.53	27.66	26.51
VerifyMAS-8B-SFT 2026.05	12.17	5.99	60.19	21.23	29.94	30.34	-
VerifyMAS-7B-SFT 2026.05	11.93	5.95	61.86	22.96	29.95	29.8	-
Qwen3-8B-Non-Thinking + SFT 2025.09	9.68	5.73	64.79	38.96	20.37	20.36	21.41
Qwen3-8B-SFT 2026.05	9.68	5.73	64.79	38.96	20.37	20.36	-
DCL 2025.09	8.33	5.3	22.93	20.23	24.73	27.7	12.61
DCL 2026.05	8.33	5.3	22.93	20.23	24.73	27.7	-
o3 2025.09	7.86	2.27	40.31	23.27	22.37	16.76	20.24
Claude-Sonnet-4 2025.09	7.68	2.34	40.73	15.51	21.21	16.55	18.16
GPT-4.1 2025.09	7.44	2.27	37.48	11.12	20.65	15.75	15.27
Qwen2.5-7B-Instruct + GRPO 2025.09	7.11	2.77	35.43	14.86	17.21	10.54	14.87
Gemini-2.5-Flash 2025.09	6.99	2.76	42.02	16.45	23.47	19.85	19.55
Gemini-2.5-Pro 2025.09	6.96	2.88	41.32	16.15	19.93	16.29	18.35
Qwen3-8B-Non-Thinking + GRPO 2025.09	6.94	2.82	45.91	17.39	20.89	15.15	17.15
Qwen2.5-14B-Instruct + Aegis-GRPO 2025.09	6.84	2.55	49.74	18.38	21.19	16.1	18.41
gpt-oss-120b 2025.09	6.53	1.71	38.58	5.53	20.38	12.05	17.07
GPT-4o-mini 2025.09	5.76	1.63	38.54	14.72	19.95	16.02	15.83
Qwen2.5-72B-Instruct 2025.09	5.6	2.2	37.46	14.51	17.72	16.58	15.01
Qwen2.5-14B-Instruct 2025.09	5.47	2.2	35.78	12.71	20.24	5.91	13.99
w/o intent 2025.09	5.43	6.83	13.7	13.9	22.67	23.8	10.16
only-mix head 2025.09	5.17	4.2	24.33	22.6	25.2	26.8	12.42
Qwen2.5-7B-Instruct + SFT 2025.09	5.05	2.8	60.03	22.7	19.61	16.9	17.99
Qwen2.5-7B-SFT 2026.05	5.05	2.8	60.03	22.7	19.61	16.9	-
Qwen2.5-7B-Instruct 2025.09	5.02	2.52	27.55	14.49	14.96	11.36	12.43
Qwen3-8B-Thinking 2025.09	4.42	1.52	34.63	9.01	17.48	14.31	13.06
Qwen3-8B-Thinking + GRPO 2025.09	4.41	1.66	36.11	15.73	17.94	12.03	17.58
Qwen3-8B-Non-Thinking 2025.09	3.96	1.4	21.34	8.16	15.81	13.89	10.12
w/o consistency 2025.09	2.93	2.8	14.47	13.67	23.47	23.2	9.52
only-bilinear 2025.09	2.67	2.4	14.6	14	24.33	24.17	10.01
Random 2025.09	0.33	0.21	4.54	3.56	11.23	11.15	4.08