Share your thoughts, 1 month free Claude Pro on usSee more

Zero-shot Reasoning Suite (PiQA, ARC-E/C, HellaSwag, WinoGrande, BoolQ)

80.7PiQA Accuracy (Zero-shot)

Updated 2mo ago

Evaluation Results

Method
FP 2026.05	80.7	77.7	53.7	79.1	73.2	81.1	74.3
FP 2026.05	80.5	77.5	49.2	79.4	72.4	80.6	73.3
ResComp-MARR 2026.05	79.38	75.17	47.78	77.11	70.96	78.35	71.46
FP 2026.05	79	74.6	46.5	76	68.9	77.7	70.5
GPTAQ-MARR 2026.05	78.73	73.99	46.84	77.27	70.48	77.46	70.79
ResComp 2026.05	78.67	74.45	47.87	77.1	70.32	80.03	71.41
GPTQ 2026.05	78.24	74.83	46.67	76.9	69.53	79.14	70.88
GPTAQ 2026.05	78.24	73.91	46.84	77.1	69.46	78.23	70.63
GPTAQ 2026.05	78.24	71.8	46.76	74.94	67.88	77.86	69.58
ResComp-MARR 2026.05	78.18	74.62	46.16	74.97	67.4	76.7	69.67
ResComp 2026.05	78.02	72.22	47.44	74.72	67.64	76.79	69.47
GPTAQ-MARR 2026.05	77.97	73.95	47.35	75.17	68.67	76.97	70.01
ResComp 2026.05	77.86	70.46	42.69	73.72	65.23	73.06	67.17
GPTQ 2026.05	77.69	71.76	42.49	73.17	65.04	75.32	67.58
RTN 2026.05	77.37	74.03	44.8	73.73	68.82	76.09	69.14
GPTAQ-MARR 2026.05	77.04	71.46	43.77	73.57	67.96	73.85	67.94
ResComp-MARR 2026.05	76.88	69.91	42.24	73.7	66.06	74.71	67.25
GPTAQ 2026.05	76.5	70.37	41.98	73.52	65.9	73.64	66.98
GPTQ 2026.05	75.73	72.18	44.28	73.81	66.69	76.15	68.14
RTN 2026.05	74.16	62.84	36.35	66.24	61.17	69.79	61.76
RTN 2026.05	74.1	65.28	40.02	70.26	65.19	74.4	64.88
GPTAQ 2026.05	67.03	54.83	31.08	48.54	56.31	62.01	53.3
GPTAQ-MARR 2026.05	66.38	51.39	30.38	51.93	59.04	64.16	53.88
ResComp-MARR 2026.05	64.74	48.27	28.75	48.67	57.62	62.91	51.83
ResComp 2026.05	63.76	51.76	26.55	47.65	54.22	60.29	50.7
GPTAQ 2026.05	62.13	44.15	25.68	42.7	52.64	58.59	47.65
GPTAQ-MARR 2026.05	61.48	43.9	25.26	43.84	57.38	61.31	48.86
ResComp-MARR 2026.05	60.94	43.56	25.17	42.81	55.49	62.05	48.34
GPTQ 2026.05	60.94	41.29	24.49	41.35	53.35	62.05	47.24
ResComp 2026.05	60.79	44.17	25.02	41.79	53.41	62.06	47.87
GPTAQ-MARR 2026.05	59.79	39.35	25.26	40.58	54.14	58.9	46.34
GPTAQ 2026.05	58.7	40.49	23.21	38.96	54.22	62.29	46.31
ResComp 2026.05	58.25	38.32	23.74	38.54	53.2	61.59	45.44
ResComp-MARR 2026.05	57.56	37.37	23.98	38.35	55.25	61.5	45.67
GPTQ 2026.05	56.8	33.38	21.76	31.77	52.33	49.79	40.97
GPTQ 2026.05	53.48	32.95	21.25	30.09	50.75	46.12	39.11
RTN 2026.05	51.09	25.63	26.62	26.87	50.2	43.15	37.26
RTN 2026.05	50.76	25.59	28.58	26.38	48.54	37.83	36.28
RTN 2026.05	48.59	27.19	27.13	25.02	51.78	37.83	36.26