Share your thoughts, 1 month free Claude Pro on usSee more

Question Answering and Commonsense Reasoning on ARC-C, ARC-E, PIQA, Winogrande, HellaSwag

63.48ARC-C Accuracy

FP16/BF16

Updated 1mo ago

Evaluation Results

Method
FP16/BF16 2026.04	63.48	83.92	83.95	79.01	84.58	78.99
QuIP 2026.04	62.12	82.45	82.1	78.06	82.83	77.51
GSQ 2026.04	62.12	82.83	82.75	78.93	83.3	77.99
EfficientQAT 2026.04	61.86	83.88	82.48	76.01	82.79	77.4
QTIP 2026.04	61.77	82.79	83.79	78.3	84.21	78.17
QTIP 2026.04	61.69	81.69	82.43	77.51	82.95	77.25
GPTQ 2026.04	60.75	80.64	83.3	77.98	82.37	77.01
GSQ 2026.04	60.2	81.4	82.4	80.6	82.6	77.4
GSQ 2026.04	59.6	81.4	82.9	80.4	82.9	77.5
GSQ 2026.04	58.87	79.55	81.07	76.24	82.11	75.57
PV-Tuning 2026.04	58.62	80.72	81.56	77.74	82.72	76.27
FP16/BF16 2026.04	55.12	79.63	80.85	73.8	79.16	73.71
EfficientQAT 2026.04	54.86	77.27	80.36	65.67	79.01	71.43
QTIP 2026.04	53.92	79.42	80.25	72.14	78.3	72.81
EfficientQAT 2026.04	52.99	78.91	79.76	71.59	76.85	72.02
GSQ 2026.04	52.99	78.37	80.03	73.56	76.66	72.32
QuIP 2026.04	52.3	76.68	78.84	71.9	75.37	71.02
QTIP 2026.04	50.68	75.42	78.18	70.09	75.02	69.88
GSQ 2026.04	50.26	74.41	78.13	69.61	74.81	69.44
PV-Tuning 2026.04	50.26	73.91	79.16	70.56	75.28	69.83
GSQ 2026.04	48.12	72.35	78.07	70.8	73.42	68.55
Full Cache 2026.02	45.76	70.02	80.85	71.51	74.68	68.56
DynaKV 2026.02	44.75	67.37	80.36	67.64	71.7	66.36
Full Cache 2026.02	44.41	71.6	79.76	70.4	72.28	67.69
EfficientQAT 2026.04	43.77	67.55	74.65	64.33	68.65	63.79
GSQ 2026.04	42.83	67.13	73.5	65.82	67.91	63.44
DynaKV 2026.02	42.71	59.96	79.82	66.54	70.33	63.87
DynaKV 2026.02	42.37	70.19	78.78	64.96	68.18	64.9
DynaKV 2026.02	40	50.79	76.71	63.14	67.08	59.54
GPTQ 2026.04	39.68	58.67	67.03	66.3	64.24	59.18
QuIP 2026.04	39.51	60.44	73.61	65.35	68.95	61.57
Palu 2026.02	39.32	63.84	77.58	67.8	69	63.51
DynaKV 2026.02	39.32	59.96	75.41	58.72	62.44	59.17
DynaKV 2026.02	38.98	59.44	78.56	65.51	67.89	62.08
MatryoshkaKV 2026.02	38.64	60.49	78.02	63.14	67.36	61.53
GPTQ 2026.04	38.23	58.63	72.8	57.14	60.11	57.38
MatryoshkaKV 2026.02	32.88	49.74	73.67	55.8	56.8	53.78
Palu 2026.02	32.88	49.74	71.38	62.43	60.74	55.43
Palu 2026.02	29.49	28.57	65.72	54.7	46.47	44.99
Palu 2026.02	29.15	27.51	71.6	60.62	57.19	49.21
MatryoshkaKV 2026.02	27.46	45.15	68.5	53.83	45.31	48.05
Palu 2026.02	27.12	37.74	61.97	53.75	40.04	44.12
Palu 2026.02	26.44	31.57	56.91	50.99	32.39	39.66
GPTQ 2026.04	24.91	27.61	52.83	51.78	30.5	37.53
QuIP 2026.04	23.72	28.96	53.86	50.83	38.65	39.2