Share your thoughts, 1 month free Claude Pro on usSee more

Zero-shot Evaluation on ARC-c, ARC-e, HellaSwag, PIQA, and WinoGrande

59.3ARC-c Accuracy

Original

Updated 1mo ago

Evaluation Results

Method
Original 2026.06	59.3	83.67	84.03	83.57	76.24	77.36
AlphaQ 2026.06	55.8	83.38	74.67	81.51	76.1	74.29
Original 2026.06	55.8	79.29	77.75	80.25	71.11	72.84
Uniform 2026.06	54.95	83.04	73.58	81.28	76.19	73.81
PMQ 2026.06	53.92	82.49	72.9	81.21	76.09	73.32
AlphaQ 2026.06	51.59	78.87	69.24	80.84	73.24	70.76
PMQ 2026.06	51.28	77.1	69.23	80.36	73.95	70.38
Hessian 2026.06	51.2	76.38	67.05	80.21	72.97	69.56
AlphaQ 2026.06	50.35	75.87	74.49	78.34	68.71	69.56
FP16 2026.04	50.3	80.1	60.2	79.5	73.7	68.8
Uniform 2026.06	50.26	77.4	58.84	79.38	73.4	67.86
Uniform 2026.06	50.07	75.93	71.72	78.07	68.01	68.76
FASQ 2-512 2026.04	50	78.5	59.5	78.7	73.4	68
FASQ 2-1024 2026.04	49.7	79	59.9	78.9	73.3	68.2
SmoothQuant (W8A8) 2026.04	49	79.6	60	79.4	73.2	68.2
Original 2026.06	48.98	76.47	78.01	80.08	71.51	71.01
AWQ 2026.04	48.5	79.3	59.1	78.6	73.1	67.7
NeUQI 2025.05	48.04	78.11	56.18	78.56	75.14	67.21
FASQ 2-256 2026.04	48	79.5	58.5	79.4	73.4	67.8
GPTQ 2026.04	47.7	78.8	59	78.4	72.6	67.3
QuIP 2026.04	47.4	78.2	58.6	78.2	73.2	67.1
AlphaQ 2026.06	46.51	75.51	77.21	79.89	71.21	70.07
FASQ 2-128 2026.04	46.3	77.1	56.4	78.2	72.2	66
PMQ 2026.06	45.99	68.43	72.13	76.61	66.22	65.88
PMQ 2026.06	45.83	75.26	76.97	79.63	71.04	69.75
SmoothQuant (W6A6) 2026.04	45	75.5	56.9	76.8	69	64.6
NeUQI 2025.05	44.71	75.97	53.62	77.64	73.72	65.13
Uniform 2026.06	44.57	74.56	76.27	79.21	70.83	69.09
AlphaQ 2026.06	44.33	69.15	77.7	80.46	68.55	68.04
Original 2026.06	44.11	69.23	77.21	80.52	68.59	67.93
PMQ 2026.06	43.81	69.02	77.13	80.31	68.5	67.75
AWQ 2026.04	43.2	74	55.1	77.7	72.1	64.4
AlphaQ 2026.06	42.72	66.97	71.81	78.21	66.12	65.17
NeUQI 2025.05	41.55	71.63	53.38	76.01	72.77	63.07
Uniform 2026.06	41.51	66.81	74.52	79.31	67.54	65.94
QuIP 2026.04	41	72.9	55.4	76.8	72.5	63.7
MagR 2025.05	39.59	71.21	52.81	76.33	69.06	61.8
RTN 2026.04	39.4	68.2	56	75	69	61.5
AlphaQ 2026.06	39.22	73.77	73.38	78.02	69.3	66.74
NeUQI 2025.05	37.88	69.32	46.86	72.31	67.4	58.76
GPTQ 2026.04	37.7	70.5	54.3	74.9	71.1	61.7
AFG 2026.06	37.57	69.2	67.88	74.32	63.8	62.55
AFG 2026.06	37.57	69.2	67.88	74.32	63.8	62.55
PMQ 2026.06	37.37	71.68	72.76	77.86	69.06	65.75
PMQ 2026.06	36.26	60.56	70.69	77.58	65.82	62.18
Uniform 2026.06	35.84	71.34	72.44	76.66	69.46	65.15
MagR 2025.05	31.91	59.64	51.13	72.74	59.59	55
AFG 2026.06	31.2	61.07	72.77	77.53	62.12	60.94
Uniform 2026.06	29.44	49.07	53.98	71.49	55.96	51.99
BSP 2026.06	28.38	54.97	55.61	68.23	62.19	53.88
NeUQI 2025.05	26.19	55.18	37.37	65.94	59.12	48.76
MagR 2025.05	25.94	48.91	45.25	68.06	56.51	48.93
NeUQI 2025.05	25.68	53.24	36.98	61.04	55.49	46.49
NeUQI 2025.05	24.83	52.1	37.5	63	59.04	47.3
NeUQI 2025.05	23.98	51.73	36.04	65.89	58.56	47.24
MagR 2025.05	23.21	26.68	36.7	52.99	54.14	38.74
GPTAQ 2025.05	23.21	25.34	26.49	52.23	46.96	34.85
MagR 2025.05	23.21	44.74	38.06	63.44	52.88	44.47
MagR 2025.05	23.12	42.09	37.76	60.99	51.3	43.05
GPTQ 2025.05	23.12	25.42	25.9	50.92	51.93	35.46
GPTQ 2025.05	22.95	25	25.46	51.31	48.7	34.68
GPTAQ 2025.05	22.7	24.87	25.78	53.21	51.22	35.56
GPTAQ 2025.05	22.35	25.63	25.6	51.31	49.8	34.94
NeUQI 2025.05	22.1	47.85	39.56	60.99	58.48	45.8
GPTQ 2025.05	21.93	25.63	25.89	52.23	49.72	35.08
GPTAQ 2025.05	21.76	26.89	25.74	53.32	47.12	34.97
GPTQ 2025.05	21.76	26.52	25.85	52.56	51.3	35.6
GPTAQ 2025.05	21.59	28.91	28.98	54.08	51.46	37
MagR 2025.05	21.59	25.59	25.51	52.99	48.46	34.83
GPTAQ 2025.05	21.59	23.4	27.87	51.74	49.09	34.74
MagR 2025.05	21.42	34.97	32.29	58.27	50.75	39.54
GPTQ 2025.05	21.42	26.52	25.7	50.92	48.93	34.7
GPTQ 2025.05	21.42	26.09	25.11	51.2	49.41	34.65
GPTAQ 2025.05	20.99	25.29	25.88	52.99	48.46	34.72
GPTQ 2025.05	20.9	23.91	26.04	53.48	48.86	34.64
GPTAQ 2025.05	20.82	23.86	27.08	52.23	51.22	35.04
GPTQ 2025.05	20.73	25.76	25.76	52.45	48.46	34.63
GPTAQ 2025.05	20.56	26.89	26.99	53.43	51.85	35.95
SmoothQuant (W4A4) 2026.04	20	26.3	26.4	54.6	50.3	35.5
RTN 2026.04	20	31.1	27.5	56.2	53.1	37.6
GPTQ 2025.05	19.97	28.07	27.05	53.75	50.04	35.78
MagR 2025.05	18.43	30.09	27.6	55.98	50.12	36.45