Share your thoughts, 1 month free Claude Pro on usSee more

Common Sense Reasoning on ARC-C, ARC-E, BoolQ, HellaSwag, PIQA, WinoGrande (zero-shot)

77.67Average Accuracy (Zero-Shot)

FP16 Baseline

Updated 22d ago

Evaluation Results

Method
FP16 Baseline 2026.03	77.67	57.34	80.47	84.1	83.32	82.64	78.14
FP16 Baseline 2026.03	77.15	55.63	78.03	87.43	84.07	82.26	75.45
Floating-point 2026.02	74.19	-	-	-	-	-	-
FP16 Baseline 2026.03	73.24	49.57	76.05	84.71	78.13	79.71	71.27
Floating-point 2026.02	73.19	-	-	-	-	-	-
SpinQuant 2026.02	72.68	-	-	-	-	-	-
AWQ 2026.02	72.64	-	-	-	-	-	-
AWQ 2026.02	72.64	-	-	-	-	-	-
SpinQuant 2026.02	72.62	-	-	-	-	-	-
MoBiQuant 2026.02	71.84	-	-	-	-	-	-
OmniQuant 2026.02	71.68	-	-	-	-	-	-
OmniQuant 2026.02	71.66	-	-	-	-	-	-
MoBiQuant 2026.02	71.54	-	-	-	-	-	-
BF16 2026.02	71.2	51.71	80.89	81.77	60.56	78.84	73.4
BF16 2026.02	71.2	51.71	80.89	81.77	60.56	78.84	73.4
BF16 2026.02	71.19	55.38	83.5	86.73	57.12	76.5	67.88
BF16 2026.02	71.19	55.38	83.5	86.73	57.12	76.5	67.88
O 2026.02	70.66	54.18	82.62	86.45	55.7	76.39	68.59
GPTQ 2026.02	70.56	54.27	82.74	86.73	56.62	76.22	66.77
FP16 Baseline 2026.03	70.54	46.42	74.62	78.01	75.9	78.84	69.46
O(R) 2026.02	70.5	53.41	82.36	86.02	56.34	76.06	68.82
O(N) 2026.02	70.44	53.75	82.2	86.02	56.39	76.22	68.03
AWQ 2026.02	70.4	53.67	82.37	85.93	55.91	75.79	68.75
RTN 2026.02	70.4	-	-	-	-	-	-
GPTQ 2026.02	70.22	-	-	-	-	-	-
GPTQ 2026.02	70.21	-	-	-	-	-	-
O 2026.02	70.16	49.15	79.59	81.13	59.7	78.67	72.69
AWQ 2026.02	70.06	48.7	79.21	80.8	59.73	78.62	73.32
O(R) 2026.02	69.97	49.32	79.12	81.04	59.79	78.24	72.3
O(N) 2026.02	69.9	48.46	78.53	81.04	60	78.18	73.16
GPTQ 2026.02	69.74	48.29	78.89	80.21	59.47	78.29	73.32
Floating-point 2026.02	69.68	-	-	-	-	-	-
AWQ 2026.02	69.4	-	-	-	-	-	-
QUIP 2026.02	69.15	46.25	77.36	81.93	58.76	77.53	73.09
AWQ 2026.02	68.65	50.56	80.93	84.43	52.8	75.9	67.25
OmniQuant 2026.02	68.41	-	-	-	-	-	-
BF16 2026.02	68.16	50.34	80.26	85.11	52.29	75.3	65.67
BF16 2026.02	68.16	50.34	80.26	85.11	52.29	75.3	65.67
O 2026.02	68.08	47.27	78.41	84.39	54.23	76.27	67.9
SpinQuant 2026.02	68	-	-	-	-	-	-
O(N) 2026.02	67.61	47.69	77.61	84.46	53.76	74.76	67.35
O(R) 2026.02	67.53	47.59	77.62	84.04	53.88	74.59	67.46
SmoothQuant 2026.02	67.47	-	-	-	-	-	-
SmoothQuant 2026.02	67.46	-	-	-	-	-	-
MoBiQuant 2026.02	67.22	-	-	-	-	-	-
O 2026.02	66.81	42.92	75.42	78.69	55.73	77.75	70.32
Floating-point 2026.02	66.71	-	-	-	-	-	-
GPTQ 2026.02	66.6	44.28	74.62	83.33	53.88	75.95	67.56
AWQ 2026.02	66.33	43.86	75.21	76.15	55.79	75.95	71.03
O(R) 2026.02	65.99	41.3	73.48	76.45	56.67	77.37	70.64
O 2026.02	65.94	46.65	78.27	84.31	49.85	74.21	62.35
O(N) 2026.02	65.87	41.81	71.71	76.73	56.71	76.88	71.35
GPTQ 2026.02	65.81	47.57	77.9	82.43	50.12	74.43	62.43
AWQ 2026.02	65.7	46.29	77.32	83.23	50.37	74.1	62.88
QUIP 2026.02	65.64	41.38	71.34	77.68	55.67	76.12	71.67
GPTQ 2026.02	65.54	-	-	-	-	-	-
SliderQuant 2026.03	65.24	37.71	64.14	75.02	73.02	75.79	65.75
RTN 2026.02	64.72	-	-	-	-	-	-
RTN 2026.02	64.19	-	-	-	-	-	-
O(R) 2026.02	64.09	44.28	74.03	82.97	49.28	72.8	61.17
SmoothQuant 2026.02	63.87	-	-	-	-	-	-
OmniQuant 2026.02	63.8	-	-	-	-	-	-
QUIP 2026.02	63.3	45.48	75.42	81.99	44.72	69.53	62.67
MoBiQuant 2026.02	63.2	-	-	-	-	-	-
O(N) 2026.02	63.19	41.81	74.79	83.46	47.61	73.12	58.33
SliderQuant 2026.03	62.74	40.7	68.1	70.14	62.12	72.09	63.3
CBQ 2026.03	62.7	37.3	62.45	72.13	69.84	73.24	61.23
O 2026.02	62.03	41.5	71.41	80.95	45.61	72.64	60.06
GPTQ 2026.02	61.76	39.33	70.92	80.95	46.81	70.92	61.64
GPTQ 2026.02	61.51	34.21	66.04	71.89	55.26	73.56	68.11
AWQ 2026.02	61.17	40.27	68.27	81.19	46.02	70.87	60.38
QUIP 2026.02	60.81	39.25	71.09	80.73	43.64	70.24	59.91
CBQ 2026.03	60.79	36.86	63.47	69.63	59.41	71.44	63.93
QUIP 2026.02	60.66	40.27	71.93	77.37	43.22	71.33	59.83
OmniQuant 2026.03	60.39	39.93	61.24	65.26	65.48	71.16	59.27
OmniQuant 2026.03	59.93	37.2	59.55	66.3	66.63	71.71	58.17
Floating-point 2026.02	59.42	-	-	-	-	-	-
SliderQuant 2026.03	59.3	32.94	59.64	67.77	62.33	71.38	61.72
OmniQuant 2026.02	56.99	-	-	-	-	-	-
O(R) 2026.02	56.39	37.79	65.35	70.58	43.39	67.25	53.99
CBQ 2026.03	56.26	33.45	56.4	61.96	60.46	70.18	55.09
SliderQuant 2026.03	56.15	33.62	59.3	62.54	55.06	66.92	59.43
MoBiQuant 2026.02	55.49	-	-	-	-	-	-
OmniQuant 2026.03	54.95	31.91	54.17	63.67	55.95	68.44	55.56
O(N) 2026.02	53.84	35.95	58.68	69.2	40.65	67.74	50.83
SmoothQuant 2026.03	49.94	31.23	46.21	57.28	52.65	61.37	50.91
CBQ 2026.03	49.69	31.06	48.99	53.67	47.31	62.57	54.54
SmoothQuant 2026.03	49.09	32.34	48.15	50.55	49.91	61.59	52.01
QUIP 2026.02	47.91	24.15	47.81	67.86	33.72	62.79	51.14
SmoothQuant 2026.03	45.29	27.13	39.77	51.07	41.32	60.88	51.54
OmniQuant 2026.03	39.48	24.4	33.84	38.78	33.75	53.32	52.8
SmoothQuant 2026.03	36.73	26.02	25.97	38.81	25.92	50.44	53.2
Baseline (Ours) 2026.03	-	43.2	75.6	79.3	57.1	78.1	69.9
Quip# 2026.03	-	32.5	42.8	62.3	-	71.2	62.4
LLVQ [spherical shaping] 2026.03	-	34.7	69.3	67.5	46.6	73.5	64.6
LLVQ [shape-gain, 2 bit gain] 2026.03	-	35.5	69.8	73	49.7	75.2	66.9
Quip 2026.03	-	19.4	26	54.6	-	-	51.8
OmniQ 2026.03	-	21.6	35.2	57.5	-	-	51.5
AQLM 2026.03	-	32.8	63.7	74.8	-	-	65.7
Quip# 2026.03	-	35.2	65.3	75.4	-	-	64.9

Showing 100 of 104 rows