Share your thoughts, 1 month free Claude Pro on usSee more

Question Answering on QA Suite Zero-shot (PIQA, ARC-E, ARC-C, BoolQ, HellaSwag, WinoGrande)

80.85PIQA Accuracy

FP16

Updated 11d ago

Evaluation Results

Method	Links
FP16 2026.04		80.85	77.78	53.41	81.28	79.16	72.84	74.22
FP16 2026.04		80.85	77.78	53.41	81.28	79.16	72.84	74.22
FP16 2024.06		80.79	58.71	46.24	82.29	80.72	77.5	71.04
RUQuant 2026.04		80.25	77.44	52.73	80.4	78.99	73.24	73.84
DuQuant 2026.04		80.2	77.27	52.05	80.12	79.14	72.77	73.59
FP16 2024.06		80.08	58.92	45.47	68.44	79.21	72.53	67.44
DuQuant+LWC 2024.06		79.98	58.29	44.8	77.89	79.22	72.21	68.73
DuQuant 2024.06		79.71	57.95	45.05	79.82	78.66	72.29	68.91
RUQuant 2026.04		79.16	57.74	44.45	67.83	76.45	69.77	65.9
FP16 2024.06		79.1	59.89	44.45	68.01	76.21	70.31	66.33
FP16 2025.04		79.05	59.89	44.71	68.47	76.23	70.24	66.43
FP16 2026.04		79.05	57.91	44.2	69.02	76.6	69.69	66.08
FP16 2026.04		79.05	57.91	44.2	69.02	76.6	69.69	66.08
FP16 2025.04		79	57.95	44.28	69.02	76.58	69.69	66.09
SmoothQuant 2026.04		78.94	75.88	49.49	77.58	77.39	70.8	71.68
OmniQuant 2026.04		78.9	73.95	47.35	74.95	76.77	70.56	70.41
QLLM 2026.04		78.78	58.29	43.77	-	75.1	68.43	64.87
DuQuant+LWC 2024.06		78.73	56.52	43.17	68.84	77.53	70.96	65.96
DuQuant 2026.04		78.62	56.94	43.43	68.35	76.19	69.22	65.46
DuQuant 2024.06		78.56	56.99	42.32	66.73	76.7	69.61	65.15
RUQuant 2026.04		78.51	56.14	41.98	68.41	73.78	65.98	64.13
SmoothQuant 2026.04		78.29	57.41	43.86	69.5	75.02	66.93	65.17
OmniQuant 2026.04		78.24	57.58	43.86	71.1	75.52	68.35	65.78
RUQuant 2026.04		78.24	55.47	41.3	68.2	74.07	67.64	64.15
Atom 2025.04		77.64	58.38	41.81	68.5	73.75	65.98	64.64
DuQuant+LWC 2024.06		77.64	57.32	41.21	66.79	74.12	65.98	63.84
QLLM 2026.04		77.48	52.99	39.33	-	71.38	65.98	61.43
FP16 2024.06		77.47	52.48	41.46	73.08	73	67.07	64.09
Atom 2025.04		77.37	56.73	42.32	67.62	74.07	68.27	65.46
DuQuant 2026.04		77.31	55.6	41.55	66.61	73.68	66.06	63.47
DuQuant 2024.06		77.26	58.04	41.55	67.55	73.62	66.69	64.12
RUQuant 2026.04		77.04	52.97	40.61	70.4	72.71	67.88	63.6
DuQuant 2026.04		76.99	52.99	40.87	70.4	72.49	67.32	63.51
QuaRot 2025.04		76.93	52.1	40.7	68.1	72.7	62.51	62.99
RUQuant 2026.04		76.93	70.45	44.97	75.2	73.36	66.14	67.84
FP16 2026.04		76.88	53.54	40.53	71.13	72.96	67.25	63.72
FP16 2026.04		76.88	53.54	40.53	71.13	72.96	67.25	63.72
QuaRot 2025.04		76.61	55.3	41.64	67.09	73.02	65.59	64.24
OmniQuant 2026.04		76.55	53.83	40.96	68.75	55.89	65.59	60.26
DuQuant 2024.06		76.44	50.04	38.99	70.98	69.39	64.72	61.76
DuQuant+LWC 2024.06		76.22	50.04	38.31	70.09	69.82	62.59	61.18
RUQuant 2026.04		76.12	69.53	42.92	71.93	74.13	67.01	66.94
RUQuant 2026.04		75.84	52.82	39.51	69.57	70.06	63.61	61.9
DuQuant 2026.04		75.68	68.48	41.81	71.99	73.07	66.22	66.21
SmoothQuant 2026.04		75.57	53.62	39.93	69.54	71.76	66.14	62.76
DuQuant 2026.04		75.24	51.89	36.77	67.86	69.54	62.12	60.57
RUQuant 2026.04		74.65	51.18	38.65	68.59	69.65	61.72	60.74
Atom 2024.06		74.48	51.6	40.61	73.76	73.78	62.12	62.73
QLLM 2024.06		73.83	50.67	38.4	-	67.91	58.56	57.87
QLLM 2024.06		73.56	52.06	39.68	-	70.94	62.9	59.83
Ours 2025.04		72.09	48.57	34.13	62.54	62.63	64.88	57.47
Ours 2025.04		71.98	49.92	36.26	65.9	60.52	61.8	57.73
Atom 2024.06		71.98	49.07	40.02	66.85	70.45	58.64	59.5
OmniQuant 2024.06		71.81	48.02	35.92	73.27	66.81	59.51	59.22
Atom 2024.06		71.44	47.74	35.49	67.71	63.89	55.01	56.88
QLLM 2024.06		71.38	47.6	34.3	-	63.7	59.43	55.28
Atom 2024.06		71.38	49.07	36.69	64.53	68	58.56	58.04
OmniQuant 2024.06		71.21	49.45	34.47	65.33	64.65	59.19	56.63
AffineQuant 2024.06		70.84	49.41	37.12	70.12	65.53	58.64	58.61
QLLM 2026.04		70.46	48.48	34.39	-	62.8	55.41	54.31
OmniQuant 2026.04		69.8	47.22	33.79	65.47	59.34	55.49	55.19
OmniQuant 2024.06		69.69	47.39	33.1	62.84	58.96	55.8	54.37
AffineQuant 2024.06		69.37	42.55	31.91	63.73	57.65	55.33	53.42
QLLM 2024.06		68.77	45.2	31.14	-	57.43	56.67	51.84
OS+ 2024.06		68.06	43.98	35.32	62.75	50.73	54.3	52.52
QLLM 2026.04		67.68	44.4	30.89	-	58.45	56.59	51.6
OS+ 2024.06		67.63	46.17	34.4	60.7	54.32	52.64	52.62
AffineQuant 2024.06		66.32	43.9	29.61	64.1	56.88	54.7	52.58
OmniQuant 2024.06		66.15	45.2	31.14	63.51	56.44	53.43	52.65
OmniQuant 2026.04		65.61	44.28	30.38	62.66	53.51	51.85	51.38
SmoothQuant 2024.06		64.47	40.44	29.82	59.38	39.9	52.24	47.71
QuaRot 2025.04		64.42	41.5	28.75	63.36	48.49	56.67	50.53
OS+ 2024.06		63	40.32	30.38	60.34	53.61	51.54	49.86
OS+ 2024.06		62.73	39.98	30.29	60.21	44.39	52.96	48.43
SmoothQuant 2026.04		62.3	40.28	30.72	60.49	42.24	49.96	47.67
Atom 2025.04		61.15	40.36	29.52	61.56	45.62	51.22	48.24
SmoothQuant 2024.06		61.04	39.18	30.8	61.8	52.29	51.06	49.36
SmoothQuant 2026.04		60.17	35.23	27.13	57.92	37.08	49.57	44.52
Atom 2025.04		59.68	36.62	29.01	58.56	44.84	52.01	46.79
QuaRot 2025.04		59.09	34.6	24.23	62.11	35.03	51.38	44.41
SmoothQuant 2024.06		58.65	35.53	27.73	60.42	35.56	48.06	44.83
SmoothQuant 2026.04		54.57	31.9	24.23	52.72	31.26	51.14	40.97
OmniQuant 2026.04		50.22	26.94	24.57	37.98	26.55	50.2	36.08
SmoothQuant 2024.06		49.8	30.4	25.8	49.1	27.4	48	38.41