Our new X account is live! Follow @wizwand_team for updates

Question Answering on QA Suite Zero-shot (PIQA, ARC-E, ARC-C, BoolQ, HellaSwag, WinoGrande)

80.79PIQA Accuracy

FP16

Updated 4d ago

Evaluation Results

Method
FP16 2024.06	80.79	58.71	46.24	82.29	80.72	77.5	71.04
FP16 2024.06	80.08	58.92	45.47	68.44	79.21	72.53	67.44
DuQuant+LWC 2024.06	79.98	58.29	44.8	77.89	79.22	72.21	68.73
DuQuant 2024.06	79.71	57.95	45.05	79.82	78.66	72.29	68.91
FP16 2024.06	79.1	59.89	44.45	68.01	76.21	70.31	66.33
FP16 2025.04	79.05	59.89	44.71	68.47	76.23	70.24	66.43
FP16 2025.04	79	57.95	44.28	69.02	76.58	69.69	66.09
DuQuant+LWC 2024.06	78.73	56.52	43.17	68.84	77.53	70.96	65.96
DuQuant 2024.06	78.56	56.99	42.32	66.73	76.7	69.61	65.15
Atom 2025.04	77.64	58.38	41.81	68.5	73.75	65.98	64.64
DuQuant+LWC 2024.06	77.64	57.32	41.21	66.79	74.12	65.98	63.84
FP16 2024.06	77.47	52.48	41.46	73.08	73	67.07	64.09
Atom 2025.04	77.37	56.73	42.32	67.62	74.07	68.27	65.46
DuQuant 2024.06	77.26	58.04	41.55	67.55	73.62	66.69	64.12
QuaRot 2025.04	76.93	52.1	40.7	68.1	72.7	62.51	62.99
QuaRot 2025.04	76.61	55.3	41.64	67.09	73.02	65.59	64.24
DuQuant 2024.06	76.44	50.04	38.99	70.98	69.39	64.72	61.76
DuQuant+LWC 2024.06	76.22	50.04	38.31	70.09	69.82	62.59	61.18
Atom 2024.06	74.48	51.6	40.61	73.76	73.78	62.12	62.73
QLLM 2024.06	73.83	50.67	38.4	-	67.91	58.56	57.87
QLLM 2024.06	73.56	52.06	39.68	-	70.94	62.9	59.83
Ours 2025.04	72.09	48.57	34.13	62.54	62.63	64.88	57.47
Ours 2025.04	71.98	49.92	36.26	65.9	60.52	61.8	57.73
Atom 2024.06	71.98	49.07	40.02	66.85	70.45	58.64	59.5
OmniQuant 2024.06	71.81	48.02	35.92	73.27	66.81	59.51	59.22
Atom 2024.06	71.44	47.74	35.49	67.71	63.89	55.01	56.88
QLLM 2024.06	71.38	47.6	34.3	-	63.7	59.43	55.28
Atom 2024.06	71.38	49.07	36.69	64.53	68	58.56	58.04
OmniQuant 2024.06	71.21	49.45	34.47	65.33	64.65	59.19	56.63
AffineQuant 2024.06	70.84	49.41	37.12	70.12	65.53	58.64	58.61
OmniQuant 2024.06	69.69	47.39	33.1	62.84	58.96	55.8	54.37
AffineQuant 2024.06	69.37	42.55	31.91	63.73	57.65	55.33	53.42
QLLM 2024.06	68.77	45.2	31.14	-	57.43	56.67	51.84
OS+ 2024.06	68.06	43.98	35.32	62.75	50.73	54.3	52.52
OS+ 2024.06	67.63	46.17	34.4	60.7	54.32	52.64	52.62
AffineQuant 2024.06	66.32	43.9	29.61	64.1	56.88	54.7	52.58
OmniQuant 2024.06	66.15	45.2	31.14	63.51	56.44	53.43	52.65
SmoothQuant 2024.06	64.47	40.44	29.82	59.38	39.9	52.24	47.71
QuaRot 2025.04	64.42	41.5	28.75	63.36	48.49	56.67	50.53
OS+ 2024.06	63	40.32	30.38	60.34	53.61	51.54	49.86
OS+ 2024.06	62.73	39.98	30.29	60.21	44.39	52.96	48.43
Atom 2025.04	61.15	40.36	29.52	61.56	45.62	51.22	48.24
SmoothQuant 2024.06	61.04	39.18	30.8	61.8	52.29	51.06	49.36
Atom 2025.04	59.68	36.62	29.01	58.56	44.84	52.01	46.79
QuaRot 2025.04	59.09	34.6	24.23	62.11	35.03	51.38	44.41
SmoothQuant 2024.06	58.65	35.53	27.73	60.42	35.56	48.06	44.83
SmoothQuant 2024.06	49.8	30.4	25.8	49.1	27.4	48	38.41