Share your thoughts, 1 month free Claude Pro on usSee more

Question Answering on HellaSwag, WinoGrande, LAMBADA, PIQA, OpenBookQA, ARC-E, and ARC-C

79.2HellaSwag Accuracy

FP16

Updated 1mo ago

Evaluation Results

Method	Links
FP16 2026.04		79.2	74.1	73.2	80.9	43	79.6	55.2	69.3
FP16 2026.04		79.1	72.9	75.5	80.7	44.6	77.6	53.5	69.1
DuQuant++ 2026.04		77.3	70.6	70.2	79.3	42	78	52.1	67.1
DuQuant++* 2026.04		77	71.7	72.3	79.5	42.4	77	51.7	67.4
DuQuant++ 2026.04		76.6	72.2	73	78.7	41.8	74.7	48.2	66.5
DuQuant++* 2026.04		76.6	71.7	74	79.5	42.6	75.3	50.1	67.1
MR-GPTQ 2026.04		76.4	69.7	71.8	77.8	43.4	74.5	49.4	66.1
MR-GPTQ 2026.04		76.2	70.6	70.4	78.4	43.4	77.2	52.6	67
FP16 2026.04		75.7	71.4	71.5	78.6	42.8	78.8	55.6	67.8
FlatQuant 2026.04		75.7	71.1	69.6	78.7	41.8	78.3	52.1	66.8
FlatQuant 2026.04		75.4	71.9	68.6	78.9	42.6	74	47.2	65.5
FP16 2026.04		74	69.5	69.6	77.7	40.4	71.8	46.3	64.2
DuQuant++ 2026.04		73.8	72.1	69.7	76.1	41.2	75.7	52.8	65.9
DuQuant++* 2026.04		73.7	71.6	70.3	76.7	41.2	77.2	50.7	65.9
MR-GPTQ 2026.04		73.2	69	68.3	76	43	75.8	51.3	65.2
FlatQuant 2026.04		73	71.1	67.3	76.6	41.2	75.8	49.4	64.9
QuaRot 2026.04		72.4	68.4	62	76	39.8	71.8	46.3	62.4
QuaRot* 2026.04		72.3	68.5	74.6	76.8	43.2	75	47.3	65.4
QuaRot* 2026.04		71.8	70.7	72.5	76.9	39.4	76.3	51.9	65.6
DuQuant++* 2026.04		70.6	65.8	67.8	75	42.2	68.5	42.5	61.8
QuaRot 2026.04		70.4	70	68.1	76.1	41.6	69.8	44	62.9
DuQuant++ 2026.04		70.3	65.1	65.3	75.4	39.2	67.9	42.9	60.9
MR-GPTQ 2026.04		70.1	65	66.6	76.1	37.8	70	42.8	61.2
QuaRot* 2026.04		69.2	68.8	71.4	75.4	39.6	75.8	48.2	64.1
QuaRot 2026.04		68	67.3	67.4	75.5	40.2	72.6	45.9	62.4
FlatQuant 2026.04		67.9	65.1	61.6	74.3	37.2	66.5	40.4	59
QuaRot* 2026.04		66.1	63.7	62.2	73.3	37.6	62.2	36.6	57.4
QuaRot 2026.04		57.9	59.8	50.8	70.3	32.6	59.2	34.8	52.2