Share your thoughts, 1 month free Claude Pro on usSee more

Zero-shot Reasoning on ARC-C, ARC-E, BoolQ, Hella, OBQA, PIQA, SIQA, Wino

65.53ARC-C Accuracy

Baseline

Updated 3d ago

Evaluation Results

Method
Baseline 2026.04	65.53	84.26	88.69	86.46	50	84.22	46.93	77.51	72.95
Baseline 2026.04	60.6	82.7	89.4	78.7	46.2	80.1	44.5	72.8	69.4
DASH-Q 2026.04	58.36	80.81	87.06	80.35	46.8	81.99	45.39	76.95	69.72
Baseline 2026.04	55.2	79.71	85.41	79.54	45	81.07	42.27	73.88	67.76
DASH-Q 2026.04	54.1	78.41	83.82	78.52	43.8	81.01	42.37	73.16	66.9
AWQ 2026.04	54.01	78.58	84.31	78.92	43.6	80.14	42.27	74.74	67.07
DASH-Q 2026.04	53.75	79.88	86.12	68.35	42.8	75.46	39.1	69.93	64.42
RTN 2026.04	53.41	77.06	84.53	78.15	43	81.01	41.45	74.27	66.61
Baseline 2026.04	53.33	66.12	88.53	79.8	50.4	78.02	42.53	76.4	66.89
QuIP 2026.04	53.24	74.75	80.09	78.47	43.8	79.43	41.86	73.24	65.61
OWQ 2026.04	53.07	79	84.07	78.91	44.2	80.52	41.56	74.51	66.98
QuaRot 2026.04	52.99	78.07	85.29	78.55	43.2	79.76	41.76	73.95	66.7
GPTQ 2026.04	52.56	76.85	84.53	77	41	79.65	39.71	73.72	65.63
QuIP 2026.04	52.3	77.9	85.17	78.79	43.6	80.41	41.2	74.35	66.72
DASH-Q 2026.04	52.13	64.1	86.18	76.79	45.8	77.31	41.5	73.4	64.65
QuaRot 2026.04	51.37	75.93	80.92	76.36	41.4	78.78	40.02	70.8	64.45
OWQ 2026.04	51.28	74.62	77.68	72.65	40	79	36.9	61.4	61.69
QuaRot 2026.04	51.02	75.08	79.63	77.96	44.8	77.58	40.74	70.24	64.63
DASH-Q 2026.04	50	75.13	82.78	76.27	42.6	80.03	40.38	72.69	64.99
QuIP 2026.04	49.06	76.18	84.01	76.23	41	78.29	40.17	72.77	64.71
OWQ 2026.04	48.63	74.75	82.57	76.99	42	78.89	39.51	72.61	64.49
OWQ 2026.04	46.93	55.09	84.5	74.18	44.6	72.47	39.41	67.8	60.62
AWQ 2026.04	46.16	71.59	80.92	75.27	41	79.43	39.15	71.67	63.15
Baseline 2026.04	45.82	69.53	73.18	77.17	43.8	79.49	39.46	70.32	62.35
RTN 2026.04	44.54	65.74	75.63	71.38	37.4	77.04	35.98	69.69	59.67
QuaRot 2026.04	43.77	54.5	83.85	73.1	44.2	72.52	37.36	63.93	59.16
AWQ 2026.04	43.6	66.25	69.91	65.21	36	76.71	36.03	57.54	56.41
AWQ 2026.04	42.66	55.26	72.29	66.88	39	72.47	35.77	66.77	56.39
GPTQ 2026.04	39.51	52.23	81.13	68.17	40.4	68.06	36.69	65.19	56.42
DASH-Q 2026.04	38.57	66.79	76.12	62.39	34.8	72.91	34.14	66.46	56.52
AWQ 2026.04	37.54	61.07	68.29	61.67	35.4	71.6	37.1	57.22	53.74
QuaRot 2026.04	37.29	58.5	67.71	64.76	35.4	71.65	36.13	60.06	53.94
DASH-Q 2026.04	36.35	63.38	63.27	66.13	39.4	76.06	35.72	64.4	55.59
QuIP 2026.04	36.01	54.34	71.47	64.86	35.6	70.51	37.41	60.3	53.81
OWQ 2026.04	36.01	56.9	75.02	63.15	36	72.74	34.9	59.27	54.25
OWQ 2026.04	34.64	60.61	58.65	65.15	34.2	73.07	35.01	61.64	52.87
GPTQ 2026.04	33.7	52.1	72.45	59.26	36.2	68.66	35.36	56.35	51.76
RTN 2026.04	32.59	41.71	58.47	43.76	29.2	57.78	34.39	55.01	44.11
QuIP 2026.04	31.57	50.67	63.58	59.99	32.4	71.93	33.88	58.41	50.3
QuaRot 2026.04	30.29	55.51	58.17	55.73	30.8	71.38	35.31	56.51	49.21
QuIP 2026.04	29.1	38.26	50.83	41.48	30.2	58.81	32.5	51.46	41.58
QuaRot 2026.04	27.39	36.62	49.33	41.29	29.2	57.94	33.67	52.57	41
AWQ 2026.04	26.11	33.21	50.12	36.38	24	57.51	34.08	50.04	38.93
QuIP 2026.04	26.02	36.07	50.06	37.01	28	57.07	33.32	51.62	39.9
GPTQ 2026.04	25.94	26.47	38.17	25.96	30.6	53.97	34.49	49.64	35.66
OWQ 2026.04	25.94	38.72	51.96	46.79	30.8	58.65	34.54	52.64	42.51
RTN 2026.04	25.68	25.42	40.09	26.99	24	51.2	35.01	48.22	34.58
GPTQ 2026.04	25.43	35.65	59.79	40.57	23.8	60.45	34.7	51.14	41.44
GPTQ 2026.04	25.43	30.93	44.19	30.35	25.4	58.32	35.52	50.12	37.53
RTN 2026.04	24.57	27.19	37.83	26.71	28.2	52.23	34.08	51.07	35.23
AWQ 2026.04	24.49	29.8	38.44	26.77	28	52.94	33.88	47.51	35.23
GPTQ 2026.04	24.15	44.28	39.82	37.55	28.6	60.94	34.14	53.99	40.43
RTN 2026.04	23.46	32.87	48.1	31.75	26	56.69	33.47	48.54	37.61
RTN 2026.04	22.44	29.55	43.91	26.96	23.4	53.26	35.62	52.01	35.89