Share your thoughts, 1 month free Claude Pro on usSee more

Zero-shot Common Sense Reasoning Suite (PIQA, ARC-e, ARC-c, BoolQ, HellaSwag, Winogrande) (test dev)

79.35PIQA

Kirin

Updated 3mo ago

Evaluation Results

Method
Kirin 2026.02	79.35	56.4	43.94	66.21	75.94	68.98	65.14
SpikeQuant 2026.02	79.33	56.48	44.03	66.15	75.53	68.11	64.94
Fp16 2026.02	79.05	59.85	44.62	68.53	76.22	70.09	66.39
Atom 2026.02	77.69	57.58	42.92	67.46	73.77	68.51	64.66
Fp16 2026.02	77.37	52.53	41.38	73.12	72.99	66.85	64.04
SpikeQuant 2026.02	77.09	55.39	40.36	73.49	72.09	65.27	63.95
Kirin 2026.02	76.77	54.55	40.53	73.91	72.1	66.22	64.01
Quantization 2026.02	76.71	55.13	41.47	67.03	73.09	65.19	63.1
Atom 2026.02	76.28	52.1	38.99	69.79	69.81	63.69	61.78
Quantization 2026.02	75.46	53.96	40.19	68.07	69.21	61.56	61.41
Fp16 2026.02	74.81	54.34	31.31	60.28	60.59	60.93	57.04
Kirin 2026.02	74.81	53.91	31.48	61.69	58.83	59.91	56.77
SpikeQuant 2026.02	74.21	53.79	31.66	61.19	59.11	59.75	56.62
Fp16 2026.02	72.42	50.8	29.69	57.68	53.73	59.67	54
Kirin 2026.02	71.86	48.78	28.67	53.66	52.25	57.77	52.17
SpikeQuant 2026.02	71.49	49.66	27.22	53.27	51.46	58.56	51.94
OPTQ 2026.02	71.38	48.19	27.82	55.43	49.86	54.82	51.25
Quantization 2026.02	71.16	50.84	28.75	59.85	55.57	60.46	54.44
OmniQuant 2026.02	69.69	47.39	33.1	62.84	58.96	55.8	54.63
OPTQ 2026.02	68.34	46.17	27.65	51.57	46.53	55.26	49.25
Quantization 2026.02	67.79	42.93	25.26	49.51	45.94	54.93	47.73
SpikeLLM 2026.02	66.49	55.3	30.12	64.16	47.43	51.54	52.51
OmniQuant 2026.02	66.15	45.2	31.14	63.51	56.44	53.43	52.65
SpikeLLM 2026.02	64.47	48.74	27.3	63.27	43.29	56.83	50.65
SmoothQuant 2026.02	64.47	41.75	30.89	62.29	46.68	51.7	49.63
LRQuant 2026.02	64.04	47.81	20.65	54.1	33.59	53.51	45.62
OneBit 2026.02	63.87	43.39	24.4	54.28	38.18	51.67	45.97
SmoothQuant 2026.02	63.11	40.03	31.57	58.47	43.38	52.8	48.23
OneBit 2026.02	62.57	41.25	24.06	59.45	34.26	51.14	45.46
LRQuant 2026.02	60.23	43.22	19.8	57.98	30.71	51.78	43.95