Share your thoughts, 1 month free Claude Pro on usSee more

Zero-shot Common-sense Reasoning Suite (BoolQ, PIQA, HellaSwag, WinoGrande, ARC-e, ARC-c, OBQA)

70.185Avg Accuracy

LLaMA3-8B

Updated 1mo ago

Evaluation Results

Method
LLaMA3-8B 2025.02	70.185	81.101	79.489	79.167	73.402	80.093	53.242	44.8
SWM 2025.02	66.954	76.789	77.639	73.77	71.744	76.599	50.939	41.2
LLaMA2-7B 2025.02	66.825	77.706	78.074	76.021	69.14	76.305	46.331	44.2
LLM-Pruner 2025.02	65.941	74.037	79.489	74.268	70.324	74.285	46.587	42.6
Vicuna-7B-v1.3 2025.02	65.902	78.104	77.312	73.939	69.376	74.327	44.454	43.8
Shortened-LLM 2025.02	63.316	66.208	78.074	72.695	62.747	75.295	44.795	43.4
LLaMA2-7B 2026.03	62.91	71.1	78.4	72.96	67.25	69.36	40.53	40.8
SWM 2025.02	62.628	77.431	74.755	69.04	68.745	69.318	38.908	40.2
SWM 2025.02	61.944	69.45	73.667	70.484	67.088	69.108	41.212	42.6
FLAP 2025.02	61.664	73.52	74.81	68.76	66.46	69.11	38.99	40
Shortened-LLM 2025.02	60.865	72.355	74.701	67.576	64.562	70.034	38.225	38.6
Wanda-sp 2025.02	60.747	62.6	76.44	70.66	63.77	69.61	42.15	40
LLM-Pruner 2025.02	59.543	67.645	76.115	66.66	63.931	65.446	36.604	40.4
LLM-Pruner 2025.02	59.268	63.731	77.476	67.128	61.878	65.783	38.481	40.4
Wanda-sp 2025.02	58.976	63.27	73.78	68.62	63.93	67.21	38.82	37.2
Shortened-LLM 2025.02	58.445	61.56	76.061	67.994	58.8	68.813	37.884	38
FLAP 2025.02	58.059	72.05	73.39	64.69	64.72	62.25	32.51	36.8
SWM 2025.02	57.652	67.554	73.83	61.472	62.747	64.352	36.007	37.6
SLEB 2025.02	57.506	62.171	73.286	64.748	63.062	64.562	37.713	37
Shortened-LLM 2025.02	56.871	63.18	72.851	62.985	58.09	66.877	37.116	37
SLEB 2025.02	56.733	62.875	73.939	63.951	59.747	63.468	35.154	38
LLM-Pruner 2025.02	56.557	64.465	74.048	61.8	59.353	64.646	34.386	37.2
SWM 2025.02	56.25	69.235	70.294	60.705	62.273	60.227	33.618	37.4
LLM-Pruner 2025.02	54.596	63.976	73.069	59.56	58.564	56.524	32.679	37.8
FLAP 2025.02	54.203	57.86	69.64	59.12	63.3	57.83	35.67	36
Shortened-LLM 2025.02	53.464	64.281	70.783	56.722	57.38	59.596	31.485	34
LLM-Pruner 2025.02	52.366	45.26	74.76	60.29	59.35	57.28	32.42	37.2
Shortened-LLM 2025.02	52.317	62.171	71.926	54.8	51.776	59.259	30.887	35.4
SWM 2025.02	51.869	63.119	65.452	56.503	58.879	52.02	31.911	35.2
FLAP 2025.02	51.194	66.97	67.85	52.1	61.48	49.49	28.07	32.4
Wanda-sp 2025.02	50.35	59.79	68.82	53.14	54.06	52.27	31.57	32.8
GradMAP 2026.02	48.92	-	-	-	-	-	-	-
GradMAP 2026.02	48.68	-	-	-	-	-	-	-
SLEB 2025.02	47.499	54.801	67.41	46.545	53.197	48.527	29.01	33
SLEB 2025.02	46.147	59.755	64.635	45.061	51.539	47.306	27.133	27.6
ShortGPT 2026.02	45.5	-	-	-	-	-	-	-
Wanda-sp 2025.02	43.991	50.76	60.45	43.06	55.96	43.52	26.19	28
GPrune-LLM (Wanda-sp) 2026.03	41.86	61.25	56.53	31.47	51.3	31.82	29.27	31.4
GPrune-LLM (FLAP) 2026.03	41.26	45.41	59.52	36.42	49.72	36.95	27.39	33.4
GradMAP 2026.02	39.6	-	-	-	-	-	-	-
GradMAP 2026.02	39.21	-	-	-	-	-	-	-
FLAP 2026.03	38.96	43.09	58.22	30.23	48.22	33.54	26.45	33
LLM-Pruner 2026.03	38.85	59.51	55.88	26.67	47.91	27.99	25	29
ShortGPT 2026.02	38.31	-	-	-	-	-	-	-
GradMAP 2026.02	38.1	-	-	-	-	-	-	-
GradMAP 2026.02	37.66	-	-	-	-	-	-	-
SLEB 2026.02	37.62	-	-	-	-	-	-	-
GradMAP 2026.02	37.51	-	-	-	-	-	-	-
GradMAP 2026.02	37.32	-	-	-	-	-	-	-
Wanda-sp 2026.03	37.1	47.55	52.77	25.98	49.09	26.85	25.85	31.6
SLEB 2026.02	36.58	-	-	-	-	-	-	-
SLEB 2025.02	36.449	46.758	51.85	26.17	51.223	25.505	28.84	24.8
SLEB 2026.02	35.41	-	-	-	-	-	-	-
SLEB 2025.02	35.399	37.829	53.264	25.921	49.961	25.926	29.096	25.8
FLAP 2025.02	34.853	37.83	52.18	26.36	49.96	26.81	24.83	26
ShortGPT 2026.02	34.49	-	-	-	-	-	-	-
FLAP 2025.02	34.147	37.83	52.34	26.05	47.99	24.79	24.83	25.2
GradMAP 2026.02	32.75	-	-	-	-	-	-	-
GradMAP 2026.02	32.69	-	-	-	-	-	-	-
SLEB 2026.02	32.2	-	-	-	-	-	-	-
ShortGPT 2026.02	30.46	-	-	-	-	-	-	-
SLEB 2026.02	30.1	-	-	-	-	-	-	-
ShortGPT 2026.02	29.26	-	-	-	-	-	-	-