Share your thoughts, 1 month free Claude Pro on usSee more

Zero-shot Classification on MMLU, OBQA, ARC, PIQA, HellaSwag, WinoGrande

37.1MMLU (5-shot)

LLAMA-7B

Updated 1mo ago

Evaluation Results

Method	Links
LLAMA-7B 2023.05		37.1	42.4	67.45	67.01	67.45	78.35	72.99	65.34
LoRAPrune 2023.05		36.81	38.6	70.2	66.77	41.89	77.48	68.64	62.7
LoRAPrune-8bit 2023.05		36.45	38.1	70.25	65.93	41.43	77.1	68.9	60.29
LLM-Pruner 2023.05		33.67	38.8	70.62	65.82	40.7	77.37	66.6	62.36
LoRAPrune 2023.05		33.6	36.2	69.61	62.75	41.21	77.48	66.68	58.98
LoRAPrune-8bit 2023.05		33.36	36.4	69.48	62.31	41.93	77.4	65.91	58.9
Compresso 2023.05		31.9	36.4	68.64	67.8	37.97	75.46	53.44	59.82
LLM-Pruner 2023.05		30.67	34.86	66.2	63.85	40.55	75.6	65.12	57.7
WANDA 2023.05		28.35	35.26	68.96	64.01	38.46	74.8	52.63	58.68
WANDA 2023.05		27.9	34.9	65.06	61.16	39.44	74.38	60.84	55.96
Magnitude 2023.05		27.84	35.8	65.36	61.33	38.74	74.87	63.9	56.67
LoRAPrune 2023.05		27.84	35.8	60.38	56.97	40.12	75.39	60.21	54.81
LoRAPrune-8bit 2023.05		27.71	35.3	60.54	56.13	40.58	74.89	59.86	54.55
Compresso 2023.05		27.68	29.8	66.23	64.8	37.2	75.63	49.16	53.79
Magnitude 2023.05		26.38	33.67	65.58	60.79	37.47	73.15	60.35	55.16
LLM-Pruner 2023.05		25.6	33.12	55.36	56.12	34.98	73.25	58.6	51.9
WANDA 2023.05		24.15	28.78	52.68	55.98	34.2	70.38	54.12	49.35
Magnitude 2023.05		21.84	30.26	53.61	55.86	36.98	67.1	53.1	49.48