Our new X account is live! Follow @wizwand_team for updates

Zero-shot Classification on MMLU, OBQA, ARC, PIQA, HellaSwag, WinoGrande

37.1MMLU (5-shot)

LLAMA-7B

Updated 4d ago

Evaluation Results

Method	Links
LLAMA-7B 2023.05		37.1	42.4	67.45	67.01	67.45	78.35	72.99	65.34
LoRAPrune 2023.05		36.81	38.6	70.2	66.77	41.89	77.48	68.64	62.7
LoRAPrune-8bit 2023.05		36.45	38.1	70.25	65.93	41.43	77.1	68.9	60.29
LLM-Pruner 2023.05		33.67	38.8	70.62	65.82	40.7	77.37	66.6	62.36
LoRAPrune 2023.05		33.6	36.2	69.61	62.75	41.21	77.48	66.68	58.98
LoRAPrune-8bit 2023.05		33.36	36.4	69.48	62.31	41.93	77.4	65.91	58.9
Compresso 2023.05		31.9	36.4	68.64	67.8	37.97	75.46	53.44	59.82
LLM-Pruner 2023.05		30.67	34.86	66.2	63.85	40.55	75.6	65.12	57.7
WANDA 2023.05		28.35	35.26	68.96	64.01	38.46	74.8	52.63	58.68
WANDA 2023.05		27.9	34.9	65.06	61.16	39.44	74.38	60.84	55.96
Magnitude 2023.05		27.84	35.8	65.36	61.33	38.74	74.87	63.9	56.67
LoRAPrune 2023.05		27.84	35.8	60.38	56.97	40.12	75.39	60.21	54.81
LoRAPrune-8bit 2023.05		27.71	35.3	60.54	56.13	40.58	74.89	59.86	54.55
Compresso 2023.05		27.68	29.8	66.23	64.8	37.2	75.63	49.16	53.79
Magnitude 2023.05		26.38	33.67	65.58	60.79	37.47	73.15	60.35	55.16
LLM-Pruner 2023.05		25.6	33.12	55.36	56.12	34.98	73.25	58.6	51.9
WANDA 2023.05		24.15	28.78	52.68	55.98	34.2	70.38	54.12	49.35
Magnitude 2023.05		21.84	30.26	53.61	55.86	36.98	67.1	53.1	49.48