Our new X account is live! Follow @wizwand_team for updates

Zero-shot NLP Reasoning Suite (BoolQ, HellaSwag, ARC, PIQA, OBQA)

68.59Average Accuracy (Zero-Shot Suite)

LLaMA-7B

Updated 4d ago

Evaluation Results

Method	Links
LLaMA-7B 2023.05		68.59	76.5	79.8	76.1	70.1	72.8	47.6	57.2
Original Llama-7B 2024.10		66.07	-	-	-	-	-	-	-
LLaMA-13B* 2023.05		64.97	68.47	78.89	76.24	70.09	74.58	44.54	42
Block 2023.05		64.02	70.31	77.91	75.16	67.88	71.09	42.41	43.4
LLaMA-7B* 2023.05		63.25	73.18	78.35	72.99	67.01	67.45	41.38	42.4
BLAST16 2024.10		62.94	-	-	-	-	-	-	-
L2 2023.05		62.12	73.24	76.77	71.86	64.64	67.59	39.93	40.8
Block 2023.05		61.79	67.68	77.15	73.41	65.11	68.35	38.4	42.4
Channel 2023.05		60.24	69.2	76.55	68.89	66.38	62.08	38.99	39.6
Element2 2023.05		60.07	69.54	76.44	68.11	65.11	63.43	37.88	40
Random 2023.05		59.78	64.19	76.06	68.89	63.3	66.88	38.31	40.8
Low-Rank 2024.10		59.57	-	-	-	-	-	-	-
Element1 2023.05		59.23	64.62	77.2	68.8	63.14	64.31	36.77	39.8
Vector 2023.05		57.23	65.75	74.7	64.52	59.35	60.65	36.26	39.4
Random 2023.05		57.09	63.33	73.18	63.54	60.85	64.44	36.26	38
Element2 2023.05		56.82	59.39	75.57	65.34	61.33	59.18	37.12	39.8
Element1 2023.05		56.69	57.06	75.68	66.8	59.83	60.94	36.52	40
BLAST16 2024.10		56.22	-	-	-	-	-	-	-
Channel 2023.05		55.57	59.08	73.39	64.02	60.54	57.95	35.58	38.4
Vector 2023.05		53.25	61.44	71.71	57.27	54.22	55.77	33.96	38.4
Random 2023.05		52.69	61.83	71.33	56.26	54.46	57.07	32.85	35
L2 2023.05		51.08	61.5	67.57	52.9	57.54	50.13	31.14	36.8
Channel 2023.05		50.29	62.39	66.87	49.17	58.96	49.62	31.83	33.2
Monarch (BLR) 2024.10		48.91	-	-	-	-	-	-	-
Low-Rank 2024.10		48.4	-	-	-	-	-	-	-
Channel 2023.05		45.38	62.75	62.73	41.4	51.07	41.38	27.9	30.4
L2 2023.05		42.65	59.66	58	37.04	52.41	33.12	28.58	29.8
Monarch (BLR) 2024.10		35.03	-	-	-	-	-	-	-
Block-Diagonal 2024.10		34.86	-	-	-	-	-	-	-