Our new X account is live! Follow @wizwand_team for updates

Zero-shot Language Modeling on LM Evaluation Harness

80.66WG

LO-BCQ

Updated 4d ago

Evaluation Results

Method	Links
LO-BCQ 2025.02		80.66	49.28	84.93	81.34	65.18	72.28	0.12
BF16 2025.02		79.95	48.8	85.23	81.56	65.27	72.16	-
LO-BCQ 2025.02		78.77	49	82.82	81.45	64.21	71.25	0.91
LO-BCQ 2025.02		78.37	49.28	84.03	81.45	64.76	71.58	0.58
VSQ 2025.02		77.27	47.85	82.29	79.82	61.4	69.73	2.43
MX4 2025.02		76.4	48.04	82.41	80.58	63.24	70.13	2.03
MXFP4 2025.02		76.32	47.75	83.06	80.58	63.24	70.19	1.97
QuaRot 2025.02		76.24	-	-	82.43	-	-	-
BF16 2025.02		70.64	40.67	76.54	79.16	57.11	64.82	-
LO-BCQ 2025.02		70.17	39.43	77.09	78.62	56.6	64.38	0.44
LO-BCQ 2025.02		69.77	42.58	77.43	77.09	56.51	64.68	0.97
VSQ 2025.02		69.61	40.57	65.81	77.2	54.82	61.6	3.22
BF16 2025.02		69.38	44.4	79.29	78.07	57.1	65.65	-
LO-BCQ 2025.02		69.3	39.62	75.35	78.89	56.64	63.96	0.86
LO-BCQ 2025.02		69.14	40.48	75.41	78.24	56.06	63.87	0.95
LO-BCQ 2025.02		68.9	42.49	77.58	77.09	55.93	64.4	1.25
LO-BCQ 2025.02		68.9	43.73	77.86	77.86	56.52	64.97	0.68
MX4 2025.02		67.96	39.04	72.26	77.86	54.77	62.38	2.44
BF16 2025.02		67.88	41.34	68.32	78.78	54.16	62.1	-
LO-BCQ 2025.02		67.72	39.43	69.45	77.75	53.71	61.61	0.49
MXFP4 2025.02		67.48	41.34	74	77.53	54.22	62.91	2.74
MXFP4 2025.02		67.01	39.71	65.35	76.12	50.22	59.68	2.42
LO-BCQ 2025.02		67	39.62	69.3	77.37	53.51	61.36	0.74
VSQ 2025.02		66.93	40.86	63.91	76.28	51.38	59.87	2.23
LO-BCQ 2025.02		66.85	40.48	69.2	77.31	53.06	61.38	0.72
MX4 2025.02		66.22	41.43	73.98	77.04	55.19	62.77	2.88
MX4 2025.02		65.11	38.28	66.27	75.63	50.77	59.21	2.89
MXFP4 2025.02		64.17	39.14	69.61	75.68	47.6	59.24	5.58
QuaRot 2025.02		63.77	-	-	76.77	-	-	-
VSQ 2025.02		55.49	31.39	65.75	67.3	43.51	52.69	12.96