Share your thoughts, 1 month free Claude Pro on usSee more

Zero-shot Language Evaluation on WikiText2, BoolQ, PIQA, HellaSwag, WinoGrande, ARC, OBQA, and MTQA (test)

7.43WikiText2 Perplexity

SimDiff (MSSD)

Updated 1mo ago

Evaluation Results

Method
SimDiff (MSSD) 2026.04	7.43	62.23	74.21	52.23	67.72	62.88	38.31	27.2	26.6	51.42
SimDiff (MASD) 2026.04	7.45	62.42	74.92	52.44	66.77	60.73	37.8	27.8	27.24	51.26
ShortGPT 2026.04	8.3	62.17	73.88	50.99	65.43	61.24	36.35	26.6	26.16	50.35
SimDiff 2026.04	8.87	62.26	72.03	50.18	67.01	58.21	36.18	22.6	25.33	49.22
SimDiff (MASD) 2026.04	10.44	62.45	71.82	47.3	66.61	54.88	33.96	20.8	24.32	47.77
SimDiff (MSSD) 2026.04	11.5	62.45	69.86	46.75	65.19	52.95	32.17	21	24.82	46.9
ShortGPT 2026.04	13.58	62.75	74.16	44.26	53.35	70.75	34.13	29.6	29.98	49.87
SimDiff 2026.04	13.58	62.75	74.16	44.26	53.35	70.75	34.13	29.6	29.98	49.87
SimDiff 2026.04	18.42	62.23	70.13	37.36	53.59	62.16	26.88	25.4	24.62	45.3
ShortGPT 2026.04	20.03	62.63	71.27	47.29	62.98	53.45	34.3	23.8	24.05	47.47
SimDiff 2026.04	23.14	46.18	68.66	35.94	52.17	56.27	24.91	22.4	24.86	41.42
SimDiff 2026.04	26.34	59.97	67.85	34.77	53.67	60.23	25.43	23.2	25.76	43.86
ShortGPT 2026.04	26.34	59.97	67.85	34.77	53.67	60.23	25.43	23.2	25.76	43.86
SimDiff 2026.04	26.34	59.97	67.85	34.77	53.67	60.23	25.43	23.2	25.76	43.86
ShortGPT 2026.04	34.71	62.39	68.44	45.49	62.27	48.02	31.91	20.2	24.32	45.38
ShortGPT 2026.04	37.7	61.65	64.91	35.62	50.59	53.28	27.99	22.8	26.2	42.88
SimDiff 2026.04	38	43.18	63.17	30.26	50.43	48.82	20.48	19.6	22.45	37.3
SimDiff 2026.04	44.34	39.17	64.04	30.89	52.25	48.74	22.1	18.2	22.71	37.26
ShortGPT 2026.04	44.34	39.17	64.04	30.89	52.25	48.74	22.1	18.2	22.71	37.26
ShortGPT 2026.04	64.45	55.57	64.15	32.9	53.43	47.52	22.35	23	23.48	40.3
SimDiff 2026.04	87.7	41.19	60.94	28.97	51.38	44.61	18.52	17	22.58	35.65
ShortGPT 2026.04	110.23	47.16	59.9	30.07	51.7	41.08	20.65	18.4	22.35	36.41
SimDiff (MSSD) 2026.04	208.23	38.47	57.78	28.26	50.28	39.94	20.22	15.4	20.77	33.89
SimDiff (MASD) 2026.04	399.39	40.24	58.16	27.5	49.57	37.25	19.8	17	20.13	33.71
ShortGPT 2026.04	634.94	46.61	59.25	29.34	50.28	36.24	23.29	18.2	20.8	35.5
ShortGPT 2026.04	858.52	42.97	58.27	28.28	49.8	36.95	21.25	16	21.94	34.43
ShortGPT 2026.04	10,916.14	37.8	55.55	26.25	49.17	29.88	21.67	17.6	19.56	32.19