Share your thoughts, 1 month free Claude Pro on usSee more

Multi-task Language Understanding on MMLU (Calibration Metrics)

0.087Error Rate (ER)

Seq. Post.

Updated 12d ago

Evaluation Results

Method	Links
Seq. Post. 2026.05		0.087	0.179	62.2	0.545	12.8	10.8	0.416	21.5	9.5
Is True 2026.05		0.087	0.086	52.3	0.653	8.7	10.2	0.653	17.4	8.8
Verbalized 2026.05		0.095	0.037	66.9	0.455	8.5	5.8	0.461	18	9.6
Verbalized 2026.05		0.126	0.094	64.6	1.054	11.4	7.3	1.059	24	12.7
Seq. Post. 2026.05		0.145	0.131	72.6	1.003	13.5	8.5	0.979	28	14.8
Is True 2026.05		0.145	0.135	54.7	1.346	14.3	13.8	1.347	28.8	14.6
Seq. Post. 2026.05		0.181	0.083	81.6	0.422	12.7	5.9	0.455	30.8	18.2
Is True 2026.05		0.181	0.152	52.7	1.07	17.2	19.1	1.074	35.3	18.2
Verbalized 2026.05		0.187	0.104	62.4	0.599	15.6	12.3	0.616	34.3	18.8
Seq. Post. 2026.05		0.234	0.046	84	0.409	13.1	7.2	0.474	36.5	23.6
Is True 2026.05		0.234	0.175	52.6	0.793	21.1	21.5	0.804	44.5	23.6
Verbalized 2026.05		0.234	0.185	61.1	1.724	20.7	18.5	1.711	44.1	23.8
Seq. Post. 2026.05		0.264	0.151	80.9	0.591	18.2	10.3	0.64	44.6	26.6
Is True 2026.05		0.264	0.193	50.2	0.671	23.9	54.4	0.669	50.3	26.6
Seq. Post. 2026.05		0.265	0.03	82.4	0.442	14.6	9.3	0.516	41.1	26.7
Is True 2026.05		0.265	0.092	49.4	0.623	20.7	27.2	0.654	47.2	26.7
Verbalized 2026.05		0.273	0.217	60.4	2.049	23.9	22.2	2.053	51.2	27.6
Verbalized 2026.05		0.28	0.202	57	1.193	24	23	1.157	52	28.6
Verbalized 2026.05		0.327	0.234	56.5	1.148	27.2	29.9	1.049	59.9	33.8
Seq. Post. 2026.05		0.372	0.316	54.1	0.889	33.3	45.8	0.821	70.5	37.5
Is True 2026.05		0.372	0.141	50.5	0.771	26.4	37.8	0.832	63.6	37.5