Share your thoughts, 1 month free Claude Pro on usSee more

Tokenization Disambiguation on IMDB English

6.58Average Perplexity (PPL)

Verification-based

Updated 2mo ago

Evaluation Results

Method	Links
Verification-based 2026.04		6.58	0	0	2.32	86	15.17	209.59	100
MWIS 2026.04		6.87	0.09	0.75	2.12	81	7.64	55.92	100
Discop 2026.04		7.26	0	0	2.28	88	4.9	0	-
ReTokSync 2026.04		7.26	0	0	2.28	88	4.94	0.82	100
Stepwise Verification 2026.04		7.36	0	0.05	2.31	89	6.65	35.71	100
Syncpool 2026.04		7.65	0	0	1.68	64	5.05	3.06	100
MWIS 2026.04		9.34	0.11	0.75	2.61	82	9.01	80.92	100
Verification-based 2026.04		11	0	0	2.82	86	441.21	8,759.64	100
Discop 2026.04		11.33	0	0	2.95	91	4.98	0	-
ReTokSync 2026.04		11.36	0	0	2.95	91	5.05	1.41	99.9
MWIS 2026.04		11.74	0.15	0.83	2.93	80	23.89	361.2	100
Stepwise Verification 2026.04		11.93	0	0.15	3.06	91	11.82	137.35	100
Syncpool 2026.04		11.98	0	0	1.01	33	5.15	3.41	100
Verification-based 2026.04		13.54	0	0	3.13	87	552.64	10,714.87	100
Stepwise Verification 2026.04		16.37	0	0.01	3.45	91	32.39	525.29	100
Syncpool 2026.04		17.06	0	0	0.53	14	5.21	0.58	100
Discop 2026.04		17.29	0	0	3.5	91	5.18	0	-
ReTokSync 2026.04		17.29	0	0	3.5	91	5.27	1.74	100
Basic 2026.04		19.93	0.96	9.15	2.72	96	7.64	55.92	100
Basic 2026.04		59.05	1.24	8.77	4.05	98	8.75	75.7	100
Basic 2026.04		219.67	1.36	7.58	5.72	98	21.69	318.73	100