Share your thoughts, 1 month free Claude Pro on usSee more

Long-context Language Modeling on RULER (Detailed Metrics)

100S1 Score

MoE Transformer

Updated 23d ago

Evaluation Results

Method	Links
MoE Transformer 2026.06		100	100	73.8	72	27.6	6	36.5	40.8	13.3	26.8	23.9	47.3	-	-
MoE Transformer 2026.06		100	99.8	81.8	74.2	45.4	38.4	37.4	40.5	15.1	16.3	8.1	50.6	-	-
Full 2026.07		100	100	100	100	98	96	100	98	-	88.67	99.2	91.56	56	62.83
Quest 2026.07		100	100	100	98	98	88	100	98.5	-	90.67	96.4	90.42	54	61.5
MosaicKV 2026.07		100	100	100	98	98	92	97.5	99.5	-	86	96.4	89.85	52	58.83
Full 2026.07		100	100	100	100	100	100	100	100	-	92	99.2	92.95	70	54.17
Quest 2026.07		100	100	100	100	100	98	98.5	99	-	88	99.2	92.13	68	54.83
MosaicKV 2026.07		100	100	98	100	98	100	99	98	-	89.3	91.2	90.47	58	54.17
Full 2026.07		100	100	100	98	98	98	96	98.5	-	84	100	90.05	56	52.11
Quest 2026.07		100	100	100	98	98	98	96.5	100	-	76.67	100	89.33	54	50.83
MosaicKV 2026.07		100	100	100	90	98	94	90	96.5	-	76.67	100	86.94	50	48.11
MoE Transformer 2026.06		99.8	100	83.8	70	4.8	21.2	39.4	32	26.4	43.3	9.3	48.2	-	-
MoE Transformer 2026.06		99.8	100	93	66.2	12.8	11.8	48.2	50.4	31.1	26.4	18.4	50.7	-	-