Machine Translation Evaluation

Benchmarks

Dataset Name	SOTA Method	Metric
WMT Metrics Shared Task 2024	MBR Distill	SPA86.4	65	4mo ago
WMT 2019 (test)	DIFFSCORE-FT	de-en0.327	25	2mo ago
WMT MQM Segment-level 22	MetricsX-XXL	Score (En-De)60.1	19	4mo ago
WMT MQM System-level 22	EAPrompt	Overall Score91.2	19	4mo ago
WMT segment-level 2019 (test)	BERTScore	Pearson R44.5	19	4mo ago
TAC summary-level 2008-2011 (test)	FrugalScore	Pearson Correlation (Pyramid)67.3	19	4mo ago
WMT MQM 2022 (test)	Remedy-R	Accuracy (System, 3 LPs)91.6	16	4mo ago
WMT 2023 (test)	Distribution-Calibrated Aggregation	MAE (EN→DE)0.588	12	4mo ago
MSLC OOD 24	XCOMET	MT Empty Score73.79	12	4mo ago
WMT17 (test)	ParaBLEU	Kendall Tau0.653	12	4mo ago
Met-BOUQuET XSTS+R+P r1 (test)	BLASER 3	Score (XX-En)65	7	4mo ago
WMT 24	Llama 4 Scout	Quality (cs-uk)0.945	6	4mo ago
WMT 21	Llama 4 Scout	Score (en-ha)54.3	6	4mo ago
WMT Domain 21		Correlation0.65	5	4mo ago
WMT De→En Top 30% 2019		Pearson Correlation (\|r\|)0.883	5	4mo ago
WMT De→En 2019 (All)	DA-BERTScore	Pearson Correlation (\|r\|)0.951	5	4mo ago
WMT En→De 2019 (Top 30%)	DA-BERTScore	Pearson Correlation (\|r\|)0.974	5	4mo ago
WMT En→De 2019 (all)	DA-BERTScore	\|r\|0.991	5	4mo ago
Aligned 300-word human-evaluation chunks	Agentic (P3) MT	HT Mean Score8.657	4	1mo ago

Showing 19 of 19 rows