Machine Translation Evaluation

Benchmarks

Dataset Name	SOTA Method	Metric
WMT Metrics Shared Task 2024	MBR Distill	SPA86.4	65	2mo ago
WMT 2019 (test)	DIFFSCORE-FT	de-en0.327	25	21d ago
WMT MQM Segment-level 22	MetricsX-XXL	Score (En-De)60.1	19	3mo ago
WMT MQM System-level 22	EAPrompt	Overall Score91.2	19	3mo ago
WMT segment-level 2019 (test)	BERTScore	Pearson R44.5	19	3mo ago
TAC summary-level 2008-2011 (test)	FrugalScore	Pearson Correlation (Pyramid)67.3	19	3mo ago
WMT MQM 2022 (test)	Remedy-R	Accuracy (System, 3 LPs)91.6	16	3mo ago
WMT 2023 (test)	Distribution-Calibrated Aggregation	MAE (EN→DE)0.588	12	3mo ago
MSLC OOD 24	XCOMET	MT Empty Score73.79	12	3mo ago
WMT17 (test)	ParaBLEU	Kendall Tau0.653	12	3mo ago
Met-BOUQuET XSTS+R+P r1 (test)	BLASER 3	Score (XX-En)65	7	2mo ago
WMT 24	Llama 4 Scout	Quality (cs-uk)0.945	6	2mo ago
WMT 21	Llama 4 Scout	Score (en-ha)54.3	6	2mo ago
WMT Domain 21		Correlation0.65	5	3mo ago
WMT De→En Top 30% 2019		Pearson Correlation (\|r\|)0.883	5	3mo ago
WMT De→En 2019 (All)	DA-BERTScore	Pearson Correlation (\|r\|)0.951	5	3mo ago
WMT En→De 2019 (Top 30%)	DA-BERTScore	Pearson Correlation (\|r\|)0.974	5	3mo ago
WMT En→De 2019 (all)	DA-BERTScore	\|r\|0.991	5	3mo ago

Showing 18 of 18 rows