Translation

Benchmarks

Dataset Name	SOTA Method	Metric
FLORES-200 it-en (devtest)	GemmaX2-9B	sacreBLEU37.0319	35	3mo ago
FLORES-200 en-it (devtest)	GemmaX2-9B-b5	sacreBLEU33.831	35	3mo ago
lexical choice (val+test)	gpt-4o	BLEU54.15	24	2mo ago
Konkani-Bench 200 samples (test)		BLEU86.76	22	2mo ago
Translation	IGDS	COMET Score83.07	21	1mo ago
Spec-Bench Trans.	SpecBound	CR6.41	21	1mo ago
FLoRes+ All total	OMT-NLLB	ChrF++50.9	16	2mo ago
FLoRes+ En-YY (total)	OMT-NLLB	ChrF++44.6	16	2mo ago
FLoRes+ En-YY, v. low resource level	OMT-NLLB	ChrF++34.6	16	2mo ago
FLoRes+ En-YY, low resource level	OMT-NLLB	ChrF++41	16	2mo ago
FLoRes+ En-YY mid resource level	OMT-LLaMA 8B	ChrF++46.5	16	2mo ago
FLoRes+ En-YY, high resource level		ChrF++58.6	16	2mo ago
FLoRes+ XX-En total	OMT-NLLB	ChrF++57.3	16	2mo ago
FLoRes+ XX-En v. low resource level	OMT-NLLB	ChrF++52.3	16	2mo ago
FLoRes+ XX-En, low resource level	OMT-NLLB	ChrF++54.6	16	2mo ago
FLoRes+ XX-En mid resource level	OMT-NLLB	ChrF++58.3	16	2mo ago
FLoRes+ XX-En, high resource level	OMT-NLLB	ChrF++65	16	2mo ago
English-Spanish	MTV	Accuracy76.7	15	3mo ago
Sand-Glass Zh-to-En (human evaluation)	DeepSeek-V3	Accuracy4.97	14	3mo ago
Copyrighted Content without copyright notice 1.0 (test)	Gemini-2.0	Cosine Similarity44.84	12	3mo ago
IWSLT en-ja 17	LongGuide	ROUGE-L41.22	12	3mo ago
Custom English-to-Hebrew	DictaLM-3.0-24B-Thinking	Win Rate (GPT-4o Judge)30.09	11	3mo ago
WMT En-Ro '16		BLEU39.9	9	3mo ago
WMT de-fr 2019 (test)	Finetuned SOTA	BLEU31.5	7	3mo ago
Konkani-Instruct 100k	Konkani-Qwen2.5-14B-Instruct	Romi Score4.5	6	2mo ago

Showing 25 of 43 rows