Translation

Benchmarks

Dataset Name	SOTA Method	Metric
FLORES-200 it-en (devtest)	GemmaX2-9B	sacreBLEU37.0319	35	1mo ago
FLORES-200 en-it (devtest)	GemmaX2-9B-b5	sacreBLEU33.831	35	1mo ago
lexical choice (val+test)	gpt-4o	BLEU54.15	24	1mo ago
Konkani-Bench 200 samples (test)		BLEU86.76	22	23d ago
Spec-Bench Trans.	SpecBound	CR6.41	21	3d ago
FLoRes+ All total	OMT-NLLB	ChrF++50.9	16	1mo ago
FLoRes+ En-YY (total)	OMT-NLLB	ChrF++44.6	16	1mo ago
FLoRes+ En-YY, v. low resource level	OMT-NLLB	ChrF++34.6	16	1mo ago
FLoRes+ En-YY, low resource level	OMT-NLLB	ChrF++41	16	1mo ago
FLoRes+ En-YY mid resource level	OMT-LLaMA 8B	ChrF++46.5	16	1mo ago
FLoRes+ En-YY, high resource level		ChrF++58.6	16	1mo ago
FLoRes+ XX-En total	OMT-NLLB	ChrF++57.3	16	1mo ago
FLoRes+ XX-En v. low resource level	OMT-NLLB	ChrF++52.3	16	1mo ago
FLoRes+ XX-En, low resource level	OMT-NLLB	ChrF++54.6	16	1mo ago
FLoRes+ XX-En mid resource level	OMT-NLLB	ChrF++58.3	16	1mo ago
FLoRes+ XX-En, high resource level	OMT-NLLB	ChrF++65	16	1mo ago
English-Spanish	MTV	Accuracy76.7	15	1mo ago
Sand-Glass Zh-to-En (human evaluation)	DeepSeek-V3	Accuracy4.97	14	1mo ago
Copyrighted Content without copyright notice 1.0 (test)	Gemini-2.0	Cosine Similarity44.84	12	1mo ago
IWSLT en-ja 17	LongGuide	ROUGE-L41.22	12	1mo ago
Custom English-to-Hebrew	DictaLM-3.0-24B-Thinking	Win Rate (GPT-4o Judge)30.09	11	1mo ago
WMT En-Ro '16		BLEU39.9	9	1mo ago
WMT de-fr 2019 (test)	Finetuned SOTA	BLEU31.5	7	1mo ago
Konkani-Instruct 100k	Konkani-Qwen2.5-14B-Instruct	Romi Score4.5	6	23d ago
Flores 66 languages	Tiny Aya Global	ChrF43.5	6	1mo ago

Showing 25 of 41 rows