Share your thoughts, 1 month free Claude Pro on usSee more

Multitask Language Understanding on MMLU (pass@1)

71.9pass@1

Tulu 3-SFT

Updated 1mo ago

Evaluation Results

Method	Links
Tulu 3-SFT 2024.10		71.9
MATRIX-Gen-SFT 2024.10		71.9
OpenHermes 2024.10		69.6
WildChat 2024.10		69.3
Tulu v2 Mix 2024.10		69
Evol-Instruct 2024.10		68.3
ShareGPT 2024.10		67.8
Magpie-SFT 2024.10		67
Magpie-SFT 2024.10		61.2
MATRIX-Gen-SFT 2024.10		59.9
ShareGPT 2024.10		58.4
Tulu v2 Mix 2024.10		58.4
Tulu 3-SFT 2024.10		57.8
Dare Ties Merging 2026.02		55.64
Task Arithmetic 2026.02		55.58
w/o Merging 2026.02		55.11
OpenHermes 2024.10		52.8
Ties Merging 2026.02		52.04
Evol-Instruct 2024.10		48
SCF-RKL 2026.02		46.17
Dare Task Arithmetic 2026.02		45.54
WildChat 2024.10		43.1
SCE 2026.02		42.85
w/o Merging 2026.02		38.68