Share your thoughts, 1 month free Claude Pro on usSee more

OpenBook Question Answering on OBQA

44.2Accuracy

UNPRUNED

Updated 20d ago

Evaluation Results

Method	Links
UNPRUNED 2024.11		44.2	-
CD-MoE-S 2024.11		40.3	-
OPENLLAMA-3B 2024.11		38.6	-
CD-MoE-S 2024.11		38.6	-
CD-MoE-SR 2024.11		38.6	-
CD-MoE-SR 2024.11		37.6	-
M-SMoE 2024.11		36.4	-
OPT-2.7B 2024.11		35.2	-
GPT-NEO-2.7B 2024.11		34.6	-
BlockDrop 2024.11		34.6	-
LayerDrop 2024.11		34.2	-
LayerDrop 2024.11		32.6	-
BLOOM-3B 2024.11		32.4	-
EMO (Stage 5) 2026.05		29.6	-
FIXED_E=128 2026.05		28.8	-
FIXED_E=32 2026.05		27.8	-
FIXED_E=16 2026.05		27.4	-
EMO (Stage 4) 2026.05		26.6	-
EMO (Stage 2) 2026.05		24.8	-
EMO (Stage 1) 2026.05		24.2	-
EMO (Stage 3) 2026.05		23	-
Qwen2.5-Omni-7B 2025.10		0.855	3.5
ASR + Qwen2.5-7B 2025.10		0.84	5
ASR + Qwen2.5-3B 2025.10		0.769	4.9
SALAD-3B 2025.10		0.767	5.1
SALAD-7B 2025.10		0.751	13.9
SALAD-7B 2025.10		0.697	19.3
SALAD-3B 2025.10		0.655	16.3
GLM-4-Voice-9B 2025.10		0.576	30.1
DiVA-Llama3.1-8B 2025.10		0.409	42.4
Qwen2-Audio-7B 2025.10		0.396	37.1
Random 2025.10		0.25	-