Multiple Choice Question Answering on ARC-Easy (test)

71.2Accuracy

SAES-SVD

Updated 3d ago

Evaluation Results

Method	Links
SAES-SVD 2026.02		71.2
Pythia-12B 2023.04		70.2
SVD-LLM 2026.02		68.3
Dip-SVD 2026.02		68.1
ClassAct 2024.11		67.34
Pythia-6.9B 2023.04		67.3
BADS 2024.11		67
AskLLM-O 2024.11		66.88
Random_Select 2024.11		66.75
Duplicate_Meta 2024.11		65.15
Mixing 2024.11		64.65
Pythia-2.8B 2023.04		64.4
Pythia-1.4B 2023.04		60.6
CDS 2024.11		60.14
Mistral (Full-Attention) 2024.07		58.2
Pythia-1B 2023.04		56.9
Dip-SVD 2026.02		54.8
BMoJo (Fading + Eidetic) 2024.07		54.7
SAES-SVD 2026.02		54.3
Mamba (SSM) 2024.07		52.4
BMoJo (Fading) 2024.07		52.3
Meta_Only 2024.11		52.23
Pythia-410M 2023.04		52.1
SVD-LLM 2026.02		52.1
Hybrid (Sliding Attention + SSM) 2024.07		48.4
Mamba (SSM) 2024.07		45
AR Transformer 2026.02		44.95
Mistral (Full-Attention) 2024.07		44.9
BMoJo (Fading + Eidetic) 2024.07		44.3
Pythia-160M 2023.04		43.5
BMoJo (Fading) 2024.07		43.1
Hybrid (Sliding Attention + SSM) 2024.07		42.7
Pythia-70M 2023.04		37.4
MDLM 2026.02		34.26
Duo++ 2026.02		28.28
Duo 2026.02		28.11
Duo++ 2026.02		28.03
Duo++ 2026.02		27.32
GenMC_T5 2022.04		0.6901
UnifiedQA_T5-FT 2022.04		0.6692
UnifiedQA_T5 2022.04		0.6643
UnifiedQA_T5* 2022.04		0.659
RoBERTa 2022.04		0.624
GenMC_T5 2022.04		0.5882
UnifiedQA_T5-FT 2022.04		0.5388
ALBERT 2022.04		0.5377
UnifiedQA_T5 2022.04		0.5366
RoBERTa 2022.04		0.5232
ALBERT 2022.04		0.4584
UnifiedQA_T5* 2022.04		0.424