Share your thoughts, 1 month free Claude Pro on usSee more

Question Answering on OpenBookQA (Normalized Accuracy)

55.6Normalized Accuracy

Llama-Instruct

Updated 9d ago

Evaluation Results

Method	Links
Llama-Instruct 2026.03		55.6
HATified-SFT 2026.03		52.6
Base 2026.04		49
Base 2026.04		47.4
Base 2026.04		46.2
Base 2026.04		45.6
Llama 3-8B 2024.12		45
Llama 3-8B E8T2 2024.12		44.8
GlobalGC 2025.02		33.6
ClipByValue 2025.02		33.2
GlobalGC 2025.02		33
AGC 2025.02		32.8
AdaGC 2025.02		32.8
Mamba-2 2026.04		32.8
Mamba-2-1.5B 2026.03		32.6
Mamba-3-MIMO-1.5B 2026.03		32.6
Mamba-2 + PoST 2026.04		32.6
Clippy 2025.02		32.4
AdaGC 2025.02		32.2
Mamba-3-SISO-1.5B 2026.03		32
RWKV-7 + PoST 2026.04		32
Gated DeltaNet + PoST 2026.04		31.8
EFLA 2025.12		31.6
GDN-1.5B 2026.03		31.6
Base 2026.04		31.2
EFLA + Loose β 2025.12		30.8
GatedFWA-NSA 2025.12		30.8
Mamba-2 2026.04		30.6
Gated DeltaNet 2026.04		30.6
ClipByValue 2025.02		30.4
AdaGC 2025.02		30.4
GlobalGC 2025.02		30.2
Mamba-2-880M 2026.03		30.2
Transformer (LLaMA) + SWA + NSA 2025.12		30.18
HGRN2 2025.12		30
Clippy 2025.02		30
Mamba-3-SISO-880M 2026.03		30
Mamba-2 + PoST 2026.04		30
GatedFWA 2025.12		29.93
DeltaNet 2025.12		29.8
PLDRv51-SOC-110M-5 2026.03		29.8
Transformer-1.5B 2026.03		29.6
Transformer (LLaMA) 2025.12		29.4
Transformer (LLaMA) + SWA 2025.12		29.22
GLA 2025.12		29.2
PLDRv51-SOC-110M-4 2026.03		29.2
DeltaNet 2025.12		29
PolychromaticLM 2026.03		29
RWKV-7 2026.04		29
GDN-880M 2026.03		28.6
Mamba-3-MIMO-880M 2026.03		28.6
RetNet 2025.12		28.4
Mamba-3-MIMO-440M 2026.03		28.4
ABL-SOC-110M-3 2026.03		28.2
Mamba 2025.12		28
GDN-440M 2026.03		27.6
ABL-SOC-110M-1 2026.03		27.6
PLDRv51-SOC-110M-3 2026.03		27.2
DeltaNet 2025.12		27
EFLA + Adaptive Decay 2025.12		27
PLDRv51-SOC-110M-1 2026.03		27
PolychromaticLM 2026.03		26.8
Transformer-880M 2026.03		26.8
EFLA 2025.12		26.6
ABL-SOC-110M-2 2026.03		26.6
RFMoE 2026.04		26.6
RFMoE 2026.04		26.4
PLDRv51-SOC-110M-2 2026.03		26.2
GPT-Neo-125M 2026.03		26.2
Transformer-440M 2026.03		26
Mamba-2-440M 2026.03		26
Mamba-3-SISO-440M 2026.03		26
SUB-SOC-110M-1 2026.03		26
ReMoE 2026.04		26
SUB-SOC-110M-2 2026.03		25.2
Random 2026.03		25
AoE 2026.04		25
MoE 2026.04		25
MoE 2026.04		24.6
MoE 2026.04		24.6
RFMoE 2026.04		24.4
Mamba-2-180M 2026.03		23.2
Mamba-3-SISO-180M 2026.03		22.8
GDN-180M 2026.03		22
Mamba-3-MIMO-180M 2026.03		22
Transformer-180M 2026.03		21.8
Dual 2025.12		17.6
Dual 2025.12		14.4
Autoregressive 2025.12		13.6
Autoregressive 2025.12		9.9
Dual 2025.12		8.5
GAIN 2026.04		0.4
GAIN 2026.04		0
GAIN 2026.04		-0.2
Autoregressive 2025.12		-0.5
GAIN 2026.04		-1
GAIN 2026.04		-1.8
LoRA 2026.04		-2.2
LoRA 2026.04		-2.8
LoRA 2026.04		-3

Showing 100 of 102 rows