Common-sense Reasoning on lm-eval-harness 11 tasks

63.430-shot Accuracy

MoE Transformer w/ dynamic conv. (head-wise)

Updated 1mo ago

Evaluation Results

Method	Links
MoE Transformer w/ dynamic conv. (head-wise) 2026.06		63.43
MoE Transformer w/ dynamic conv. (low-rank) 2026.06		63.42
MoE Transformer w/ static conv. 2026.06		62.97
MoE Transformer 2026.06		62.46
Transformer w/ dynamic conv. (low-rank) 2026.06		60.7
Transformer w/ dynamic conv. (low-rank) 2026.06		59.7
Transformer w/ static conv. 2026.06		58.94
Transformer w/ dynamic conv. (head-wise) 2026.06		58.46
Transformer 2026.06		58.35
Transformer w/ more params (wider FFN) 2026.06		58.23
Gated DeltaNet w/ dynamic conv. (low-rank) 2026.06		49.22
Transformer w/ dynamic conv. (low-rank) 2026.06		48.9
Transformer w/ dynamic conv. (low-rank) 2026.06		48.81
Transformer w/ dynamic conv. (head-wise) 2026.06		47.61
Mamba-2 w/ dynamic conv. (low-rank) 2026.06		47.34
Gated DeltaNet w/ dynamic conv. (head-wise) 2026.06		47.27
Transformer 2026.06		47.26
Mamba-2 w/ dynamic conv. (head-wise) 2026.06		47.24
Gated DeltaNet (w/o conv.) 2026.06		46.93
Transformer w/ static conv. 2026.06		46.86
Gated DeltaNet w/ static conv. 2026.06		46.76
Transformer w/ more params (wider FFN) 2026.06		46.64
Mamba-2 (w/o conv.) 2026.06		46.41
Mamba-2 w/ static conv. 2026.06		45.78