Sparse Autoencoding

Benchmarks

Dataset Name	SOTA Method	Metric
Pythia-70M layer 3	Dense-SAE	Relative Error0.342	13	22d ago
Pythia-70M residual-stream activations 5,000-token (test)	Expander-SAE	Relative Error0.341	12	22d ago
Pythia-160M residual-stream activations 5,000-token (test)	Dense-SAE	Relative Error0.332	8	22d ago
gemma2-2b-layer-13 activations	HSAE	L0100.7	6	4mo ago
Gemma-2-2B activations (layer 20)	Matryoshka	KL Score0.9727	5	25d ago
Llama-3.2-1B Layer 12 activations (held-out)	Dense-SAE	Relative Error0.499	4	22d ago
Llama-3.2-1B Layer 6 activations (held-out)	Dense-SAE	Relative Error56.8	4	22d ago
Qwen2.5-3B Layer 24 activations (held-out)	Dense-SAE	Relative Error0.591	4	22d ago
Qwen2.5-3B Layer 12 activations (held-out)	Dense-SAE	Relative Error48.9	4	22d ago
Pythia-160M Layer 8 activations (held-out)	Dense-SAE	Relative Reconstruction Error0.332	4	22d ago
Pythia-70M Layer 3 activations (held-out)	Dense-SAE	Relative Error0.342	4	22d ago
Gemma-2-2B (layer 12)	C²R	KL Score0.9628	4	25d ago
DINOv2 ImageNet-1k Base (patch tokens)	iSAE-ME	MSE0.191	4	1mo ago
Pythia activations (The Pile) 160M (layer 12)	iSAE-ME	MSE0.148	4	1mo ago
Pythia 70M (layer-3 residuals)	AEN-SAE	Achieved L033.9	4	2mo ago
Mistral-7B activations	SASA	KL Score95.3	2	1mo ago
GPT-2 activations	Standard SAE	KL Score98	2	1mo ago
qwen3-4b layer-18 activations	Baseline	L0 Norm50	2	4mo ago
gemma2-2b layer-20 activations	HSAE	L0 Norm50	2	4mo ago
gemma2-2B-layer-6 activations	Baseline	L0 Norm50	2	4mo ago
SlimPajama, Proprietary, and FineWeb-Edu	-	-	0	29d ago

Showing 21 of 21 rows