SAEBench
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Sparse Probing | SAEBench | Average F1 Score81.9 | 16 | |
| Reconstruction | SAEBench held-out data | MSE0.03 | 16 | |
| Sparse Autoencoder Evaluation | SAEBench intrinsic evaluation Gemma 2-2B | RAVEL Score0.7625 | 4 | |
| Sparse Autoencoder Evaluation | SAEBench Pythia-160M (test) | RAVEL Score0.502 | 4 | |
| Sparse Autoencoder Evaluation | SAEBench Pythia-70M | RAVEL Score0.308 | 4 | |
| Spurious Correlation Removal | SAEBench Spurious Correlation Removal Pythia-70M activations | Bias (Professor vs Nurse)2.1 | 3 | |
| Binary Classification | SAEBench Pythia-160m (full) | Bias in BIOS Set 1 Acc95.3 | 2 | |
| Sparse Autoencoder Evaluation | SAEBench | Top-1 Sparse Probing Accuracy67.3 | 2 |
Showing 8 of 8 rows