Dendritic In-Context Learning in a Single-Layer Spiking Neural Network
About
In-context learning (ICL) operates via implicit gradient descent embedded in the forward pass of modern AI architectures -- Transformers, Mamba, state-space models, and MLPs. Capturing this capability in biologically plausible Spiking Neural Networks (SNNs) has remained an open challenge: existing SNNs fail the Garg-2022 benchmark at non-trivial task dimensions. We trace this failure to a structural assumption: prior SNN designs route adaptation through inference-time synaptic plasticity, viewing the dendritic compartment as a passive conduit for error or teacher signals. We challenge this assumption. The subthreshold dynamics of a single dendritic compartment already implement a complete online learning algorithm. By treating the compartment as the computational substrate rather than a passive conduit, we propose DendriCL -- a single-layer compartmental spiking architecture whose apical recurrence is structurally identical to leaky online Widrow-Hoff LMS. This dynamics-only update collapses the architectural depth required for general-purpose ICL to a single layer. DendriCL is uniquely seed-stable at super-dimensional Garg-2022 ICL -- where dense Transformers exhibit grokking-style instability and fail past moderate task dimension -- and a linear probe recovers the reference online-LMS trajectory directly from the apical membrane at R^2 = 0.93, showing the algorithm is structurally embedded in the dynamics rather than implicitly discovered during training. Taken together, ICL requires neither attention, depth, nor inference-time plasticity: a single compartment with online-LMS dynamics is sufficient.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Linear Regression In-context Learning | Garg Linear Regression (d=20) | R^283.3 | 18 | |
| Linear Regression In-context Learning | Garg Linear Regression d=10 | R-squared0.824 | 15 | |
| Linear Regression In-context Learning | Garg Linear Regression d=30 | R^20.807 | 13 | |
| Linear Regression In-context Learning | Garg Linear Regression d=25 | R^20.809 | 13 | |
| Linear Regression In-context Learning | Garg Linear Regression (d=40) | R^278.7 | 12 | |
| In-Context Learning | Garg ICL d=20 2022 (held-out tasks) | R20.833 | 10 | |
| In-Context Learning (Linear Regression) | Garg (d=40) 2022 | R^20.787 | 9 | |
| In-Context Learning | Garg ICL (d=10) (held-out tasks) | R20.824 | 9 | |
| In-Context Learning (Linear Regression) | Garg d=30 2022 | R^20.807 | 8 | |
| In-Context Learning (Linear Regression) | Garg d=50 2022 | R^20.649 | 6 |