Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Dendritic In-Context Learning in a Single-Layer Spiking Neural Network

About

In-context learning (ICL) operates via implicit gradient descent embedded in the forward pass of modern AI architectures -- Transformers, Mamba, state-space models, and MLPs. Capturing this capability in biologically plausible Spiking Neural Networks (SNNs) has remained an open challenge: existing SNNs fail the Garg-2022 benchmark at non-trivial task dimensions. We trace this failure to a structural assumption: prior SNN designs route adaptation through inference-time synaptic plasticity, viewing the dendritic compartment as a passive conduit for error or teacher signals. We challenge this assumption. The subthreshold dynamics of a single dendritic compartment already implement a complete online learning algorithm. By treating the compartment as the computational substrate rather than a passive conduit, we propose DendriCL -- a single-layer compartmental spiking architecture whose apical recurrence is structurally identical to leaky online Widrow-Hoff LMS. This dynamics-only update collapses the architectural depth required for general-purpose ICL to a single layer. DendriCL is uniquely seed-stable at super-dimensional Garg-2022 ICL -- where dense Transformers exhibit grokking-style instability and fail past moderate task dimension -- and a linear probe recovers the reference online-LMS trajectory directly from the apical membrane at R^2 = 0.93, showing the algorithm is structurally embedded in the dynamics rather than implicitly discovered during training. Taken together, ICL requires neither attention, depth, nor inference-time plasticity: a single compartment with online-LMS dynamics is sufficient.

Juwei Shen, Yujie Wu, Changwen Chen• 2026

Related benchmarks

TaskDatasetResultRank
Linear Regression In-context LearningGarg Linear Regression (d=20)
R^283.3
18
Linear Regression In-context LearningGarg Linear Regression d=10
R-squared0.824
15
Linear Regression In-context LearningGarg Linear Regression d=30
R^20.807
13
Linear Regression In-context LearningGarg Linear Regression d=25
R^20.809
13
Linear Regression In-context LearningGarg Linear Regression (d=40)
R^278.7
12
In-Context LearningGarg ICL d=20 2022 (held-out tasks)
R20.833
10
In-Context Learning (Linear Regression)Garg (d=40) 2022
R^20.787
9
In-Context LearningGarg ICL (d=10) (held-out tasks)
R20.824
9
In-Context Learning (Linear Regression)Garg d=30 2022
R^20.807
8
In-Context Learning (Linear Regression)Garg d=50 2022
R^20.649
6
Showing 10 of 12 rows

Other info

Follow for update