Monotonic Kolmogorov-Arnold Networks: A Theoretical and Empirical Study of Monotonicity as an Inductive Bias
About
Monotonicity has been a long-running architectural inductive bias for neural networks, motivated by tabular, scientific, and economic settings where outputs are known to respond monotonically to certain inputs. Existing approaches are MLP- or flow-based and lack per-edge functional transparency; the only Kolmogorov--Arnold Network (KAN) variant with monotonicity, MonoKAN, enforces the constraint only on a restricted parameter subset and requires a projection-style training procedure. We close this gap with \textbf{MKAN}, a KAN with hard monotonicity guaranteed for \emph{all} parameter values via exponential reparameterization of B-spline coefficients, positive edge weights, and a monotone base activation. Training reduces to standard unconstrained gradient descent. Our headline theoretical contribution is a \emph{representation-cost} theorem: any $C^K, K >0$ feature extractor inducing a ball-shaped semantic-neighborhood partition admits a monotone realization of the equivalent neighborhood structure at $N' = N^* + k \le 2N^*$, where $k$ is the number of non-monotone coordinates of the original. The bound is architecture-agnostic and gives a principled sizing rule for monotone encoders. Empirically, MKAN is competitive with state-of-the-art monotone NNs on the SMM/ICML-2024 benchmark while being the only method that combines hard unconstrained monotonicity with KAN's per-edge functional transparency; the $2N^*$ prediction is validated in a self-supervised feature-size sweep on four real datasets, and on a controlled monotone-generative dataset MKAN recovers ground-truth factors with substantially higher Spearman alignment than KAN, MLP, and linear baselines.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Classification | COMPAS (test) | Accuracy69.6 | 16 | |
| Classification | LoanDefaulter (test) | Test Accuracy65.49 | 10 | |
| Regression | BlogFeedback SMM ICML-2024 regression benchmark suite (test) | RMSE0.154 | 10 | |
| Classification | Heart Disease (test) | Test Accuracy89.6 | 7 | |
| Regression | Auto MPG SMM ICML-2024 regression benchmark suite (test) | MSE6.7 | 6 | |
| Disentangled Representation Learning | synthetic ellipses dataset | Reconstruction L2 Norm12.1 | 5 | |
| Monotone multivariate function approximation | SMM Monotone multivariate function approximation (Nin=4) (test) | Median MSE0.01 | 4 | |
| Classification | ChestXray (test) | Accuracy (Pretrained)67.7 | 4 | |
| Monotone multivariate function approximation | SMM Monotone multivariate function approximation (Nin=2) ICML-2024 (test) | Median MSE0.01 | 4 | |
| Monotone multivariate function approximation | SMM (Nin=6) ICML-2024 (test) | Median MSE0.05 | 4 |