Shapley-Inspired Feature Weighting in $k$-means with No Additional Hyperparameters
About
Clustering algorithms often assume all features contribute equally to the data structure, an assumption that usually fails in high-dimensional or noisy settings. Feature weighting methods can address this, but most require additional parameter tuning. We propose SHARK (Shapley Reweighted $k$-means), a feature-weighted clustering algorithm motivated by the use of Shapley values from cooperative game theory to quantify feature relevance, which requires no additional parameters beyond those in $k$-means. We prove that the $k$-means objective can be decomposed into a sum of per-feature Shapley values, providing an axiomatic foundation for unsupervised feature relevance and reducing Shapley computation from exponential to polynomial time. SHARK iteratively re-weights features by the inverse of their Shapley contribution, emphasising informative dimensions and down-weighting irrelevant ones, and is equivalent to replacing the arithmetic mean of feature dispersions with their harmonic mean. Experiments on synthetic and real-world data sets show that SHARK consistently matches or outperforms existing methods, achieving superior robustness and accuracy, particularly in scenarios where noise may be present. Software: https://github.com/rickfawley/SHARK.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Clustering | Wine | ARI0.822 | 53 | |
| Clustering | Glass | -- | 51 | |
| Clustering | Yeast | ARI16 | 50 | |
| Clustering | Synthetic Data Summary | Mean Relative Rank1 | 10 | |
| Clustering | Synthetic Data 1000x10-3k+5NF | ARI0.88 | 5 | |
| Clustering | Synthetic Data (1000x10-10k+5NF) | ARI0.68 | 5 | |
| Clustering | Synthetic Data 2000x20-10k+10NF | ARI0.958 | 5 | |
| Clustering | Synthetic Data 2000x20-20k+10NF | ARI91.7 | 5 | |
| Clustering | Synthetic Data 2000x30-5k+15NF | ARI99.8 | 5 | |
| Clustering | Synthetic Data 2000x30-10k+15NF | ARI99.5 | 5 |