Discrete Prototypical Memories for Federated Time Series Foundation Models
About
Leveraging Large Language Models (LLMs) as federated learning (FL)-based time series foundation models offers a promising way to transfer the generalization capabilities of LLMs to time series data while preserving access to private data. However, the semantic misalignment between time-series data and the text-centric latent space of existing LLMs often leads to degraded performance. Meanwhile, the parameter-sharing mechanism in existing FL methods model heterogeneous cross-domain time-series data into a unified continuous latent space, which contradicts the fact that time-series semantics frequently manifest as discrete and recurring regimes. To address these limitations, we propose \textsc{FeDPM}, a federated framework for time-series foundation models based on discrete prototypical memories. Specifically, we learn local prototypical memory priors for intra-domain time-series data. We then align cross-domain memories to promote a unified discrete latent space and introduce a domain-specific memory update mechanism to balance shared and personalized prototypical knowledge. Extensive experiments demonstrate the efficiency and effectiveness of \textsc{FeDPM}. The code is publicly available at https://anonymous.4open.science/r/FedUnit-64D1.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Time Series Forecasting | ETTh2 | MSE0.304 | 561 | |
| Long-term forecasting | ETTm1 | MSE0.538 | 375 | |
| Long-term forecasting | ETTm2 | MSE0.307 | 310 | |
| Time Series Forecasting | Weather | MSE0.163 | 295 | |
| Time Series Forecasting | Exchange | MSE0.085 | 199 | |
| Time Series Forecasting | Electricity | MSE0.205 | 114 | |
| Long-term forecasting | Exchange | MSE0.155 | 64 | |
| Time Series Forecasting | ETTm1 few-shot 10% data | MSE0.508 | 54 | |
| Time Series Forecasting | ETTm2 | MSE0.178 | 53 | |
| Long-term forecasting | ETTm1 5% few-shot | MSE0.472 | 32 |