Discrete Prototypical Memories for Federated Time Series Foundation Models
About
Leveraging Large Language Models (LLMs) as federated learning (FL)-based time series foundation models offers a promising way to transfer the generalization capabilities of LLMs to time series data while preserving access to private data. However, the semantic misalignment between time-series data and the text-centric latent space of existing LLMs often leads to degraded performance. Meanwhile, the parameter-sharing mechanism in existing FL methods model heterogeneous cross-domain time-series data into a unified continuous latent space, which contradicts the fact that time-series semantics frequently manifest as discrete and recurring regimes. To address these limitations, we propose \textsc{FeDPM}, a federated framework for time-series foundation models based on discrete prototypical memories. Specifically, we learn local prototypical memory priors for intra-domain time-series data. We then align cross-domain memories to promote a unified discrete latent space and introduce a domain-specific memory update mechanism to balance shared and personalized prototypical knowledge. Extensive experiments demonstrate the efficiency and effectiveness of \textsc{FeDPM}. The code is publicly available at https://anonymous.4open.science/r/FedUnit-64D1.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Time Series Forecasting | ETTh2 | MSE0.304 | 796 | |
| Time Series Forecasting | Weather | MSE0.163 | 497 | |
| Long-term forecasting | ETTm1 | MSE0.538 | 422 | |
| Long-term forecasting | ETTm2 | MSE0.307 | 350 | |
| Time Series Forecasting | ETTm2 | MSE0.178 | 300 | |
| Time Series Forecasting | Electricity | MSE0.205 | 237 | |
| Time Series Forecasting | Exchange | MSE0.085 | 227 | |
| Long-term forecasting | Exchange | MSE0.155 | 73 | |
| Time Series Forecasting | ETTm1 few-shot 10% data | MSE0.508 | 54 | |
| Long-term forecasting | ETTm1 5% few-shot | MSE0.472 | 32 |