Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Towards a General Time Series Forecasting Model with Unified Representation and Adaptive Transfer

About

With the growing availability of multi-domain time series data, there is an increasing demand for general forecasting models pre-trained on multi-source datasets to support diverse downstream prediction scenarios. Existing time series foundation models primarily focus on scaling up pre-training datasets and model sizes to enhance generalization performance. In this paper, we take a different approach by addressing two critical aspects of general forecasting models: (1) how to derive unified representations from heterogeneous multi-domain time series data, and (2) how to effectively capture domain-specific features to enable adaptive transfer across various downstream scenarios. To address the first aspect, we propose Decomposed Frequency Learning as the pre-training task, which leverages frequency-based masking and reconstruction to decompose coupled semantic information in time series, resulting in unified representations across domains. For the second aspect, we introduce the Time Series Register, which captures domain-specific representations during pre-training and enhances adaptive transferability to downstream tasks. Our model achieves the state-of-the-art forecasting performance on seven real-world benchmarks, demonstrating remarkable few-shot and zero-shot capabilities.

Yihang Wang, Yuying Qiu, Peng Chen, Kai Zhao, Yang Shu, Zhongwen Rao, Lujia Pan, Bin Yang, Chenjuan Guo• 2024

Related benchmarks

TaskDatasetResultRank
Time Series ForecastingETTh1
MSE0.391
601
Time Series ForecastingETTh2
MSE0.331
438
Time Series ForecastingETTm2
MSE0.246
382
Time Series ForecastingETTm1
MSE0.341
334
Time Series ForecastingWeather
MSE0.217
223
Time Series ForecastingElectricity
MSE0.155
161
Time Series ForecastingTraffic
MSE0.39
145
Short-term forecastingM4 Quarterly
MASE1.165
67
Short-term forecastingM4 Yearly
MASE3.014
63
Short-term forecastingM4 Monthly
SMAPE12.65
61
Showing 10 of 43 rows

Other info

Code

Follow for update