Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Towards a General Time Series Forecasting Model with Unified Representation and Adaptive Transfer

About

With the growing availability of multi-domain time series data, there is an increasing demand for general forecasting models pre-trained on multi-source datasets to support diverse downstream prediction scenarios. Existing time series foundation models primarily focus on scaling up pre-training datasets and model sizes to enhance generalization performance. In this paper, we take a different approach by addressing two critical aspects of general forecasting models: (1) how to derive unified representations from heterogeneous multi-domain time series data, and (2) how to effectively capture domain-specific features to enable adaptive transfer across various downstream scenarios. To address the first aspect, we propose Decomposed Frequency Learning as the pre-training task, which leverages frequency-based masking and reconstruction to decompose coupled semantic information in time series, resulting in unified representations across domains. For the second aspect, we introduce the Time Series Register, which captures domain-specific representations during pre-training and enhances adaptive transferability to downstream tasks. Our model achieves the state-of-the-art forecasting performance on seven real-world benchmarks, demonstrating remarkable few-shot and zero-shot capabilities.

Yihang Wang, Yuying Qiu, Peng Chen, Kai Zhao, Yang Shu, Zhongwen Rao, Lujia Pan, Bin Yang, Chenjuan Guo• 2024

Related benchmarks

TaskDatasetResultRank
Time Series ForecastingETTh1
MSE0.391
836
Time Series ForecastingETTh2
MSE0.331
796
Time Series ForecastingETTm2
MSE0.246
536
Time Series ForecastingWeather
MSE0.217
497
Time Series ForecastingETTm1
MSE0.341
363
Time Series ForecastingETTm2
MSE0.299
300
Time Series ForecastingElectricity
MSE0.234
237
Time Series ForecastingTraffic
MSE0.39
211
Short-term forecastingM4 Yearly
MASE3.014
168
Short-term forecastingM4 Quarterly
MASE1.165
166
Showing 10 of 45 rows

Other info

Code

Follow for update