Cross-RAG: Zero-Shot Retrieval-Augmented Time Series Forecasting via Cross-Attention
About
Recent advances in time series foundation models (TSFMs) demonstrate strong expressive capacity through large-scale pretraining across diverse time series domains. Zero-shot time series forecasting with TSFMs, however, exhibits limited generalization to unseen datasets, which retrieval-augmented forecasting addresses by leveraging an external knowledge base. Existing approaches rely on a fixed number of retrieved samples that may introduce irrelevant information. To this end, we propose Cross-RAG, a zero-shot retrieval-augmented forecasting framework that selectively attends to query-relevant retrieved samples. Cross-RAG models input-level relevance between the query and retrieved samples via query-retrieval cross-attention, while jointly incorporating information from the query and retrieved samples. Extensive experiments demonstrate that Cross-RAG consistently improves zero-shot forecasting performance across various TSFMs and RAG methods, and additional analyses confirm its effectiveness across diverse retrieval scenarios. Code is available at https://github.com/seunghan96/cross-rag/.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Time Series Forecasting | ETTh1 (test) | MSE0.341 | 398 | |
| Time Series Forecasting | ETTm1 (test) | MSE0.29 | 315 | |
| Long-term forecasting | ETTh2 | MSE0.243 | 310 | |
| Time Series Forecasting | ETTh2 (test) | MSE0.243 | 250 | |
| Time Series Forecasting | Weather (test) | MSE0.144 | 248 | |
| Time Series Forecasting | ETTm2 (test) | MSE0.143 | 186 | |
| Time Series Forecasting | Electricity (test) | MSE0.112 | 130 | |
| Univariate Time Series Forecasting | Exchange (test) | MSE0.064 | 46 | |
| Forecasting | Weather | MAE0.181 | 41 | |
| Forecasting | ETTh1 | MSE0.347 | 22 |