Selective Time Series Forecasting via Metalearning
About
Deep learning methods have achieved state-of-the-art in time series forecasting, yet their accuracy varies considerably across samples, as some instances remain inherently difficult to predict. Reject option mechanisms, which allow models to abstain from high-risk predictions, are well established in classification and regression but underexplored in forecasting. Existing abstention strategies typically rely on proxies, such as the width of the prediction interval or learned confidence scores derived from forecasts. However, these approaches are inherently tied to the training domain, limiting their ability to generalize. We propose a selective forecasting framework that addresses this limitation by modeling the empirical percentile of forecasting errors, that is, a scale-invariant statistic, based on structural characteristics extracted from recent lags via metalearning. By decoupling the rejection decision from the forecast itself and grounding it in domain-agnostic features, the framework enables effective abstention transfer across heterogeneous time series. Experiments in both in-domain and transfer learning settings show that rejecting samples predicted as challenging consistently improves forecasting accuracy across coverage levels.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Forecasting Rejection | M3 Monthly (Source holdout) | Spearman's ρ0.899 | 16 | |
| Forecasting Rejection | M1 Monthly Zero-shot transfer | Spearman Correlation (ρ)0.628 | 8 | |
| Forecasting Rejection | M1 Monthly Domain adaptation | Spearman Correlation (ρ)0.82 | 8 | |
| Forecasting Rejection | M3 Quarterly (Source holdout) | Spearman ρ0.75 | 8 | |
| Forecasting Rejection | M1 Quarterly Domain adaptation | Spearman Correlation (ρ)0.755 | 8 | |
| Forecasting Rejection | Tourism Monthly Zero-shot transfer | Spearman's ρ0.559 | 8 | |
| Forecasting Rejection | Tourism Monthly Domain adaptation | Spearman (ρ)0.741 | 8 | |
| Forecasting Rejection | Tourism Quarterly Zero-shot transfer | Spearman's ρ0.544 | 8 | |
| Forecasting Rejection | Tourism Quarterly Domain adaptation | Spearman ρ0.764 | 8 | |
| Forecasting Rejection | M1 Quarterly Zero-shot transfer | Spearman Correlation (ρ)0.529 | 8 |