The Procrustean Bed of Time Series: The Optimization Bias of Point-wise Loss

Optimizing time series models via point-wise loss functions (e.g., MSE) relying on a heuristic point-wise i.i.d. assumption disregards the causal temporal structure. Focusing on the core independence issue under covariance stationarity, this paper aims to provide a first-principles analysis of the Expectation of Optimization Bias (EOB). Our analysis reveals a fundamental paradigm paradox: The more deterministic and structured the time series, the more severe the bias incurred by point-wise loss function. We derive the first closed-form quantification for the non-deterministic EOB across linear and non-linear systems, and prove EOB is an intrinsic data property, governed exclusively by sequence length and the defined Structural Signal-to-Noise Ratio. This theoretical discovery motivates our principled debiasing program that eliminates the bias through sequence length reduction and structural orthogonalization. We present a concrete solution via DFT or DWT, and propose a novel harmonized $\ell_p$ norm framework to rectify gradient optimization pathologies of high-variance sequences. Extensive experiments validate EOB Theory's generality and the superior performance of debiasing program, achieving 5.2% and 5.1% average improvement of MSE and MAE conducted on the iTransformer across 11 datasets, respectively.