GIST: Gauge-Invariant Spectral Transformers for Scalable Graph Neural Operators

Adapting transformer positional encodings to graphs and meshes faces a fundamental tension: exact spectral methods require cubic-complexity eigendecomposition and inadvertently break gauge invariance through numerical solver artifacts, while existing efficient approximations sacrifice gauge symmetry by design. Both failure modes cause catastrophic generalization loss in inductive settings, where models fail when encountering different spectral decompositions of similar graphs or different discretizations of the same domain. We propose GIST (Gauge-Invariant Spectral Transformer), a graph transformer resolving this tension by restricting attention to pairwise inner products of efficient approximate spectral embeddings. We prove these inner products estimate an exactly gauge-invariant graph kernel at end-to-end $\mathcal{O}(N)$ complexity, and establish a formal connection between gauge invariance and discretization invariance: gauge invariance guarantees discretization-invariant learning with bounded mismatch error, making GIST the first scalable graph transformer with provable neural operator guarantees. Empirically, GIST matches state-of-the-art on standard graph benchmarks (e.g., achieving 99.50\% micro-F1 on PPI) while uniquely scaling to mesh-based neural operator benchmarks with up to 750K nodes, achieving state-of-the-art on the AirfRANS, ShapeNet-Car, DrivAerNet, and DrivAerNet++ benchmarks.