Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

CoreQ: Learning-Free Mismatch Correction and Successive Rounding for Quantization

About

Post-training quantization (PTQ) enables efficient deployment of large language models by mapping pretrained weights to low-bit formats without retraining, typically using a small calibration set to minimize a layer-wise calibration objective. However, this sequential procedure induces a mismatch: errors from earlier quantized layers alter the inputs received by later layers, causing the activations to deviate from those of the full-precision model. Recent approaches introduce mismatch-aware calibration objectives to compensate for this effect, but leave open how much of the observed mismatch should shift each layer's calibration target. Fully applying this correction can overfit limited calibration data, while scaling the mismatch correction with a fixed coefficient ignores varying reliability of mismatch estimates across layers. To address these limitations, we propose CoreQ, a learning-free PTQ framework that applies a closed-form coefficient for mismatch correction derived from a geometric decomposition of the mismatch. The resulting coefficient adapts the correction across layers, reduces overfitting to finite calibration data, and requires no hyperparameter tuning. Given the corrected target, CoreQ minimizes the induced triangular least-squares objective with an efficient greedy successive-rounding solver and a bounded beam-search extension, K-CoreQ, that trades modest additional compute for improved performance. Across multiple LLM families, scales, bit-widths, and quantization settings, CoreQ improves perplexity and downstream accuracy over strong PTQ baselines.

Seohyeon Cha, Huancheng Chen, Dongjun Kim, Haoran Zhang, Kevin Chan, Gustavo de Veciana, Haris Vikalo• 2026

Related benchmarks

TaskDatasetResultRank
Language ModelingWikiText2
Perplexity3.44
3785
Language ModelingWikiText-2 (test)
PPL5
2333
Language ModelingC4
Perplexity5.59
1565
Language ModelingC4 (test)
Perplexity10.74
464
Language ModelingC4
C4 Loss6.57
121
Zero-shot EvaluationEvaluation Tasks Zero-shot Aggregate
Avg. Accuracy71.11
74
Zero-shot performance evaluationLM Eval Harness (HellaSwag, BoolQ, WinoGrande, PiQA, ARC-easy, ARC-challenge) zero-shot
Mean Accuracy73.26
60
Showing 7 of 7 rows

Other info

Follow for update