Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Coverage-Based Calibration for Post-Training Quantization via Weighted Set Cover over Outlier Channels

About

Post-Training Quantization (PTQ) compresses large language models to low bit-widths using a small calibration set, and its quality depends strongly on which samples are chosen. We identify a failure mode in which calibration samples fail to activate outlier channels, hidden dimensions with unusually large activations, causing the quantizer to underestimate their dynamic range and producing per-channel reconstruction errors that dominate layer-wise loss. Motivated by this observation, we argue that PTQ calibration quality is governed more by weighted outlier-channel coverage than by generic sample representativeness, and formulate calibration selection as a weighted set cover problem over outlier channels. The objective is monotone submodular, and the greedy algorithm, COVERCAL, operates on pre-computed activation statistics and requires no GPU time at selection. We further show that the weight choice is internally consistent: under a stylized clipping model, missed weighted coverage upper-bounds surrogate loss, justifying the weighted coverage objective as principled rather than purely empirical. Across LLaMA-2, LLaMA-3, and Mistral, under AWQ and GPTQ backends and five downstream evaluations, COVERCAL improves over random, max-perplexity, max-activation-variance, and stratified baselines, with the largest gains at small calibration budgets. At INT4 with 128 samples, COVERCAL improves MMLU by 1.2 to 1.5 points over random calibration and reduces perplexity degradation by 15 to 30\%; with 64 samples, it matches or exceeds random calibration at 256. The contribution is not a new PTQ backend but a formulation of calibration selection as weighted outlier coverage, with a simple, efficient algorithm and a surrogate-based justification.

Ibne Farabi Shihab, Sanjeda Akter, Anuj Sharma• 2026

Related benchmarks

TaskDatasetResultRank
Language ModelingWikiText-2
Perplexity (PPL)5.31
2320
Commonsense ReasoningWinoGrande--
1442
Question AnsweringARC Challenge
Accuracy (ARC)60.3
598
Sentence CompletionHellaSwag
Accuracy82.4
364
Language UnderstandingMMLU (test)
MMLU Average Accuracy54.6
167
Language UnderstandingMMLU 5-shot
Accuracy64.6
153
Language UnderstandingMMLU
MMLU Accuracy64.1
132
Question AnsweringARC Challenge (test)
Accuracy58.9
103
Commonsense ReasoningWinoGrande 5-shot
Accuracy76.9
85
Language ModelingWikitext (test)
Perplexity4.94
66
Showing 10 of 14 rows

Other info

Follow for update