FAIR-Calib: Frontier-Aware Instability-Reweighted Calibration for Post-Training Quantization of Diffusion Large Language Models
About
Diffusion Large Language Models (dLLMs) refine tokens iteratively but commit them irreversibly, leading to a "stability lag" where early decisions remain fragile even after being written. We reveal that Post-Training Quantization (PTQ) error easily flips these borderline decisions at the write frontier, which are then permanently locked in and amplified. To address this, we propose Frontier-Aware Instability-Reweighted Calibration (FAIR-Calib), a two-stage PTQ framework for dLLMs. Stage I probes a full-precision teacher to estimate a position prior that combines frontier hits and masked-stage reliability. Stage II performs off-policy, layer-wise calibration by minimizing a reweighted hidden-state MSE, effectively prioritizing the protection of fragile frontier states without requiring expensive end-to-end diffusion rollouts. We further theoretically justify our weighted objective as a surrogate for output KL divergence. Empirically, FAIR-Calib consistently outperforms state-of-the-art baselines on LLaDA and Dream (W4A4), significantly reducing frontier decision flips and suppressing post-commit mismatches across diverse benchmarks.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Commonsense Reasoning | WinoGrande | Accuracy76.01 | 1581 | |
| Question Answering | PIQA | Accuracy82.7 | 589 | |
| Question Answering | ARC-E | Accuracy94 | 544 | |
| Language Understanding | MMLU | MMLU Accuracy64.11 | 307 | |
| Question Answering | ARC-C | Accuracy87.46 | 283 | |
| Reading Comprehension | BoolQ | Accuracy (BoolQ)88.29 | 258 | |
| Code Generation | HumanEval | Accuracy46.95 | 212 | |
| commonsense inference | HellaSwag | Accuracy74.5 | 171 | |
| Mathematical Reasoning | GSM8K | Accuracy72.86 | 166 | |
| Truthful Question Answering | TruthfulQA MC2 | MC2 Accuracy48.01 | 66 |