Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

FAIR-Calib: Frontier-Aware Instability-Reweighted Calibration for Post-Training Quantization of Diffusion Large Language Models

About

Diffusion Large Language Models (dLLMs) refine tokens iteratively but commit them irreversibly, leading to a "stability lag" where early decisions remain fragile even after being written. We reveal that Post-Training Quantization (PTQ) error easily flips these borderline decisions at the write frontier, which are then permanently locked in and amplified. To address this, we propose Frontier-Aware Instability-Reweighted Calibration (FAIR-Calib), a two-stage PTQ framework for dLLMs. Stage I probes a full-precision teacher to estimate a position prior that combines frontier hits and masked-stage reliability. Stage II performs off-policy, layer-wise calibration by minimizing a reweighted hidden-state MSE, effectively prioritizing the protection of fragile frontier states without requiring expensive end-to-end diffusion rollouts. We further theoretically justify our weighted objective as a surrogate for output KL divergence. Empirically, FAIR-Calib consistently outperforms state-of-the-art baselines on LLaDA and Dream (W4A4), significantly reducing frontier decision flips and suppressing post-commit mismatches across diverse benchmarks.

Haoyu Huang, Linlin Yang, Sheng Xu, Boyu Liu, Guodong Guo, Zhongqian Fu, Hang Zhou, Baochang Zhang• 2026

Related benchmarks

TaskDatasetResultRank
Commonsense ReasoningWinoGrande
Accuracy76.01
1581
Question AnsweringPIQA
Accuracy82.7
589
Question AnsweringARC-E
Accuracy94
544
Language UnderstandingMMLU
MMLU Accuracy64.11
307
Question AnsweringARC-C
Accuracy87.46
283
Reading ComprehensionBoolQ
Accuracy (BoolQ)88.29
258
Code GenerationHumanEval
Accuracy46.95
212
commonsense inferenceHellaSwag
Accuracy74.5
171
Mathematical ReasoningGSM8K
Accuracy72.86
166
Truthful Question AnsweringTruthfulQA MC2
MC2 Accuracy48.01
66
Showing 10 of 16 rows

Other info

Follow for update