Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Forgetting That Sticks: Quantization-Permanent Unlearning via Circuit Attribution

About

Standard unlearning evaluations measure behavioral suppression in full precision, immediately after training, despite every deployed language model being quantized first. Recent work has shown that 4-bit post-training quantization can reverse machine unlearning; we show this is not a tuning artefact but a systematic dual failure: gradient-based methods that achieve meaningful forgetting lose it under compression, while methods that survive quantization barely change the model. Both failures trace to the same root cause: across all baselines, per-parameter updates lie 47-828x below the NF4 quantization bin width; updates diffused across billions of parameters cannot clear quantization bin boundaries, a consequence we formalize as a sparsity-permanence tradeoff. We present MANSU (Mechanistic-Aligned Null-Space Unlearning), which resolves both modes by combining causal circuit attribution to isolate the minimal forget-set subgraph, circuit-restricted null-space projection with a diagonal-Fisher retain bound, and a per-parameter magnitude floor guaranteeing quantization survival by construction. We additionally introduce Circuit Attribution Divergence (CAD), a mechanistic verification metric distinguishing structural erasure from behavioral suppression, a distinction existing metrics cannot make. Across multiple model families and hazard benchmarks, MANSU is the first method to jointly satisfy all four properties with margin on each (meaningful forgetting, retain preservation, non-positive PTQ gap, and structural erasure), while gradient-based baselines recover up to +0.05 accuracy under compression.

Saisab Sadhu, Pratinav Seth, Vinay Kumar Sankarapu• 2026

Related benchmarks

TaskDatasetResultRank
General Knowledge EvaluationMMLU
MMLU Accuracy73.7
127
Utility EvaluationMMLU
MMLU Score47.8
45
Knowledge RetentionWMDP retain
Retain40.5
36
Machine UnlearningWMDP-cyber 1.0 (test)
BF16 Score49.7
28
Machine UnlearningWMDP-bio 1.0 (test)
BF16 Accuracy61.7
28
Machine UnlearningWMDP-chem 1.0 (test)
BF160.334
28
UnlearningWMDP retain
Retain51.1
22
UnlearningWMDP (forget split)
BF16 Precision43.2
22
Instruction FollowingIFEval
IFScore55.1
21
Knowledge RetentionWMDP cyber (retain)
Rt54.1
16
Showing 10 of 24 rows

Other info

Follow for update