Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Causal Path Alignment: Anchoring the Optimization Trajectory for Controllable In-Parameter Knowledge Editing

About

Knowledge editing is pivotal for efficiently updating the parametric memory of Large Language Models (LLMs), enabling them to function as evolving agents in dynamic environments. However, mainstream in-parameter knowledge editing approaches suffer from Subject-Dominant Memory Interference: modifying a specific fact inadvertently corrupts the broader structural knowledge associated with the same subject within LLMs. We diagnose the root cause as a shortcut learning pathology, where the optimization objective overfits subject representations while bypassing the essential relational context. To rectify this, we propose Causal Path Alignment (CPA), a principled framework designed to anchor the optimization trajectory to valid causal pathways. CPA enforces parameter updates to route through relation-aware intermediate states, thereby preventing the erasure of contextual dependencies. Experimental results across diverse LLM backbones demonstrate that CPA consistently eliminates the shortcut, significantly improving relation specificity while exhibiting minimal side-effects. Moreover, CPA serves as a model-agnostic plug-in for existing editors, paving the way for reliable and trustworthy in-parameter knowledge editing.

Xiyu Liu, Zhengxiao Liu, Naibin Gu, Zheng Lin, Weiping Wang• 2025

Related benchmarks

TaskDatasetResultRank
Knowledge EditingzsRE
Generality71
268
Knowledge EditingCOUNTERFACT RS
Efficacy99.6
23
Knowledge EditingEVOKE
DP7.8
23
Showing 3 of 3 rows

Other info

Follow for update