HiCoLoRA: Addressing Context-Prompt Misalignment via Hierarchical Collaborative LoRA for Zero-Shot DST
About
Zero-shot Dialog State Tracking (zs-DST) is essential for enabling Task-Oriented Dialog Systems (TODs) to generalize to new domains without costly data annotation. A central challenge lies in the semantic misalignment between dynamic dialog contexts and static prompts, leading to inflexible cross-layer coordination, domain interference, and catastrophic forgetting. To tackle this, we propose Hierarchical Collaborative Low-Rank Adaptation (HiCoLoRA), a framework that enhances zero-shot slot inference through robust prompt alignment. It features a hierarchical LoRA architecture for dynamic layer-specific processing (combining lower-layer heuristic grouping and higher-layer full interaction), integrates Spectral Joint Domain-Slot Clustering to identify transferable associations (feeding an Adaptive Linear Fusion Mechanism), and employs Semantic-Enhanced SVD Initialization (SemSVD-Init) to preserve pre-trained knowledge. Experiments on multi-domain datasets MultiWOZ and SGD show that HiCoLoRA outperforms baselines, achieving SOTA in zs-DST. Code is available at https://github.com/carsonz/HiCoLoRA.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Dialogue State Tracking | MultiWOZ 2.1 (test) | -- | 105 | |
| Dialogue State Tracking | SGD | JGA (Overall)55.01 | 24 | |
| Dialogue State Tracking | MultiWOZ zero-shot 2.1 | Attraction Accuracy38.86 | 11 | |
| Dialogue State Tracking | SGD Messaging | JGA67.79 | 9 | |
| Dialogue State Tracking | SGD (train) | JGA55.99 | 9 | |
| Dialogue State Tracking | SGD Media | JGA76.2 | 7 | |
| Dialogue State Tracking | SGD Flights | Joint Goal Accuracy (JGA)30.57 | 5 | |
| Dialogue State Tracking | SGD Music | JGA35.46 | 5 |