Persona-Pruner: Sculpting Lightweight Models for Role-Playing
About
Language Models (LMs) have shown remarkable potential as role-playing chatbots, delivering consistent, stylized interactions when given a specification of a character or user persona. However, applying these capabilities to real-world applications (e.g., ecosystems with numerous NPCs interacting simultaneously) exposes a critical inefficiency due to the excessive computational cost. In this paper, we question the necessity of dedicating a full, generalist model to a single persona, hypothesizing that a specific character identity relies on only a fraction of the model's total capacity. We observe that naively pruning LMs often severely degrades the role-playing performance for a specific persona; it does not distinguish between redundant knowledge and essential character traits. We propose Persona-Pruner, a framework that sculpts a lightweight role-playing model by isolating persona-specific sub-networks from a single description. Our experiments consistently show that Persona-Pruner preserves role-playing performance substantially more effectively than existing state-of-the-art LLM pruning techniques, reducing the performance drop from the dense model by up to 93.8% over the strongest baseline on RoleBench in LLM-as-a-judge score, while still maintaining general LLM capabilities. Code is available at https://github.com/jsu-kim/Persona-Pruner.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Question Answering | PIQA | Accuracy75 | 589 | |
| Role-playing | Alpaca-P | LLM-as-a-Judge Score83.9 | 91 | |
| Role-playing evaluation | RoleBench | LLM-as-a-Judge Score85.67 | 44 | |
| General Knowledge QA | OBQA | Normalized Accuracy40 | 44 | |
| General Knowledge QA | PIQA | Accuracy73 | 44 | |
| Role-playing | RoleBench (test) | LLM-as-a-Judge Score84.33 | 42 | |
| General Capability | OBQA (test) | Normalized Accuracy40 | 42 | |
| Question Answering | OBQA | Accuracy42 | 16 | |
| Language Understanding | MMLU | Accuracy59.01 | 7 | |
| Role-playing | Alpaca-P 50P x 100Q scale (test) | Alpaca-P Score81.97 | 5 |