Persona Switch: Mixing Distinct Perspectives in Decoding Time
About
Role-play prompting is known to steer the behavior of language models by injecting a persona into the prompt, improving their zero-shot reasoning capabilities. However, such improvements are inconsistent across different tasks or instances. This inconsistency suggests that zero-shot and role-play prompting may offer complementary strengths rather than one being universally superior. Building on this insight, we propose Persona Switch, a novel decoding method that dynamically combines the benefits of both prompting strategies. Our method proceeds step-by-step, selecting the better output between zero-shot and role-play prompting at each step by comparing their output confidence, as measured by the logit gap. Experiments with widely-used LLMs demonstrate that Persona Switch consistently outperforms competitive baselines, achieving up to 5.13% accuracy improvement. Furthermore, we show that output confidence serves as an informative measure for selecting the more reliable output.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Commonsense Reasoning | CSQA | Accuracy73.3 | 366 | |
| Reasoning | GSM8K | Accuracy0.8575 | 83 | |
| Mathematical Reasoning | AQUA-RAT | Accuracy59.06 | 57 | |
| Logic reasoning | Tracking Shuffled Objects BBH | Accuracy70.4 | 54 | |
| Symbolic Reasoning | Last Letter Concatenation | Accuracy84.2 | 46 |