DuIVRS-2: An LLM-based Interactive Voice Response System for Large-scale POI Attribute Acquisition
About
Accurate Point of Interest (POI) attribute acquisition is essential for location-based services, yet traditional modular Interactive Voice Response (IVR) systems suffer from error accumulation and high maintenance overhead. We present DuIVRS-2, a large language model (LLM)-based end-to-end framework designed for large-scale POI attribute acquisition at Baidu Maps. To address the long-tail distribution of real-world interactions, our methodology first employs a finite state machine (FSM)-guided data augmentation strategy to synthesize a balanced and diverse training dataset. We then streamline dialogue management via a selective generation scheme combined with a Chain-of-Thought (CoT) mechanism, which ensures output stability and effectively eliminates hallucinations in industrial settings. To facilitate continuous policy refinement with minimal manual effort, we design a cooperative iterative learning framework that leverages a dual-evaluator voting system. Deployed in production for two months, DuIVRS-2 processed 0.4 million calls daily and achieved a 83.9\% Task Success Rate (TSR), outperforming its predecessor by 4 percentage points while maintaining a low reaction time of 130ms. This work provides a production-proven reference for developing robust, cost-effective LLM agents for large-scale industrial dialogue applications.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| POI Attribute Acquisition (Task-oriented Dialogue) | DuIVRS logs (Deffect) 1.0 (test) | Completion Rate81.62 | 9 | |
| POI Attribute Acquisition (Task-oriented Dialogue) | DuIVRS logs Drobust 1.0 (test) | Completion Rate (CR)76.22 | 9 | |
| POI Attribute Acquisition (Task-oriented Dialogue) | DuIVRS logs Dgeneral 1.0 (test) | Completion Rate73.7 | 9 | |
| POI attribute acquisition | Baidu Maps Online A/B (test) | TSR83.9 | 3 |