SIA: A Synthesize-Inject-Align Framework for Knowledge-Grounded and Secure E-commerce Search LLMs with Industrial Deployment
About
Large language models offer transformative potential for e-commerce search by enabling intent-aware recommendations. However, their industrial deployment is hindered by two critical challenges: (1) knowledge hallucination due to insufficient encoding of dynamic, fine-grained product knowledge, and (2) security vulnerabilities under jailbreak attacks that threaten compliance. To address these issues, we propose SIA--a Synthesize-Inject-Align framework for building knowledgeable and secure e-commerce search LLMs. Our approach first synthesizes high-quality natural language corpus by combining structured knowledge graphs with unstructured behavioral logs, augmented with reasoning chains and safety-aware data. We then introduce a parameter-efficient pre-training strategy based on Depth Up-Scaling to inject domain knowledge while preserving general capabilities. Finally, a dual-path alignment method via multi-task instruction tuning and adversarial training strengthens both task performance and safety robustness. The framework has been deployed at JD.com, China's largest self-operated e-commerce platform, where A/B tests across five core search scenarios demonstrate significant improvements in key business metrics, validating its industrial effectiveness and scalability.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| General Knowledge Evaluation | MMLU | MMLU Accuracy78.45 | 45 | |
| Safety and Compliance Evaluation | JDSec | SP Score28.6 | 4 | |
| E-commerce Knowledge Question Answering | EKQA | Accuracy58.01 | 3 | |
| E-commerce Query-Entity Correlation | EQEC | Accuracy74.29 | 3 | |
| Product Attribute Understanding | PAU | Accuracy91.72 | 3 | |
| General Competence Evaluation | C-Eval | Accuracy83.98 | 2 | |
| General Competence Evaluation | CMMLU | Accuracy84.94 | 2 | |
| General Competence Evaluation | MMLU-Pro | Accuracy66.82 | 2 | |
| Industrial Security Risk Identification | Online Evaluation Security and Compliance | CSR72.03 | 2 |