Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

SIA: A Synthesize-Inject-Align Framework for Knowledge-Grounded and Secure E-commerce Search LLMs with Industrial Deployment

About

Large language models offer transformative potential for e-commerce search by enabling intent-aware recommendations. However, their industrial deployment is hindered by two critical challenges: (1) knowledge hallucination due to insufficient encoding of dynamic, fine-grained product knowledge, and (2) security vulnerabilities under jailbreak attacks that threaten compliance. To address these issues, we propose SIA--a Synthesize-Inject-Align framework for building knowledgeable and secure e-commerce search LLMs. Our approach first synthesizes high-quality natural language corpus by combining structured knowledge graphs with unstructured behavioral logs, augmented with reasoning chains and safety-aware data. We then introduce a parameter-efficient pre-training strategy based on Depth Up-Scaling to inject domain knowledge while preserving general capabilities. Finally, a dual-path alignment method via multi-task instruction tuning and adversarial training strengthens both task performance and safety robustness. The framework has been deployed at JD.com, China's largest self-operated e-commerce platform, where A/B tests across five core search scenarios demonstrate significant improvements in key business metrics, validating its industrial effectiveness and scalability.

Zhouwei Zhai, Mengxiang Chen, Anmeng Zhang• 2026

Related benchmarks

TaskDatasetResultRank
General Knowledge EvaluationMMLU
MMLU Accuracy78.45
45
Safety and Compliance EvaluationJDSec
SP Score28.6
4
E-commerce Knowledge Question AnsweringEKQA
Accuracy58.01
3
E-commerce Query-Entity CorrelationEQEC
Accuracy74.29
3
Product Attribute UnderstandingPAU
Accuracy91.72
3
General Competence EvaluationC-Eval
Accuracy83.98
2
General Competence EvaluationCMMLU
Accuracy84.94
2
General Competence EvaluationMMLU-Pro
Accuracy66.82
2
Industrial Security Risk IdentificationOnline Evaluation Security and Compliance
CSR72.03
2
Showing 9 of 9 rows

Other info

Follow for update