Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

LM-Lexicon: Improving Definition Modeling via Harmonizing Semantic Experts

About

We introduce LM-Lexicon, an innovative definition modeling approach that incorporates data clustering, semantic expert learning, and model merging using a sparse mixture-of-experts architecture. By decomposing the definition modeling task into specialized semantic domains, where small language models are trained as domain experts, LM-Lexicon achieves substantial improvements (+7% BLEU score compared with the prior state-of-the-art model) over existing methods on five widely used benchmarks. Empirically, we demonstrate that 1) the clustering strategy enables fine-grained expert specialization with nearly 10% improvement in definition quality; 2) the semantic-aware domain-level routing mechanism achieves higher expert efficacy (+1%) than conventional token-level routing; and 3) further performance gains can be obtained through test-time compute and semantic expert scaling. Our work advances definition modeling while providing insights into the development of efficient language models for semantic-intensive applications.

Yang Liu, Jiaye Yang, Weikang Li, Jiahui Liang, Yang Li, Lingyong Yan• 2026

Related benchmarks

TaskDatasetResultRank
Definition Generationwordnet
BLEU47.9
23
Definition GenerationOxford
BLEU30.87
23
Definition ModelingWiki
BLEU62.07
18
Definition ModelingUrban
BLEU36.16
18
Definition Modeling3D-EX
BLEU Score54.84
18
Showing 5 of 5 rows

Other info

GitHub

Follow for update