Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Let Them Steal: Trapping Large Language Model Extraction Attacks with Knowledge Honeypot

About

Large language models deployed as commercial APIs are vulnerable to model extraction attacks, while existing defenses either act too late or degrade utility for legitimate users. We propose \textbf{Knowledge Trap}, a defense that redirects extraction attacks toward low-transferability knowledge through a \emph{Honeypot Knowledge Graph} (HKG) and breadcrumb-guided exploration. Instead of blocking queries or perturbing outputs, Knowledge Trap consumes the attacker's limited query budget on knowledge with negligible downstream utility while preserving benign-user performance. Experiments in medical and financial domains show that Knowledge Trap reduces surrogate Agreement by 6.2\% on average without degrading legitimate-user accuracy, outperforming existing defenses that impose measurable user impact. These results suggest that defending knowledge-space traversal is a practical direction for mitigating LLM extraction attacks.

Yuyang Dai, Yushun Dong• 2026

Related benchmarks

TaskDatasetResultRank
Model Stealing DefenseMedQA
Agreement59.1
8
Model Stealing DefenseMedMCQA
Agreement (%)47.4
8
Model Stealing DefenseConvFinQA
Agreement (%)35.4
8
Model Stealing DefenseFinQA
Agreement (%)26.3
8
Model Stealing DefenseCaseHold
Agreement (%)38.6
8
Showing 5 of 5 rows

Other info

Follow for update