Robust LLM Watermarking with Minimal Semantic Distortion for IP Protection
About
Proprietary large language models (LLMs) face risks of intellectual property (IP) violation, as adversaries can replicate an LLM by collecting input-output pairs to train a surrogate model, causing financial setbacks. Watermarks offer a promising defense to verify ownership, but existing methods often struggle with semantic distortion, factual inconsistency, and adversarial attacks. In addition, key-conditioned watermarks for provider-specific detection, especially in cross-provider and multi-user scenarios, remain largely underexplored. To address these challenges, we propose SAFESEAL, a novel key-conditioned watermarking framework that achieves strong detectability with minimal impact on model utility, effectively balancing detectability, utility, and robustness. SAFESEAL preserves named entities while substituting linguistic terms with context-aware synonyms through a key-conditioned Tournament sampling mechanism, maintaining semantic fidelity and factual consistency. For detection, we introduce a key-conditioned contrastive detector that jointly encodes the text and key, enabling provider-specific and robust watermark verification. We derive theoretical bounds on the utility-detectability trade-off and significantly reduce latency through lightweight models, batching, and parallelism. Extensive experiments show that SAFESEAL outperforms baselines in utility, detectability, and robustness, achieving a BERTScore of 0.983, entity similarity of 0.963, a 98.2% detection rate, and the highest human ratings for text quality and content preservation, with latency comparable to the fastest baseline. To promote transparency and community-driven progress, we release the first public watermark leaderboard and an interactive demo.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Knowledge Evaluation | MMLU | MMLU Accuracy58.9 | 64 | |
| Watermark Detection | LLaMA 2 | Detection Rate98.2 | 13 | |
| Model Stealing Attacks | Mistral | BERT Score0.985 | 9 | |
| Model Stealing Attack | LLaMA 2 | BERT Score97.7 | 9 | |
| Watermark Generation | LLM Text Generation | Mean Latency (s)3.42 | 8 | |
| Watermark Detection | Mistral | Detection Rate92.7 | 7 | |
| Watermark Detection | DeepSeek | Detection Rate98.1 | 7 |