Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Attacks and Defenses Against LLM Fingerprinting

About

As large language models are increasingly deployed in sensitive environments, fingerprinting attacks pose significant privacy and security risks. We present a study of LLM fingerprinting from both offensive and defensive perspectives. Our attack methodology uses reinforcement learning to automatically optimize query selection, achieving better fingerprinting accuracy with only 3 queries compared to randomly selecting 3 queries from the same pool. Our defensive approach employs semantic-preserving output filtering through a secondary LLM to obfuscate model identity while maintaining semantic integrity. The defensive method reduces fingerprinting accuracy across tested models while preserving output quality. These contributions show the potential to improve fingerprinting tools capabilities while providing practical mitigation strategies against fingerprinting attacks.

Kevin Kurian, Ethan Holland, Sean Oesch• 2025

Related benchmarks

TaskDatasetResultRank
Fingerprint SpoofingLLMmap official query set (8:2)
LLMmap Attack Success Rate (ASR)40
21
Fingerprint SpoofingUltraChat (8:2)
MET ASR33.3
21
Cross-family weak-to-weak spoofingLLMmap Gemma-2B to Qwen2-1.5B
ASR3.3
14
Fingerprint SpoofingLLMmap (test)
ASR0.00e+0
14
Fingerprint SpoofingMET on UltraChat (test)
ASR0.00e+0
14
Fingerprint SpoofingLLM-idio on UltraChat (test)
ASR1.8
14
LLM black-box fingerprintingLLM instances
Query Efficiency (Verification)8
12
Cross-family weak-to-weak spoofingLLM-idio Qwen2-1.5B to Gemma-2B
ASR8.9
7
Cross-family weak-to-weak spoofingLLMmap Qwen2-1.5B to Gemma-2B
ASR0.00e+0
7
Cross-family weak-to-weak spoofingMET Qwen2-1.5B to Gemma-2B
ASR0.00e+0
7
Showing 10 of 11 rows

Other info

Follow for update