Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Are Robust LLM Fingerprints Adversarially Robust?

About

Model fingerprinting has emerged as a promising paradigm for claiming model ownership. However, robustness evaluations of these schemes have mostly focused on benign perturbations such as incremental fine-tuning, model merging, and prompting. Lack of systematic investigations into {\em adversarial robustness} against a malicious model host leaves current systems vulnerable. To bridge this gap, we first define a concrete, practical threat model against model fingerprinting. We then take a critical look at existing model fingerprinting schemes to identify their fundamental vulnerabilities. Based on these, we develop adaptive adversarial attacks tailored for each vulnerability, and demonstrate that these can bypass model authentication completely for ten recently proposed fingerprinting schemes while maintaining high utility of the model for the end users. Our work encourages fingerprint designers to adopt adversarial robustness by design. We end with recommendations for future fingerprinting methods.

Anshul Nasery, Edoardo Contente, Alkin Kaz, Pramod Viswanath, Sewoong Oh• 2025

Related benchmarks

TaskDatasetResultRank
Fingerprint SpoofingLLMmap official query set (8:2)
LLMmap Attack Success Rate (ASR)28
21
Fingerprint SpoofingUltraChat (8:2)
MET ASR6.7
21
Fingerprint SpoofingLLMmap (test)
ASR15
14
Cross-family weak-to-weak spoofingLLMmap Gemma-2B to Qwen2-1.5B
ASR16.7
14
Fingerprint SpoofingMET on UltraChat (test)
ASR6.7
14
Fingerprint SpoofingLLM-idio on UltraChat (test)
ASR2
14
Cross-family weak-to-weak spoofingLLMmap Qwen2-1.5B to Gemma-2B
ASR16
7
Cross-family weak-to-weak spoofingLLM-idio Gemma-2B to Qwen2-1.5B
ASR2.2
7
Cross-family weak-to-weak spoofingMET Qwen2-1.5B to Gemma-2B
ASR3.3
7
Cross-family weak-to-weak spoofingLLM-idio Qwen2-1.5B to Gemma-2B
ASR1
7
Showing 10 of 10 rows

Other info

Follow for update