Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Hey, That's My Model! Introducing Chain & Hash, An LLM Fingerprinting Technique

About

Growing concerns over the theft and misuse of Large Language Models (LLMs) underscore the need for effective fingerprinting to link a model to its original version and detect misuse. We define five essential properties for a successful fingerprint: Transparency, Efficiency, Persistence, Robustness, and Unforgeability. We present a novel fingerprinting framework that provides verifiable proof of ownership while preserving fingerprint integrity. Our approach makes two main contributions. First, a chain and hash technique that cryptographically binds fingerprint prompts to their responses, preventing collisions and enabling irrefutable ownership claims. Second, we address a realistic threat model in which instruction-tuned models' output distribution can be significantly altered through meta-prompts. By incorporating random padding and varied meta-prompt configurations during training, our method maintains robustness even under significant output style changes. Experiments show that our framework securely proves ownership, resists both benign transformations (e.g., fine-tuning) and adversarial fingerprint removal, and extends to fingerprinting LoRA adapters\footnote{We release our code at: https://github.com/microsoft/Chain-Hash.

Mark Russinovich, Yanan Cai, Ahmed Salem• 2024

Related benchmarks

TaskDatasetResultRank
Attack Success RateCTCC fingerprinting scenario b
SVA100
18
General Capability HarmlessnessGeneral LLM Task Benchmark
Average Accuracy59.8
12
Input Perturbation RobustnessInput Perturbation Remove 5%
FSR (5% Removal)100
10
Fingerprinting EffectivenessFingerprinted Model Clean
FSR100
10
Input Perturbation RobustnessInput Perturbation Remove 10%
FSR92
10
Fine-tuning RobustnessShareGPT
FSR1.00e+3
10
Fine-tuning RobustnessAlpaca Dataset
FSR0.00e+0
10
Fine-tuning RobustnessDolly Dataset
FSR0.00e+0
10
Input Stealthiness AssessmentNarrative-based corpus
PPL86.31
8
Input Stealthiness EvaluationFingerprint Input Triggers (test)
Perplexity (PPL)86.31
6
Showing 10 of 14 rows

Other info

Follow for update