Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Dataset Usage Inference without Shadow Models or Held-out Data

About

How much of my data was used to train a machine learning model? Dataset Usage Inference (DUI) aims to answer this by estimating what fraction of a dataset contributed to a model's training. However, existing DUI methods rely on assumptions that rarely hold in practice: they require training expensive shadow models to imitate the target model, and they assume access to both known training samples and an in-distribution held-out set confirmed to be absent from training. These conditions make current approaches impractical for modern large models and real data ownership disputes. We introduce a practical DUI framework that removes these constraints. Our method requires neither shadow models nor real held-out data. Instead, it generates synthetic non-member samples, extracts diverse membership signals, and casts DUI as a mixture proportion estimation problem to estimate what share of the candidate dataset was used during training. Experiments on large image generative models show that our method reliably quantifies dataset usage, providing a practical tool for data owners to determine how much of their data was used to train a model.

Wojciech {\L}apacz, Stanis{\l}aw Pawlak, Jan Dubi\'nski, Franziska Boenisch, Adam Dziedzic• 2026

Related benchmarks

TaskDatasetResultRank
Dataset usage estimationRAR-XL--
12
Dataset usage estimationRAR-XXL--
12
Dataset usage estimationVAR 24--
12
Dataset usage estimationVAR-30--
12
Dataset usage estimationDiT-RF-XL--
12
Membership Inference Computation Cost AnalysisRAR-XXL (suspect set of 1,000 images)
Shadow Model Cost (h)0.00e+0
3
Showing 6 of 6 rows

Other info

Follow for update