Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Data Measurements for Decentralized Data Markets

About

Decentralized data markets can provide more equitable forms of data acquisition for machine learning. However, to realize practical marketplaces, efficient techniques for seller selection need to be developed. We propose and benchmark federated data measurements to allow a data buyer to find sellers with relevant and diverse datasets. Diversity and relevance measures enable a buyer to make relative comparisons between sellers without requiring intermediate brokers and training task-dependent models.

Charles Lu, Mohammad Mohammadi Amiri, Ramesh Raskar• 2024

Related benchmarks

TaskDatasetResultRank
Downstream Task AccuracyFEVER card (test)
Accuracy (FEVER card)48.3
21
Evidence RetrievalFEVER frozen 100-client instance (held-out)
Accuracy44.2
7
Retrieval-Augmented GenerationFEVER
Utility Score0.0158
7
Showing 3 of 3 rows

Other info

Follow for update