Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

FEMNIST

Benchmarks

Task NameDataset NameSOTA ResultTrend
Image ClassificationFEMNIST (test)
Accuracy97.31
104
Image ClassificationFEMNIST (LEAF) (test)
Accuracy82.45
39
Image ClassificationFEMNIST
Accuracy88.2
36
Image ClassificationFEMNIST (test)
Mean Accuracy70.6
14
Image ClassificationFEMNIST C=10 original (test)
Accuracy82.02
13
Image ClassificationFEMNIST (non-iid)
Accuracy84
13
Multi-class classificationFEMNIST realistic non-IID LEAF benchmark (test)
Top-1 Acc85.2
11
Image ClassificationFEMNIST LEAF
Accuracy80.61
10
Image ClassificationFEMNIST Practical
Mean Accuracy100
9
Image ClassificationFEMNIST 10 new writers (test)
Overall Accuracy88.18
9
Federated Image ClassificationFEMNIST non-i.i.d.
Test Error14.76
8
Federated Image ClassificationFEMNIST i.i.d.
Test Error Rate14.79
8
Image ClassificationFEMNIST non-i.i.d.
Test Error Rate14.76
8
Image ClassificationFEMNIST (i.i.d.)
Test Error14.79
8
Image ClassificationFEMNIST (test)
Accuracy (alpha=0.1)80.6
7
Handwritten character recognitionFEMNIST 7 (test)
Avg Accuracy79.9
7
Backdoor DefenseFEMNIST
Clean Accuracy73.5
6
Image ClassificationFEMNIST Dir-0.1 (test)
Accuracy86.3
6
Client Contribution Estimation CorrelationFEMNIST (Natural split)
Average Pearson Correlation0.41
5
Image ClassificationFEMNIST (Natural split)
Accuracy61.17
5
Federated UnlearningFEMNIST
Average Unlearning Time (s)475
5
Personalized Federated LearningFEMNIST (test)
Top 10% Most Samples Accuracy76.24
5
Image ClassificationFEMNIST (Non-IID 2)
Accuracy80.32
4
Image ClassificationFEMNIST (Non-IID 1)
Accuracy81.12
4
Image ClassificationFEMNIST IID
Accuracy80.24
4
Showing 25 of 30 rows