Meta-classification of one-class classification models using ranking correlation and nearest neighbor
About
Machine Learning (ML) techniques have been applied to various problems. However, applying ML to ML models is an unexplored direction. For this purpose, this paper considers a meta-classification of one-class classification (OCC) models, because all ML models could be approximated as OCC models. The proposal represents OCC models as normality rankings and classifies them using nearest-neighbor and ranking-correlation metrics. The experiment classifies OCC models, where classes correspond to training datasets, algorithms, and hyperparameters. The proposal achieves high accuracy when class labels are datasets. Moreover, it can classify algorithms when the training datasets contain the same class. In addition, the discussion highlights that the classification of OCC models is essentially the classification of datasets that treats multiple samples as a single input. The experiment demonstrates the classification of datasets using sleeping records. The proposed method can provide a unified solution for classifying OCC models, datasets, and rankings. Source code is uploaded to the public repository https://github.com/ToshiHayashi/ClassOCC.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Hyperparameter Classification | NSL-KDD Normal | Accuracy94 | 10 | |
| Hyperparameter Classification | NSL-KDD (Abnormal) | Accuracy (%)100 | 10 | |
| Binary classification (normal vs abnormal) | KDD Cup subsets (train) | Accuracy100 | 4 | |
| 3-class classification (normal vs abnormal vs mix) | KDD Cup subsets (train) | Accuracy99.4 | 2 | |
| Classification of base learners (4 classes) | KDD Normal | Accuracy (KDD Normal)100 | 2 | |
| Classification of base learners (4 classes) | Abnormal KDD | Accuracy100 | 2 | |
| Classification of ensemble models (15 classes) | KDD Normal | Accuracy85.5 | 2 | |
| Classification of ensemble models (15 classes) | Abnormal KDD | Accuracy34.1 | 2 |