ASDKit: A Toolkit for Comprehensive Evaluation of Anomalous Sound Detection Methods
About
In this paper, we introduce ASDKit, a toolkit for anomalous sound detection (ASD) task. Our aim is to facilitate ASD research by providing an open-source framework that collects and carefully evaluates various ASD methods. First, ASDKit provides training and evaluation scripts for a wide range of ASD methods, all handled within a unified framework. For instance, it includes the autoencoder-based official DCASE baseline, representative discriminative methods, and self-supervised learning-based methods. Second, it supports comprehensive evaluation on the DCASE 2020--2024 datasets, enabling careful assessment of ASD performance, which is highly sensitive to factors such as datasets and random seeds. In our experiments, we re-evaluate various ASD methods using ASDKit and identify consistently effective techniques across multiple datasets and trials. We also demonstrate that ASDKit reproduces the state-of-the-art-level performance on the considered datasets.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Anomalous Sound Detection | DCASE 2020 (dev) | Official Performance Metric76.9 | 46 | |
| Anomalous Sound Detection | DCASE 2023 (dev) | Performance Metric62.3 | 17 | |
| Anomalous Sound Detection | DCASE 2023 (eval) | Official Performance Score62.6 | 17 | |
| Anomalous Sound Detection | DCASE 2020 | Dataset-wise Harmonic Mean77.2 | 16 | |
| Anomalous Sound Detection | DCASE 2024 | Dataset-wise Harmonic Mean56.7 | 16 | |
| Anomalous Sound Detection | DCASE 2023 | Dataset-wise Harmonic Mean62.4 | 16 | |
| Anomalous Sound Detection | DCASE 2024 (eval) | Official Performance Metric55.7 | 16 | |
| Anomalous Sound Detection | DCASE 2020 (eval) | Official Performance Metric77.5 | 15 | |
| Anomalous Sound Detection | DCASE 2024 (dev) | Performance Score57.7 | 14 | |
| Anomalous Sound Detection | DCASE 2022 | Dataset-wise Harmonic Mean60.9 | 12 |