A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detection
About
Mahalanobis distance (MD) is a simple and popular post-processing method for detecting out-of-distribution (OOD) inputs in neural networks. We analyze its failure modes for near-OOD detection and propose a simple fix called relative Mahalanobis distance (RMD) which improves performance and is more robust to hyperparameter choice. On a wide selection of challenging vision, language, and biology OOD benchmarks (CIFAR-100 vs CIFAR-10, CLINC OOD intent detection, Genomics OOD), we show that RMD meaningfully improves upon MD performance (by up to 15% AUROC on genomics OOD).
Jie Ren, Stanislav Fort, Jeremiah Liu, Abhijit Guha Roy, Shreyas Padhy, Balaji Lakshminarayanan• 2021
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Image Classification | ImageNet-1k (val) | -- | 1469 | |
| Out-of-Distribution Detection | iNaturalist | AUROC96.1 | 219 | |
| Out-of-Distribution Detection | Textures | AUROC0.8938 | 168 | |
| Out-of-Distribution Detection | OpenOOD average of NINCO, iNat, SSB-Hard, OpenImages-O, Textures | FPR @ 95%26.9 | 130 | |
| Out-of-Distribution Detection | ImageNet | FPR9533.1 | 108 | |
| Out-of-Distribution Detection | OpenImage-O | AUROC92.32 | 107 | |
| Near-OOD Detection | CIFAR-100 Near-OOD (test) | AUROC88.83 | 93 | |
| OOD Detection | CIFAR-10 | FPR@9525.35 | 85 | |
| Out-of-Distribution Detection | ImageNet-1k Textures ID OOD | AUROC89.38 | 85 | |
| Out-of-Distribution Detection | NINCO | AUROC87.31 | 82 |
Showing 10 of 83 rows
...