A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detection
About
Mahalanobis distance (MD) is a simple and popular post-processing method for detecting out-of-distribution (OOD) inputs in neural networks. We analyze its failure modes for near-OOD detection and propose a simple fix called relative Mahalanobis distance (RMD) which improves performance and is more robust to hyperparameter choice. On a wide selection of challenging vision, language, and biology OOD benchmarks (CIFAR-100 vs CIFAR-10, CLINC OOD intent detection, Genomics OOD), we show that RMD meaningfully improves upon MD performance (by up to 15% AUROC on genomics OOD).
Jie Ren, Stanislav Fort, Jeremiah Liu, Abhijit Guha Roy, Shreyas Padhy, Balaji Lakshminarayanan• 2021
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Image Classification | ImageNet-1k (val) | -- | 1498 | |
| Out-of-Distribution Detection | SUN OOD with ImageNet-1k In-distribution (test) | AUROC87.41 | 267 | |
| Out-of-Distribution Detection | iNaturalist | AUROC96.1 | 252 | |
| Out-of-Distribution Detection | Textures | AUROC0.8938 | 186 | |
| Out-of-Distribution Detection | ImageNet-1k ID iNaturalist OOD | FPR9510.62 | 143 | |
| OOD Detection | ImageNet 1k (test) | FPR9534.6 | 134 | |
| Out-of-Distribution Detection | OpenOOD average of NINCO, iNat, SSB-Hard, OpenImages-O, Textures | FPR @ 95%26.9 | 130 | |
| Out-of-Distribution Detection | OpenImage-O | AUROC92.61 | 117 | |
| Out-of-Distribution Detection | ImageNet | AUROC89.3 | 113 | |
| Out-of-Distribution Detection | ImageNet-1k Textures ID OOD | AUROC89.38 | 96 |
Showing 10 of 113 rows
...