JAFAR: Jack up Any Feature at Any Resolution
About
Foundation Vision Encoders have become essential for a wide range of dense vision tasks. However, their low-resolution spatial feature outputs necessitate feature upsampling to produce the high-resolution modalities required for downstream tasks. In this work, we introduce JAFAR, a lightweight and flexible feature upsampler that enhances the spatial resolution of visual features from any Foundation Vision Encoder to an arbitrary target resolution. JAFAR employs an attention-based module designed to promote semantic alignment between high-resolution queries, derived from low-level image features, and semantically enriched low-resolution keys, using Spatial Feature Transform (SFT) modulation. Notably, despite the absence of high-resolution supervision, we demonstrate that learning at low upsampling ratios and resolutions generalizes remarkably well to significantly higher output scales. Extensive experiments show that JAFAR effectively recovers fine-grained spatial details and consistently outperforms existing feature upsampling methods across a diverse set of downstream tasks. Project page at https://jafar-upsampler.github.io
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Semantic segmentation | ADE20K (val) | mIoU41.96 | 2731 | |
| Semantic segmentation | PASCAL VOC (val) | mIoU84.38 | 338 | |
| Semantic segmentation | COCO Stuff | mIoU61.82 | 195 | |
| Semantic segmentation | Pascal VOC | mIoU0.8436 | 172 | |
| Semantic segmentation | COCO Stuff (val) | mIoU61.71 | 126 | |
| Monocular Depth Estimation | NYU V2 | Delta 1 Acc91.8 | 113 | |
| Semantic segmentation | Pascal VOC 21 classes (val) | mIoU0.8444 | 103 | |
| Semantic segmentation | COCO Stuff-27 (val) | mIoU60.78 | 75 | |
| Open-Vocabulary Segmentation | Cityscapes | mIoU25.26 | 49 | |
| Semantic segmentation | VOC | mIoU84.38 | 44 |