OccFace: Unified Occlusion-Aware Facial Landmark Detection with Per-Point Visibility
About
Accurate facial landmark detection under occlusion remains challenging, especially for human-like faces with large appearance variation and rotation-driven self-occlusion. Existing detectors typically localize landmarks while handling occlusion implicitly, without predicting per-point visibility that downstream applications can benefits. We present OccFace, an occlusion-aware framework for universal human-like faces, including humans, stylized characters, and other non-human designs. OccFace adopts a unified dense 100-point layout and a heatmap-based backbone, and adds an occlusion module that jointly predicts landmark coordinates and per-point visibility by combining local evidence with cross-landmark context. Visibility supervision mixes manual labels with landmark-aware masking that derives pseudo visibility from mask-heatmap overlap. We also create an occlusion-aware evaluation suite reporting NME on visible vs. occluded landmarks and benchmarking visibility with Occ AP, F1@0.5, and ROC-AUC, together with a dataset annotated with 100-point landmarks and per-point visibility. Experiments show improved robustness under external occlusion and large head rotations, especially on occluded regions, while preserving accuracy on visible landmarks.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Facial Landmark Detection | WFLW (Full) | NME (%)3.77 | 65 | |
| Facial Landmark Localization | 300-W (Full set) | NME2.83 | 46 | |
| Facial Landmark Localization | 300-W (Challenging set) | NME3.36 | 32 | |
| Facial Landmark Localization | 300W Common | NME2.33 | 20 | |
| Facial Landmark Detection | COFW | NME4.53 | 14 | |
| Facial Landmark Localization | Genie-Face-100 | NME3.01 | 3 | |
| Landmark Localization | Genie-Face-74 | NME2.87 | 3 | |
| Landmark Localization | Genie-Face-100 | NME3.01 | 3 | |
| Visibility Prediction | Genie-Face-100 | Occ AP58.2 | 3 |