Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

From Scene-Centric to Observer-Centric: Modeling Observer-Aware Relations for 3D Scene Graph Generation

About

3D Scene Graph Generation (3DSGG) represents 3D scenes as structured object--relation--object graphs for spatial understanding. In observer-centric spatial perception, the same scene may be expressed under different local observer frames while its structure remains unchanged. However, existing models typically assume a fixed scene-aligned reference frame and may produce semantically inconsistent predictions when the scene is re-expressed in another observer frame. We attribute this failure to the heterogeneous frame dependency of relational predicates. Directional predicates such as $\textit{left}$, $\textit{front}$, $\textit{right}$, and $\textit{behind}$ are $\textbf{Observer-Dependent Relations}$, whereas most contact, support, and semantic predicates, such as $\textit{standing on}$ and $\textit{attached to}$, are approximately $\textbf{Observer-Independent Relations}$. Conventional models do not distinguish these frame responses, leading to degraded relation prediction under observer-frame reorientation. We introduce $\textbf{Observer-Aware Relations (OAR)}$, which combines observer-aware geometric encoding and relation specialization, supported by frame-stable object encoding, for unified multi-label predicate prediction. Experiments on 3DSSG show that OAR consistently outperforms baselines across controlled observer-frame reorientations without training-time frame-reorientation augmentation, while remaining competitive on the standard benchmark. The project page is available at https://oar-predicate.github.io/.

Jingjun Sun, Chaowei Wang, Zhirui Liu, Jiaxu Tian, Ming Yang, Yaoxing Wang, Yan Di, Shan Gao• 2026

Related benchmarks

TaskDatasetResultRank
3D scene graph generation3DSSG (standard unrotated)
Object Recall @160.51
6
Predicate Classification3DSSG standard (test)
Recall @ 2073.1
6
Scene Graph Classification3DSSG standard (test)
Recall @ 2036.4
6
Showing 3 of 3 rows

Other info

Follow for update