Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

NEAT: Neighborhood-Guided, Efficient, Autoregressive Set Transformer for 3D Molecular Generation

About

Transformer-based autoregressive models offer a promising alternative to diffusion- and flow-matching approaches for generating 3D molecular structures. However, standard transformer architectures require a sequential ordering of tokens, which is not uniquely defined for the atoms in a molecule. Prior work has addressed this by using canonical atom orderings, but these do not ensure permutation invariance of atoms, which is essential for tasks like prefix completion. We introduce NEAT, a Neighborhood-guided, Efficient, Autoregressive, Set Transformer that treats molecular graphs as sets of atoms and learns an order-agnostic distribution over admissible tokens at the graph boundary. NEAT achieves state-of-the-art performance in autoregressive 3D molecular generation whilst ensuring atom-level permutation invariance by design.

Daniel Rose, Roxane Axel Jacob, Johannes Kirchmair, Thierry Langer• 2025

Related benchmarks

TaskDatasetResultRank
3D Molecule GenerationQM9 (test)
Validity94.4
55
Unconditional 3D Molecular GenerationGEOM (test)
Atom Stability81.1
13
3D Molecular Prefix CompletionGEOM
Atom Stable0.811
2
Showing 3 of 3 rows

Other info

Follow for update