Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

A Hierarchical Quantized Tokenization Framework for Task-Adaptive Graph Representation Learning

About

Foundation models in language and vision benefit from a unified discrete token interface that converts raw inputs into sequences for scalable pre-training and inference. For graphs, an effective tokenizer should yield reusable discrete codes that capture both node semantics and relational structure across scales, yet prior quantization-based graph tokenizers typically combine residual vector quantization (RVQ) levels with fixed rules and often focus on a single structural view, limiting cross-task transfer. We present a hierarchical quantized tokenization framework with task-conditioned routing and dual-view token streams. It produces multi-scale codes and two synchronized sequences: a local stream that preserves node-level information and a diffusion-style multi-hop stream that summarizes connectivity. A lightweight router learns task-dependent mixtures over RVQ depths to select an appropriate granularity, while a gated cross-attention module aligns and fuses the two streams into a single token sequence without altering the downstream backbone encoder. Experiments on node classification and link prediction show consistent gains over strong quantized baselines at matched compute, with ablations verifying contributions from hierarchical quantization, adaptive routing, and fusion.

Yang Xiang, Li Fan, Chenke Yin, Lutz Oettershagen, Chengtao Ji• 2025

Related benchmarks

TaskDatasetResultRank
Node ClassificationPubmed
Accuracy90.18
363
Node ClassificationAmazon Photo
Accuracy96.03
313
Graph RegressionZINC
MAE0.051
144
Graph ClassificationCIFAR10
Accuracy77.784
118
Node Classificationogbn-proteins
ROC AUC80.12
113
Graph ClassificationMNIST
Accuracy98.213
103
Link Predictionogbl-collab
Hits@5067.67
53
Link Predictionogbl-ppa
Hits@10062.86
49
Node ClassificationCoraFull
Accuracy85.62
26
Link PredictionCora
MRR44.86
11
Showing 10 of 12 rows

Other info

Follow for update