Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

FedLAB: Traceable Semantic Codebooks for Federated Multimodal Graph Foundation Learning

About

Multimodal graph foundation models aim to learn reusable knowledge from graphs enriched with text, images, attributes, and relational topology, thereby supporting diverse graph-centric and modality-centric tasks. In practice, however, such multimodal graphs are often distributed across decentralized clients, where raw contents and local structures cannot be centrally shared due to privacy constraints. This motivates federated multimodal graph foundation learning, which requires not only transferable representation learning but also intrinsic semantic traceability under strict data isolation. Existing methods usually exchange or store knowledge through parameters, prototypes, embeddings, or compact codebooks, which support optimization and transfer but do not explicitly expose how modality evidence, node semantics, and topology context jointly support predictions. To bridge this gap, we propose FedLAB, a traceable semantic codebook framework that organizes multimodal graph knowledge into typed hierarchical codebooks for modality evidence, node semantics, and topology context. FedLAB further refines these trace units through federated semantic barycenter pre-training while keeping raw multimodal contents and graph structures local. Extensive experiments on 10 benchmarks and 6 downstream tasks show that FedLAB improves over state-of-the-art baselines by up to 7.53\%, while preserving a native semantic trace interface.

Zekai Chen, Kairui Yang, Xuaner Chen, Xunkai Li, Xun Wu, Rong-Hua Li, Guoren Wang• 2026

Related benchmarks

TaskDatasetResultRank
Node ClassificationGrocery
Accuracy84.2
139
Node ClassificationToys
Accuracy84.14
77
Graph-to-ImageSemArt
CLIP-S Score77.93
36
Modality MatchKU
AUC63.5
28
Modality MatchBili Food
AUC65.6
28
Modality RetrievalQB (Full)
R@594.49
25
Link PredictionDY
AUC74.71
19
Graph-to-TextFlickr30K--
14
Link PredictionBili music
AUC74.32
10
Modality RetrievalBili Cartoon
Recall@581.48
10
Showing 10 of 10 rows

Other info

Follow for update