Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

OmniFysics: Towards Physical Intelligence Evolution via Omni-Modal Signal Processing and Network Optimization

About

The autonomous evolution of networked AI systems relies heavily on robust environmental perception. However, physical understanding remains brittle in current models because key physical signals are visually ambiguous and sparsely represented in web-scale data. To bridge the gap between data-centric learning and knowledge-based physical rules, we present OmniFysics, a compact omni-modal network that unifies signal processing and understanding across images, audio, video, and text. To enable autonomous optimization and inject explicit physical knowledge, we construct a dynamic physical data engine. Within this engine, FysicsAny acts as an adaptive mechanism that produces physics-grounded supervision by mapping salient objects to verified physical attributes via hierarchical retrieval and physics-law-constrained signal verification. Concurrently, FysicsOmniCap distills web videos utilizing advanced audio-visual cross-modal signal processing, generating high-fidelity data pairs that emphasize dynamic physical cues. We optimize the OmniFysics network through staged multimodal alignment and evolutive instruction tuning, integrating latent-space flow matching for generation and an adaptive intent router for efficient execution. Experiments demonstrate that this evolutive optimization paradigm not only achieves competitive performance on standard multimodal benchmarks but also significantly advances physics-oriented evaluations.

Minghao Han, Dingkang Yang, Yue Jiang, Yizhou Liu, Lihua Zhang• 2026

Related benchmarks

TaskDatasetResultRank
Text-to-Image GenerationGenEval
GenEval Score63
360
Video UnderstandingVideoMME
Overall Score63.8
222
Text-to-Image GenerationDPG-Bench
DPG Score76.49
131
Video UnderstandingWorldSense
Score45.39
25
Audio UnderstandingMMAR
MMAR56.8
12
Physical PerceptionPAI-Bench
PAI-Bench Score57.7
9
Physical PerceptionQuantiPhy
QuantiPhy Score38.5
9
Physical PerceptionFysicsEval
Prediction Score32.6
9
Physical PerceptionPhysBench
PhysBench Score47.2
9
Physical PerceptionPhysUniBench
PhysUniBench Score50.8
9
Showing 10 of 13 rows

Other info

Follow for update