Navigating User Behavior toward Personalized Multimodal Generation
About
Modern AIGC pipelines deliver high-fidelity images and videos but presuppose a well-formed creation instruction, while end users rarely articulate visual details, leaving generators misaligned with user demand. We study personalized content generation, which turns a user's interaction history into an executable instruction for downstream synthesis, and identify two obstacles: behavior must be encoded in a form legible to language reasoning, and the model must acquire instruction-writing skill absent from both pretraining and behavior data. We propose NaviGen, which represents each item with a dual identifier coupling a collaborative code and a textual code as a behavioral substrate and a semantic bridge in one token stream. On this representation, a two-stage SFT+RL pipeline first distills preference reasoning and instruction writing from evolutionarily searched supervision, then aligns generation with user intent through hierarchical and self-consistent rewards. Experiments across product, game, and short-video domains show that NaviGen improves personalized image and video generation, strengthens next-item prediction, and yields more specific, relevant, and visually generatable instructions. Our code is released at: https://github.com/iLearn-Lab/NaviGen.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Personalized Image Generation | Games | Consistency0.3346 | 9 | |
| Personalized Image Generation | Short Videos | Consistency33.9 | 9 | |
| Personalized Image Generation | Product | Consistency0.3173 | 9 | |
| Personalized Video Generation | Games | Consistency0.6449 | 6 | |
| Personalized Video Generation | Product | Consistency0.6336 | 6 | |
| Personalized Video Generation | Short Videos | Consistency64.78 | 6 | |
| Collaborative identifier prediction | Product | Recall@101.26 | 5 | |
| Collaborative identifier prediction | Short Videos | Recall@101.55 | 5 | |
| Collaborative identifier prediction | Games | Recall@104.31 | 5 |