Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Food-R1: A Unified Multi-Task Food Vision-Language Model with Reinforcement Learning

About

Recent studies have explored Vision-Language Models (VLMs) for food analysis. However, most existing methods rely primarily on supervised fine-tuning (SFT), which often limits reasoning and generalization capabilities. Moreover, high-quality large-scale nutritional annotations remain scarce. To address these issues, we introduce CalorieBench-80K, a large-scale benchmark with curated calorie labels and dietary advice annotations. To the best of our knowledge, it is the first food image benchmark to incorporate Chain-of-Thought (CoT) annotations for calorie reasoning. We also propose Food-R1, a unified food VLM trained in a multi-task learning paradigm to equip the model with broad capabilities. Food-R1 undergoes CoT-based cold-start instruction tuning, followed by reinforcement fine-tuning (RFT) using Group Relative Policy Optimization (GRPO) to improve reasoning and performance. Experiments on CalorieBench-80K and representative benchmarks show that Food-R1 consistently outperforms strong baselines across food-related tasks. The code, model weights, and benchmark annotations are available at the project repository.

Yu Zhu, Yongkang Li, Wenjie Zhu, Haoyi Jiang, Wenyu Liu, Wei Yang, Bin Li, Xinggang Wang• 2026

Related benchmarks

TaskDatasetResultRank
ClassificationFood101
Top-1 Accuracy95.21
83
Recipe GenerationRecipe1M
SacreBLEU7.76
12
Calorie estimationCalorieBench-80K (test)
MAE42.56
9
Dietary advice generationCalorieBench 80K
ROUGE-10.65
9
Ingredient RecognitionVireoFood-172
F1 Score84.61
4
Nutrition EstimationNutrition5k (test)
Calorie pMAE14.4
3
Referring Nutrition EstimationNutrition5k refer@3rd 1.0 (test)
Calorie MAE13.3
2
Referring Nutrition EstimationNutrition5k refer@1st 1.0 (test)
Calorie MAE28.2
2
Referring Nutrition EstimationNutrition5k refer@2nd 1.0 (test)
Calorie MAE24.4
2
Showing 9 of 9 rows

Other info

Follow for update