Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Learning to Solve, Forgetting to Retain: Correct-Set Turnover in RLVR

About

Reinforcement learning with verifiable rewards (RLVR) improves the ability of large language model, yet headline accuracy gains often conceal a hidden cost: previously solved problems quietly become unsolvable as training proceeds. We frame this phenomenon as \emph{correct-set turnover}, representing the coupled dynamics of solution acquisition and regression over the mastered set. Under this view, retention becomes an explicit optimization target alongside acquisition. We analytically and empirically establish the \emph{repair-window principle}: the cost of restoring a regressed prompt grows sharply with review delay, defining a low-cost window that standard RLVR pipelines fail to exploit. To address this, we propose \textbf{\method{}}, a retention-aware review mechanism that tracks mastered prompts and periodically reintroduces them to \textbf{remind} the model of previous solutions. By utilizing pre-rollout batch replacement, \method{} incurs zero additional rollout overhead. Evaluated across 20 benchmarks spanning image-text, video, and text-only tasks with Qwen3-VL and Qwen2.5-Math, \method{} consistently improves performance over GRPO, DAPO, and replay baselines, demonstrating robust generalizability across modalities and algorithms.

Chuanyu Qin, Chenxu Yang, Qingyi Si, Naibin Gu, Peng Fu, Zheng Lin• 2026

Related benchmarks

TaskDatasetResultRank
Image-Text UnderstandingMMBench
Accuracy91.22
40
Multimodal ReasoningZeroBench
Accuracy26.05
22
Image ReasoningMathVision
Accuracy57.32
22
Vision-Language ReasoningMMStar
Accuracy74.7
14
Image-Text ReasoningMathVista
Accuracy79.85
8
Image-Text ReasoningWeMath
Accuracy68.29
8
Image-Text ReasoningMathVerse
Accuracy61.52
8
Image-Text ReasoningMMMU-P
Accuracy58.78
8
Showing 8 of 8 rows

Other info

Follow for update