Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

VisCritic: Visual State Comparison as Process Reward for GUI Agents

About

GUI agents powered by vision-language models show strong potential for automating digital tasks, yet frequently fail in long-horizon scenarios due to the absence of step-level verification. Existing process reward models verify actions through textual reasoning alone, missing the visual nature of GUI state changes. We introduce VisCritic, a visual process reward framework that verifies agent actions by directly comparing pre-action and post-action screenshots in visual feature space. VisCritic employs a Siamese vision transformer to extract change-aware representations, coupled with an Action-Aware Critic Head that jointly evaluates action success, task progress, and error type. A critic-training data construction pipeline generates weakly supervised samples from existing trajectories without additional human labels for critic training. Experiments and offline analyses across five benchmarks demonstrate that VisCritic serves as a plug-and-play enhancement for diverse GUI agents, generally improving benchmark metrics while providing visual diagnostic cues.

Jiachen Qian• 2026

Related benchmarks

TaskDatasetResultRank
Web AgentWebArena
Success Rate29.4
56
Mobile UI ControlAndroidWorld
Overall Task Success Rate29.8
45
Web navigationMind2Web--
41
GUI Agent InteractionOSWorld--
16
Offline Action Consistency DiagnosticAITW
Action Accuracy74.2
15
Action Success PredictionAndroidWorld (smallest held-out split)
Precision87.1
3
Showing 6 of 6 rows

Other info

Follow for update