Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

UserHarness: Harnessing User Minds for Stronger Agent Theory-of-Mind

About

Understanding what a user believes and intends is central to building effective agent assistants. This ability is often evaluated through Theory-of-Mind (ToM) tasks, where success requires reasoning from the user's perspective. However, many existing approaches address ToM with complex pipelines that model behavior indirectly, without explicitly reconstructing the user's mental state. This misses the core structure of the problem: users act based on their beliefs, which are updated through observations of the environment; beliefs and intentions jointly determine actions, which in turn change the environment; and social reasoning often requires nested beliefs about what others believe or intend. We propose UserHarness, a simple framework that reframes ToM reasoning as explicit user-mind reconstruction. UserHarness decomposes the user's mental state, its relation to the external environment, and the actions that follow from it, enabling agents to track what the user observes, believes, intends, and does. Across five benchmarks, UserHarness reaches up to 95.94% macro accuracy, improving over existing inference methods by more than 15% relative and over the strongest prompt-only harness by about 20% relative. These results suggest that robust user understanding requires reasoning from the roots of the user's mind, positioning user harnessing as a promising foundation for more adaptive future assistants.

Cheng Qian, Jiayu Liu, Heng Ji• 2026

Related benchmarks

TaskDatasetResultRank
Theory of MindHiToM
Accuracy87.08
64
Theory of MindToMi
Accuracy100
55
Theory of MindBigToM
Accuracy98.67
48
Theory of Mind reasoningMMToM-QA
Overall Accuracy98.5
44
Theory of Mind reasoningMuMa-ToM
Accuracy95.89
40
Showing 5 of 5 rows

Other info

Follow for update