Share your thoughts, 1 month free Claude Pro on usSee more

Theory of Mind reasoning on MMToM-QA Text-only

1Belief Inference 1.1

SymbolicToM

Updated 22d ago

Evaluation Results

Method	Links
SymbolicToM 2024.01		1	0.61	0.74	0.783	0.733	0.667	0	0.507	0.477	0.63
SymbolicToM w/ GPT-4 2025.06		1	0.61	0.74	0.783	0.733	0.667	0	0.507	0.477	0.63
GPT-4 2024.01		0.97	0.12	0.77	0.62	0.48	0.427	0.027	0.427	0.34	0.48
GPT-4 2025.06		0.97	0.12	0.77	0.62	0.48	0.427	0.027	0.427	0.34	0.48
Human 2024.01		0.96	0.958	0.813	0.91	0.858	0.767	0.65	0.683	0.74	0.825
SimToM 2024.01		0.96	0.15	0.82	0.643	0.613	0.44	0.027	0.547	0.407	0.525
Human 2025.06		0.96	0.958	0.813	0.91	0.858	0.767	0.65	0.683	0.74	0.825
SimToM w/ GPT-4 2025.06		0.96	0.15	0.82	0.643	0.613	0.44	0.027	0.547	0.407	0.525
DeepSeek-R1-671B 2025.06		0.92	0.502	0.724	0.715	0.683	0.44	0.45	0.482	0.514	0.615
Scalable Bayesian ToM Planner 2025.06		0.901	0.705	0.874	0.827	0.688	0.755	0.753	0.718	0.729	0.778
BIP-ALM 2024.01		0.89	0.68	0.9	0.823	0.547	0.667	0.507	0.627	0.587	0.705
BIP-ALM 2024.01		0.88	0.69	0.88	0.817	0.773	0.68	0.307	0.707	0.617	0.717
BIPALM w/ GPT-J-6B 2025.06		0.88	0.69	0.88	0.817	0.773	0.68	0.307	0.707	0.617	0.717
OpenAI-o3-mini 2025.06		0.831	0.476	0.623	0.643	0.64	0.386	0.387	0.46	0.468	0.556
GPT-3.5 2024.01		0.81	0.11	0.39	0.437	0.467	0.16	0.213	0.48	0.33	0.383
LLAMA 2 2024.01		0.64	0.55	0.5	0.563	0.493	0.48	0.413	0.387	0.443	0.503
GPT-J 2024.01		0.56	0.53	0.38	0.49	0.52	0.507	0.507	0.56	0.523	0.507