Share your thoughts, 1 month free Claude Pro on usSee more

Mental State Inference on MMToM-QA human 1.0 (test)

100Sub-score 1.1

GPT-4

Updated 3mo ago

Evaluation Results

Method	Links
GPT-4 2024.01		100	50.9	65.3	36.7	31.5	64	40	40	13.3	53.3
SimToM 2024.01		100	56.7	70	43.4	31.6	80	46.7	40	20	66.7
SymbolicToM 2024.01		100	53.3	70	36.7	78.9	44	40	73.3	0	33.3
BIP-ALM 2024.01		93.8	76.6	78.1	75	52.6	88	73.3	86.7	73.3	66.7
GPT-3.5 2024.01		93.8	45.7	58.1	33.3	52.6	28	53.3	0	33.3	46.7
GPT-4V 2024.01		93.4	48.9	52.7	45	36.8	28	33.3	73.3	13.3	60
BIP-ALM 2024.01		87.5	77	85.7	68.3	73.7	96	60	86.7	60	66.7
LLaMA 2 2024.01		75	53.6	57.2	50	52.6	44	46.7	40	60	53.3
BIP-ALM 2024.01		75	62.6	71.9	53.3	52.6	88	40	73.3	33.3	66.7
BIP-ALM 2024.01		75	67.9	75.9	60	52.6	100	66.7	80	53.3	40
BIP-ALM 2024.01		75	63.1	69.6	56.7	57.9	76	60	53.3	60	53.3
GPT-J 2024.01		62.5	45.6	49.5	41.7	42.1	44	33.3	53.3	26.7	53.3
BIP-ALM 2024.01		62.5	59.2	60.1	58.3	57.9	60	46.7	53.3	73.3	60
GPT-4V 2024.01		56.3	44.8	41.2	48.3	47.4	20	33.3	40	60	60
Video-LLaMA 2 2024.01		50	36.9	40.5	33.3	47.4	24	33.3	33.3	20	46.7
InstructBLIP 2024.01		43.8	49.1	49.9	48.3	57.9	48	53.3	46.7	40	53.3
InstructBLIP 2024.01		43.8	49.1	48.1	50	52.6	48	46.7	46.7	46.7	60
Video-LLaMA 2 2024.01		31.3	45.2	42	48.3	26.8	68	53.3	60	40	40
LLaVA 2024.01		24	50.9	60.1	41.7	56.3	100	40	33.3	40	53.3
LLaVA 2024.01		4	39.6	40.9	38.3	18.8	100	46.7	26.7	40	40