Share your thoughts, 1 month free Claude Pro on usSee more

Scene Graph Generation on PSG v1.0 (test)

0Failure Rate

SceneGraphVLM

Updated 20d ago

Evaluation Results

Method
SceneGraphVLM 2026.05	0	0.643	63.2	64.2	58.9	59.8	60.2	61.1	27.2	30.4	19.4	21.9	21.3	23.9	40.8	42.5
SceneGraphVLM 2026.05	0	0.643	67.9	69	60.1	61.1	63	64	33.8	37.2	21.7	24.1	25.3	28	44.2	46
Gemini-3-flash 2026.05	0.09	4.671	31.2	49.7	30.3	50.9	29.2	47.7	11.6	15.6	12.8	17.4	11.7	15.8	20.4	31.8
R1-SGG 2026.05	0.14	8.564	47.6	61	38	57.4	41	59.2	10.5	11.7	36.4	40.5	15.8	17.6	28.4	38.6
GPT-5-mini 2026.05	0.32	26.65	7.5	15.5	9.7	20.3	8	16.6	2	2.6	3.7	4.9	2.4	3.2	5.2	9.9
InternVL3.5-1B-Instruct (full finetune LLM head) 2026.05	0.37	1.115	34.1	35	33.7	34.6	33.5	34.3	13.4	15.1	8.5	9.7	9.8	11.1	21.6	22.7
GPT-5.4-nano 2026.05	0.64	2.456	7.5	14.8	5.9	12.4	6.2	12.6	1.3	1.7	0.7	0.9	0.8	1.1	3.5	6.9
DeepSeek-VL2-tiny-3B 2026.05	0.64	3.747	5.8	7.1	1.7	2.1	2.5	3	0	0	0	0	0	0	1.2	1.5
GPT-5.4-mini 2026.05	0.74	2.213	23.4	32.5	16.7	24.4	18.6	26.5	5.8	7.4	3.2	4.2	3.8	5	11.2	15.7
GPT-5-nano 2026.05	0.78	32.639	7.8	13.8	6.3	11.6	6.5	11.7	1.1	1.5	0.7	0.9	0.8	1.1	3.6	6.4
InternVL3-1B (full finetune LLM head) 2026.05	0.78	0.894	36	36.8	32.1	32.8	33.4	34.2	13.5	15	8	9	9.5	10.7	21.5	22.4
SceneGraphVLM (InternVL2.5-1B) 2026.05	1.15	0.704	61	62.4	44.6	45.6	50.6	51.7	22.2	27.5	11.4	14.5	14.3	18.2	32.5	34.9
InternVL3-1B (full finetune LLM head) 2026.05	1.24	0.916	33.9	34.8	31.3	32.1	32.1	32.9	13.1	14.5	7.1	8	8.7	9.8	20.4	21.3
SceneGraphVLM (InternVL2.5-1B) 2026.05	3.45	0.704	62.9	64.1	43.8	44.6	50.6	51.6	23	27.7	11.1	13.7	14.2	17.4	32.4	34.5
InternVL2.5-1B 2026.05	6.85	1.169	3.3	4.4	1	1.4	1.4	2	0	0	0	0	0	0	0.7	1
Qwen3.5-4B 2026.05	7.58	3.042	39.9	47.4	21.7	27.1	26.5	32.3	8	10.3	3.7	4.7	4.5	5.8	15.5	19
InternVL3.5-1B-Instruct (LoRA r=16, α=32) 2026.05	10.34	1.003	24.9	25.5	21.8	22.3	22.7	23.3	8	9.3	4	4.7	5	5.9	13.9	14.6
InternVL3-1B 2026.05	11.4	2.669	10.5	12.4	3.4	4.1	4.7	5.7	0.3	0.5	0.1	0.1	0.1	0.2	2.4	3
Qwen3.5-2B 2026.05	14.61	1.64	23	31.9	18.7	27.2	19.4	27.6	0.4	0.5	0.1	0.2	0.2	0.3	9.8	13.9
InternVL3.5-1B 2026.05	23.81	2.981	10.5	12.4	3.4	4.1	8.4	15.3	0.7	1.4	0.4	0.8	0.5	1	4.5	8.1
Qwen3-VL-4B 2026.05	45.37	7.778	36.2	37.9	7.8	8.6	12	12.9	0.7	0.9	0.2	0.3	0.3	0.4	6.1	6.7
Qwen3-VL-2B 2026.05	50.09	7.46	25.7	27.7	11	12.5	14.2	15.8	2	3.3	0.5	0.9	0.8	1.3	7.5	8.5
Qwen3.5-0.8B 2026.05	67.92	1.074	14	16.4	4.5	5.5	6.3	7.5	0	0.1	0	0	0	0	3.2	3.8
Ovis2.5-2B-Instruct 2026.05	75.97	5.738	7	10.4	4.9	7.8	5.4	8.4	1.3	1.8	0.4	0.7	0.6	0.9	3	4.7
SmolVLM-500M-Instruct 2026.05	86.08	2.871	0.3	0.3	0.1	0.1	0.1	0.1	0	0	0	0	0	0	0.1	0.1