Share your thoughts, 1 month free Claude Pro on usSee more

Multimodal Question Answering on SUPERGLASSES 1.0 (Leaderboard)

49.68Accuracy (Easy)

SUPERLENS‡ (Ours)

Updated 9d ago

Evaluation Results

Method
SUPERLENS‡ (Ours) 2026.02	49.68	34.72	30.17	48.76	34.19	55.78	34.7	40.33	29.15	44.1	1
GPT-4o 2026.02	45.1	36.81	32.96	44.87	35.61	49.9	34.19	38.37	34.42	41.91	3
Gemini 2.5 Pro 2026.02	45.1	39.94	36.31	45.36	38.06	48.92	37.02	41.95	35.43	43.02	2
Qwen2.5-VL-72B 2026.02	41.54	31.89	25.14	41.29	29.94	50.1	24.94	33.5	24.62	37.65	4
SUPERLENS† (Ours) 2026.02	40.84	31.74	26.26	40.01	31.35	42.65	29.05	39.19	28.39	37.24	5
Qwen2.5-VL-32B 2026.02	40.52	30.4	24.58	40.5	28.13	49.41	24.16	32.2	22.36	36.54	7
Claude 4 Sonnet 2026.02	40.14	32.34	29.05	40.32	30.45	48.63	22.88	35.12	24.87	37.16	6
Qwen2.5-VL-7B 2026.02	37.72	24.59	20.67	36.92	24.13	45.59	25.71	24.23	20.35	32.82	8
LLaMA-3.2-90B 2026.02	34.99	25.93	22.91	34.55	25.29	39.9	23.39	28.78	22.61	31.59	9
InternVL3-8B 2026.02	31.17	19.23	17.88	29.99	20.26	40.2	12.08	21.95	14.82	26.88	10
Qwen2.5-VL-3B 2026.02	30.83	20.68	18.53	29.03	21.68	36.37	17.22	21.63	12.06	25.56	12
DeepSeek-VL2-27B 2026.02	29.9	19.08	12.85	28.48	19.61	37.25	14.4	19.84	15.83	25.64	11
MiMo-VL-7B 2026.02	29.71	18.03	10.06	28.96	16.65	36.08	16.97	20	12.31	25.02	12
DeepSeek-VL2-16B 2026.02	28.75	16.1	10.61	27.5	16.26	35.88	15.42	16.75	12.56	23.91	13
GLM-4.1V-9B 2026.02	27.35	17.88	15.08	26.53	18.06	33.82	14.14	19.02	15.08	23.82	14
LLaMA-3.2-11B 2026.02	27.16	17.44	14.53	26.35	17.55	33.14	13.37	19.84	14.57	23.53	15
LLaVA-Onevision-7B 2026.02	25	13.41	13.97	23.92	14.71	31.27	10.54	17.24	10.55	20.97	16
DeepSeek-VL2-3B 2026.02	24.11	12.22	9.5	23.13	12.52	30.78	10.8	14.15	8.79	19.74	17
Qwen2.5-VL-7B 2026.02	22.58	14.01	12.29	22.71	12.52	28.73	11.83	14.8	10.3	19.45	18
Qwen2.5-VL-7B 2026.02	21.12	9.69	10.61	20.64	9.81	23.43	10.03	17.56	7.54	17.18	20
Phi-3-Vision-4B 2026.02	20.74	11.18	10.06	19.98	11.61	26.18	7.2	13.98	9.55	17.3	19
LLaMA-3.2-11B 2026.02	19.97	12.97	7.82	19.06	13.03	24.41	9	14.8	10.05	17.13	21
Qwen2.5-VL-7B 2026.02	19.78	10.28	9.5	19.25	10.32	20.59	10.8	18.54	7.79	16.39	22
LLaMA-3.2-11B 2026.02	17.56	9.39	10.61	17.06	9.94	19.51	6.94	16.59	7.54	14.78	23
LLaVA-Onevision-0.5B 2026.02	16.79	8.49	6.15	15.79	9.29	21.08	8.23	10.41	5.28	13.71	24
LLaMA-3.2-11B 2026.02	15.9	8.79	10.61	15.3	9.81	16.57	6.43	17.07	7.29	13.54	25
LLaVA-v1.5-13B 2026.02	13.93	7.15	9.5	13.36	8.26	16.96	4.88	10.89	6.28	11.73	26
LLaVA-v1.5-7B 2026.02	12.4	6.71	7.26	11.35	8.52	15	4.63	9.76	5.53	10.45	27