3D Question Answering

Benchmarks

Dataset Name	SOTA Method	Metric
ScanQA (val)	PolyV	METEOR23.1	217	11d ago
SQA3D (test)	PolyV	EM@164.8	98	18d ago
SQA3D	Cog3DMap-8B	EM61.3	69	11d ago
ScanQA w/ objects (test)	BridgeQA	EM@131.29	55	1mo ago
ScanQA w/o objects (test)	BridgeQA	EM@130.87	51	1mo ago
ScanQA	3DRS	EM (Exact Match)30.3	38	11d ago
VSI-Bench	Loc3R-VLM	Average Score63.2	37	15d ago
ScanQA v1.0 (test)	Grounded 3D-LLM	ROUGE72.7	26	1mo ago
MSQA	SCENECOT	Count Accuracy47.9	25	5d ago
Beacon3D	MAG-3D	Case Score65	23	5d ago
SQA3D	GPT4Scene	Exact Match (EM)60.6	21	12d ago
ScanQA (test)	NaviLLM	BLEU-413.9	20	1mo ago
NuscenesQA v1.0 (test)	MSMDFusion + MCAN	Existence Accuracy (All)85.4	19	1mo ago
ScanQA v1.0 (val)	Efficient-VLN	BLEU-416.2	13	1mo ago
OpenEQA	LLaVA-OV-7B	LLM-Match56.2	12	17d ago
ObjaXL-LVIS 1K (test)	GPT4Point (FLANT5XL)	Accuracy27.6	8	1mo ago
SQA3D v1.0 (test)	PAVE-7B (w/ 3D info)	EM@159	8	1mo ago
Beacon3D ScanNet (test)	Loc3R-VLM	Class Accuracy44.8	7	1mo ago
MMScan QA	LLaVA-3D	Overall Score55.4	7	1mo ago
Real-3DQA	GPT4Scene	EM33.1	5	23d ago
City-3DQA (City-wise)	3DCity-LLM	Accuracy (Single-hop)79.1	5	24d ago
City-3DQA Sentence-wise	3DCity-LLM	Accuracy (Single-hop)82.41	5	24d ago
ScanQA Leaderboard as of Nov 2023 (test)		EM30.82	5	1mo ago
MSQA Vision-only	MAG-3DSeed-1.6	Count Accuracy23.6	4	5d ago
Open-EQA HM3D	PolyV	LLM-Match Score63.4	3	1mo ago

Showing 25 of 27 rows