AMBER

Benchmarks

Task Name	Dataset Name	SOTA Result
Hallucination Evaluation	AMBER	CHAIR24.5	267
Generative Hallucination	AMBER Generative	Coverage (%)70.4	81
Hallucination Assessment	AMBER	CHAIR_s10.6	56
Hallucination Assessment	AMBER (test)	CHAIR5.6	44
Discriminative Hallucination Evaluation	AMBER (test)	Accuracy86.8	42
Object Hallucination Mitigation on Generative Tasks	AMBER	CHAIR12.1	38
Object Hallucination Assessment	AMBER	CHAIR_I16.2	35
Hallucination Detection	AMBER sampled 5k	A-ROC85.99	30
Hallucination Evaluation (Generative)	AMBER-g	CHAIR Score2.2	29
Multi-modal Hallucination Evaluation	AMBER	CHAIR9.2	28
Hallucination Evaluation	AMBER Generative Task	Coverage67.1	26
Action-relation hallucination evaluation	AMBER Relation	Accuracy81.25	25
Fine-grained hallucination evaluation	AMBER	CHAIR8	24
Discriminative Hallucination Evaluation	AMBER-d	F1 Score89.5	23
Discriminative Object Hallucination	AMBER Discriminative Task	F1 Score87.4	22
Generative Hallucination	AMBER generative subset	CHAIR10.9	22
Visual Hallucination Evaluation	AMBER Discriminative	Accuracy89	18
Visual Hallucination Evaluation	AMBER Generative	CHAIRi6.3	18
Generative Hallucination Evaluation	AMBER (test)	CHAIR Score7.9	18
Discriminative Hallucination Evaluation	AMBER	Accuracy84.3	18
Watermarking	AMBER	AUC99.99	18
Generative Hallucination Evaluation	AMBER	Score90.79	14
Multimodal Watermarking	AMBER	PPL2.98	14
Hallucination Evaluation (Discriminative)	AMBER-d	Accuracy89.2	12
Object Hallucination Evaluation	AMBER	Hallucination Score (Hal)42.3	11

Showing 25 of 36 rows