RAGTruth

Benchmarks

Task Name	Dataset Name	SOTA Result
Hallucination Detection	RAGTruth (test)	AUROC0.9096	99
Hallucination Detection	RAGTruth	AUROC0.8535	58
Hallucination detection	RAGTruth CNN/DM (subsample)	AUROC0.69	45
Hallucination detection	RAGTruth MS MARCO (subsample)	AUROC0.77	45
Hallucination Detection	RAGTruth RT-Summ 1.0 (test)	F1 Score0.6966	30
Hallucination Detection	RAGTruth RT-D2T 1.0 (test)	F1 Score0.7383	30
Hallucination Detection	RAGTruth RT-QA 1.0 (test)	F1 Score0.7885	30
Hallucination Detection	RAGTruth Llama2-13B (test)	Acc83.33	21
Hallucination Detection	RAGTruth Llama2-7B (test)	Accuracy75.76	21
Hallucination Detection	RAGTruth LLaMA3-8B	Recall78.6	19
Hallucination Detection	RAGTruth LLaMA2-13B	Recall80.68	19
Hallucination Detection	RAGTruth LLaMA2-7B	Recall0.8328	19
Summarization	RAGTruth summarization (test)	ROUGE-152	18
Question Answering	RAGTruth	F1 Score45.89	17
Hallucination Detection	RAGTruth summarization task	Precision77	14
Response-level Hallucination Detection	RAGTruth QA	AUROC91.89	13
Span-level Hallucination Detection	RagTruth-Avg (test)	F1 Score76.63	12
Grounded Text Generation	RAGTruth	F1 Score33.14	11
Groundedness	RagTruth	Kendall's Tau0.57	11
Faithfulness detection	RAGTruth	Accuracy90.3	10
Hallucination Detection	RAGTruth	Summary Consistency Rate92.67	10
Hallucination Detection	RAGTruth Llama-13B	Recall89.47	10
Hallucination Detection	RAGTruth Llama-7B	Recall92.54	10
Token-Level Hallucination Detection	RAGTruth QA	AUROC95.6	7
Hallucination Mitigation	RAGTruth	Hallucination Rate32.7	6

Showing 25 of 33 rows