ChatDoctor

Benchmarks

Task Name	Dataset Name	SOTA Result
Untargeted Reconstruction Attack	ChatDoctor 250 adversarial prompts	Repeat Prompt40	42
Answer Accuracy	ChatDoctor	BRT Accuracy40.8	26
Retrieval-based Question Answering	ChatDoctor	ROUGE-L13.6	23
Privacy Leakage and Utility Assessment	ChatDoctor iCliniq	Retrieved Context Count750	10
Tracing	ChatDoctor (test)	TSR99.75	10
Instruction-following	ChatDoctor	BRT46.9	9
Factual Consistency Evaluation	ChatDoctor	FC Score4.08	7
Medical Question Answering	ChatDoctor	Task Performance64.89	7
Preprocessing efficiency	ChatDoctor	Preprocessing Time (s/sample)13.52	6
Property Inference Defense	ChatDoctor QA	Generation Attack MAE0.13	6
Property Inference Defense	ChatDoctor CC	Generation Attack MAE0.0354	6
QA	ChatDoctor	MAE0.0396	5
CC	ChatDoctor	MAE0.0304	5
Privacy-Utility Trade-off Evaluation	ChatDoctor worst-case attack setting	CRR97.4	5
Knowledge Base Extraction	ChatDoctor RAG-Thief attack	CRR73.4	5
Knowledge Base Extraction	ChatDoctor Pirates attack	CRR97.4	5
Monitor Accuracy	ChatDoctor	Monitor Accuracy96	4
Privacy extraction attack	ChatDoctor	Privacy Leakage Rate17	4
Backdoor Detection	ChatDoctor	F1-score0.0058	3

Showing 19 of 19 rows