Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Matching Patients to Clinical Trials with Large Language Models

About

Patient recruitment is challenging for clinical trials. We introduce TrialGPT, an end-to-end framework for zero-shot patient-to-trial matching with large language models. TrialGPT comprises three modules: it first performs large-scale filtering to retrieve candidate trials (TrialGPT-Retrieval); then predicts criterion-level patient eligibility (TrialGPT-Matching); and finally generates trial-level scores (TrialGPT-Ranking). We evaluate TrialGPT on three cohorts of 183 synthetic patients with over 75,000 trial annotations. TrialGPT-Retrieval can recall over 90% of relevant trials using less than 6% of the initial collection. Manual evaluations on 1,015 patient-criterion pairs show that TrialGPT-Matching achieves an accuracy of 87.3% with faithful explanations, close to the expert performance. The TrialGPT-Ranking scores are highly correlated with human judgments and outperform the best-competing models by 43.8% in ranking and excluding trials. Furthermore, our user study reveals that TrialGPT can reduce the screening time by 42.6% in patient recruitment. Overall, these results have demonstrated promising opportunities for patient-to-trial matching with TrialGPT.

Qiao Jin, Zifeng Wang, Charalampos S. Floudas, Fangyuan Chen, Changlin Gong, Dara Bracken-Clarke, Elisabetta Xue, Yifan Yang, Jimeng Sun, Zhiyong Lu• 2023

Related benchmarks

TaskDatasetResultRank
Patient-trial pair retrievalSIGIR treat-any 2016 original (test)
Micro Recall67.4
7
Patient-trial pair retrievalSIGIR relevant-to-any 2016 original (test)
Micro Recall84.6
7
Clinical Trial RetrievalSIGIR treat-chief 2016
Micro Recall62.44
7
Clinical Trial RetrievalSIGIR treat-any 2016
Micro Recall53.66
7
Clinical Trial RetrievalSIGIR relevant-to-any 2016
Micro Recall70.27
7
Clinical Trial RetrievalSIGIR patient-trial treat-any 2016
Average Retrieved Trials2.98
7
Clinical Trial RetrievalSIGIR patient-trial (relevant-to-any) 2016
Average Retrieved Trials per Patient8.93
7
Clinical Trial RetrievalSIGIR Clinical Trial Matching treat-chief 2016
Macro Recall70.14
7
Clinical Trial RetrievalSIGIR Clinical Trial Matching treat-any 2016
Macro Recall56.5
7
Clinical Trial RetrievalSIGIR Clinical Trial Matching relevant-to-any 2016
Macro Recall69.23
7
Showing 10 of 16 rows

Other info

Follow for update