Matching Patients to Clinical Trials with Large Language Models
About
Patient recruitment is challenging for clinical trials. We introduce TrialGPT, an end-to-end framework for zero-shot patient-to-trial matching with large language models. TrialGPT comprises three modules: it first performs large-scale filtering to retrieve candidate trials (TrialGPT-Retrieval); then predicts criterion-level patient eligibility (TrialGPT-Matching); and finally generates trial-level scores (TrialGPT-Ranking). We evaluate TrialGPT on three cohorts of 183 synthetic patients with over 75,000 trial annotations. TrialGPT-Retrieval can recall over 90% of relevant trials using less than 6% of the initial collection. Manual evaluations on 1,015 patient-criterion pairs show that TrialGPT-Matching achieves an accuracy of 87.3% with faithful explanations, close to the expert performance. The TrialGPT-Ranking scores are highly correlated with human judgments and outperform the best-competing models by 43.8% in ranking and excluding trials. Furthermore, our user study reveals that TrialGPT can reduce the screening time by 42.6% in patient recruitment. Overall, these results have demonstrated promising opportunities for patient-to-trial matching with TrialGPT.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Patient-trial pair retrieval | SIGIR treat-any 2016 original (test) | Micro Recall67.4 | 7 | |
| Patient-trial pair retrieval | SIGIR relevant-to-any 2016 original (test) | Micro Recall84.6 | 7 | |
| Clinical Trial Retrieval | SIGIR treat-chief 2016 | Micro Recall62.44 | 7 | |
| Clinical Trial Retrieval | SIGIR treat-any 2016 | Micro Recall53.66 | 7 | |
| Clinical Trial Retrieval | SIGIR relevant-to-any 2016 | Micro Recall70.27 | 7 | |
| Clinical Trial Retrieval | SIGIR patient-trial treat-any 2016 | Average Retrieved Trials2.98 | 7 | |
| Clinical Trial Retrieval | SIGIR patient-trial (relevant-to-any) 2016 | Average Retrieved Trials per Patient8.93 | 7 | |
| Clinical Trial Retrieval | SIGIR Clinical Trial Matching treat-chief 2016 | Macro Recall70.14 | 7 | |
| Clinical Trial Retrieval | SIGIR Clinical Trial Matching treat-any 2016 | Macro Recall56.5 | 7 | |
| Clinical Trial Retrieval | SIGIR Clinical Trial Matching relevant-to-any 2016 | Macro Recall69.23 | 7 |