Analysis of Eligibility Criteria Clusters Based on Large Language Models for Clinical Trial Design

Abstract

ABSTRACT Objectives Clinical trials (CTs) are essential for improving patient care by evaluating new treatments’ safety and efficacy. A key component in CT protocols is the study population defined by the eligibility criteria. This study aims to evaluate the effectiveness of large language models (LLMs) in encoding eligibility criterion information to support CT protocol design. Materials and Methods We extracted eligibility criterion sections, phases, conditions, and interventions from CT protocols available in the ClinicalTrials.gov registry. Eligibility sections were split into individual rules using a criterion tokenizer and embedded using LLMs. The obtained representations were clustered. The quality and relevance of the clusters for protocol design was evaluated through 3 experiments: intrinsic alignment with protocol information and human expert cluster coherence assessment, extrinsic evaluation through CT-level classification tasks, and eligibility section generation. Results Sentence embeddings fine-tuned using biomedical corpora produce clusters with the highest alignment to CT-level information. Human expert evaluation confirms that clusters are well-structured and coherent. Despite the high information compression, clusters retain significant CT information, up to 97% of the classification performance obtained with raw embeddings. Finally, eligibility sections automatically generated using clusters achieve 95% of the ROUGE scores obtained with a generative LLM. Conclusions We show that clusters derived from sentence-level LLM embeddings are effective in summarizing complex eligibility criterion data while retaining relevant CT protocol details. Clustering-based approaches provide a scalable enhancement in CT design that balances information compression with accuracy.

Ähnliche Arbeiten