Performance evaluation of an AI-powered system for clinical trial eligibility using mCODE data standards.

Abstract

e13621 Background: Automated data capture in oncology can expedite clinical trial enrollment by relieving the burdens of manual chart abstraction and ensuring consistent data standards. We developed and evaluated a GPT-4o–based AI platform from Massive Bio that automatically extracts and maps unstructured EHR data to the Minimal Common Oncology Data Elements (mCODE) 3.0 framework, facilitating broader and faster patient recruitment, in a real world dataset under ASCO's principles for mCODE. Methods: We developed a fine-tuned AI pipeline that ingests unstructured EHR data free-text oncology notes and structured fields for a cohort of 102 randomly selected patients drawn from a 3,800-patient database. The pipeline mapped tumor type, cancer stage, extent of disease, relapse status, resectability, and NGS/IHC extracted information to mCODE 3.0 profiles (Cancer Disease Status, Tumor Profiles, and Genomic Profiles). Human chart review served as the gold standard (i.e. ground truth), enabling calculation of F1 scores for each extracted data category. Key endpoints included accuracy for tumor type, stage, extent, resectability, relapse status, and NGS/IHC biomarker data. We also validated the system’s alignment with mCODE’s required elements, assessing both completeness and interoperability of the extracted dataset. Results: The model exhibited strong performance for tumor type (98% accuracy) and extent of disease (90%), accurately differentiating localized versus metastatic presentations. Tumor stage reached 86%, with minor discrepancies in sub-stage details. Relapse status (77%) and resectability (69%) were somewhat lower due to incomplete surgical documentation and variations in clinician definitions. Genomic data extraction (78%) reliably captured well-known variants (e.g., BRCA1/2, TP53) but showed reduced consistency for complex, multi-variant panels. Notably, patients' demographic data elements achieved 100% concordance. Successful mapping to multiple mCODE profiles demonstrated robust interoperability, highlighting the potential for accelerated trial screening with minimal manual intervention. Conclusions: By leveraging oncology tailored, fine-tuned GPT-4o frontier model capabilities and structured mCODE schemas, this AI-based solution substantially improves the speed and accuracy of clinical trial eligibility determinations. The system’s performance underscores the feasibility of automated oncology data standardization, which can enhance real-world evidence generation, reduce labor costs, and expand patient access to novel treatments. Ongoing refinements will focus on capturing nuanced surgical decisions, mitigating missing data, and refining multi-gene variant interpretations, further driving the promise of precision oncology through streamlined, interoperable data exchange.

Ähnliche Arbeiten