Leveraging large language models to populate structured clinical case report forms from unstructured medical notes in radiation oncology

Abstract

<h2>Abstract</h2><h3>Background and purpose</h3> Large language models (LLMs) have shown growing potential for clinical text processing, but their systematic application in radiation oncology—especially for non-English clinical documentation—remains underexplored. This study investigated whether pretrained LLMs can automatically extract, analyze, and structure radiotherapy-relevant information from routine unstructured medical notes, with the goal of supporting automated population of electronic case report forms (eCRFs). <h3>Materials and methods</h3> This study examined prostate cancer patients treated with the MR-Linac, for whom ground truth data exist in the MOMENTUM database. A total of 100 patients were included, with 90 used for prompt development and 10 for independent testing. Medical notes were extracted, anonymized, and categorized by time points. The Llama-3.1-8b model was used, with prompts designed using chain-of-thought (CoT) logic with five in-context examples. The model output was post-processed, and extracted data was compared against ground truth. <h3>Results</h3> Medical notes were successfully processed, with predicted values generated in an average time of 16 s per note. The LLM achieved matching accuracies of 83.6% and 83.8% on the development and testing datasets. Analysis revealed that the model disagreed with specific values in 8.1% of development dataset cases and 8.6% of testing dataset cases. An independent manual review before model evaluation showed approximately 7.5% of routinely collected test data did not match reviewed values, indicating inaccuracies in the routinely acquired ground truth. <h3>Conclusion</h3> This study demonstrated the effectiveness of LLMs in structuring clinical data from medical non-English notes, with high accuracy in extracting and categorizing information. While multi-institutional validation is needed, the results indicate a significant healthcare impact through efficient data management, processing notes in 16 s, and accurately populating CRFs with minimal staff involvement.

Ähnliche Arbeiten