Clinical Assessment of Fine-Tuned Open-Source LLMs in Cardiology: From Progress Notes to Discharge Summary

Abstract

The generation of accurate discharge summaries from clinical progress notes represents a critical challenge in healthcare documentation, particularly in specialized domains like cardiology where limited annotated data and complex medical terminology pose significant barriers to automation. To address this challenge and improve clinical workflow efficiency, we developed a comprehensive approach combining synthetic data generation with fine-tuned large language models (LLMs), specifically leveraging Llama3.1-8B for automated discharge summary creation. Our methodology involved constructing a hybrid dataset by combining 4658 real-world cardiology discharge summaries with 12,661 high-quality synthetic records generated via the OpenAI API and validated through a T5-based binary classifier that filtered out low-quality outputs. The fine-tuned Llama3.1-8B model demonstrated superior performance across multiple evaluation metrics including ROUGE, BLEU, and BERTScore, while qualitative assessment by three expert cardiologists confirmed the model's ability to generate clinically coherent, complete, and medically relevant discharge summaries with high accuracy in capturing patient conditions and treatment details. This research makes significant contributions to the healthcare informatics community by demonstrating the feasibility of using fine-tuned open-source LLMs for specialized clinical documentation tasks, establishing a validated framework for synthetic medical data augmentation in low-resource scenarios, and providing evidence that AI-assisted clinical documentation can achieve both technical excellence and clinical utility, thereby offering a scalable solution to reduce administrative burden on healthcare professionals while maintaining high standards of patient care documentation.

Ähnliche Arbeiten