Hybrid Student-Teacher Large Language Model Refinement for Cancer Toxicity Symptom Extraction

Abstract

Large Language Models (LLMs) offer significant potential for clinical symptom extraction, but their deployment in healthcare settings is constrained by privacy concerns, computational limitations, and operational costs. This study investigates the optimization of compact LLMs for cancer toxicity symptom extraction using a novel iterative refinement approach. We employ a student-teacher architecture, where the teacher model, GPT-4o, dynamically selects the most effective strategy for the student models (Zephyr-7b-beta and Phi3-mini-128) between prompt refinement, Retrieval-Augmented Generation (RAG), and fine-tuning. Our experiments on 294 clinical notes covering 12 post-radiotherapy toxicity symptoms demonstrate the effectiveness of this approach. Using 5-fold cross-validation, we observed significant improvements in F1 scores across all symptoms. The Phi3 model showed an average F1 score increase of 26%, while Zephyr achieved a 13% improvement. Notably, these enhancements were achieved at substantially lower costs, with Phi-3 being 48 times cheaper and Zephyr 30 times cheaper than GPT-4o. These results highlight the potential of iterative refinement techniques to enhance the capabilities of compact LLMs for clinical applications, offering a balance between performance, cost-effectiveness, and privacy preservation in healthcare settings.

Ähnliche Arbeiten