Assessing the Clinical Support Capabilities of ChatGPT-4o and ChatGPT-4o Mini in Managing Lumbar Disc Herniation

Abstract

<title>Abstract</title> <bold>Purpose:</bold> This study evaluated and compared the clinical support capabilities of ChatGPT-4.0 and ChatGPT-4.0-mini in diagnosing and treating lumbar disc herniation (LDH) with radiculopathy. <bold>Methods:</bold> Twenty-one questions (across 5 categories) from NASS Clinical Guidelines were input into ChatGPT-4.0 and ChatGPT-4.0-mini. Five orthopedic surgeons assessed their responses using a 5-point Likert scale for accuracy and completeness, and a 7-point scale for safety. Flesch Reading Ease scores were calculated to assess readability. Additionally, ChatGPT-4.0 analyzed lumbar images from 53 patients, comparing its recognizable agreement with orthopedic surgeons using Kappa values. <bold>Results:</bold> Both models demonstrated strong clinical support capabilities with no significant differences in accuracy or safety. However, ChatGPT-4.0 provided more comprehensive and consistent responses. The Flesch Reading Ease scores for both models indicated that their generated content was “very difficult to read,” potentially limiting patient accessibility. In evaluating lumbar disc herniation images, ChatGPT-4.0 achieved an overall accuracy of 0.81, with LDH recognition precision, recall, and F1 scores exceeding 0.80. The AUC was 0.80, and the Kappa value was 0.61, indicating moderate agreement between the model's predictions and actual diagnoses, though with room for improvement. <bold>Conclusion: </bold>While both models are effective, ChatGPT-4.0 offers more comprehensive clinical responses, making it more suitable for high-integrity medical tasks. However, the difficulty in reading AI-generated content and occasional use of misleading terms, such as “tumor,” indicate a need for further improvements to reduce patient anxiety.

Ähnliche Arbeiten