Evaluating ChatGPT as a Patient Education Tool: Insights on Quality, Readability, and Reliability for Trigger Finger FAQs

Abstract

Aim: Trigger finger (TF), or stenosing tenosynovitis, causes pain, snapping, and finger locking. It greatly affects patients' quality of life, prompting frequent inquiries to healthcare providers. ChatGPT, an AI language model, has gained popularity as a tool for patient education. This study evaluated the quality, readability, and usability of ChatGPT’s responses to common TF FAQs.Methods A set of FAQs regarding TF was developed based on reputable sources such as WebMD, Mayo Clinic, and NHS Trusts. Two experienced surgeons reviewed and refined the questions before submitting them to ChatGPT-4 for response generation. The quality of the responses was evaluated using the Global Quality Score (GQS) and DISCERN scale, while readability was assessed using the Flesch Reading Ease Score (FRES) and Flesch–Kincaid Grade Level (FKGL). Inter-rater reliability was determined using Cohen’s Kappa.Results: Twenty responses were evaluated, yielding a mean GQS of 3.8, indicating moderate to high quality (SD = 0.71). DISCERN scores averaged 37.08 ± 7.64, reflecting fair to good quality. Readability analysis showed a FRES score of 43.40, suggesting the content is challenging for those without a college education. The mean FKGL was 12.17, indicating advanced reading requirements. Prognosis-related questions had better readability scores than treatment-related responses, which were more complex. Conclusion: ChatGPT shows promise for patient education with moderate to high-quality responses about TF. However, advanced reading levels may limit wider accessibility. Improving readability and tailoring responses to diverse needs are vital for effectiveness. Human oversight is essential to ensure accuracy and usability of AI-generated content.

Ähnliche Arbeiten