29. The Free Basic Version of ChatGPT-3.5 Produces More Readable Cosmetic Surgery Patient Educational Materials than Subscription-based Advanced ChatGPT-4.0

Abstract

BACKGROUND: Health literacy is crucial in plastic surgery, as patients must navigate complex medical decisions related to both elective and reconstructive procedures. One effective way to enhance health literacy is by providing patient education materials that are accessible at a sixth-grade reading level. Many online resources, however, exceed this level. With artificial intelligence (AI), like ChatGPT, we can generate patient education materials that align closely with the reading level of the average American. However, with multiple versions of ChatGPT available—such as the subscription-based advanced model (4.0) and the free basic model (3.5)—it remains unclear which version offers the most comprehensible, accurate, and accessible information. This study aims to compare the readability and detail of patient information sheets produced by both versions of ChatGPT to determine which can most effectively support health literacy for plastic surgery patients. METHODS: We used 98 patient information sheets on various cosmetic procedures from the ASPS, ABCS, and AACS websites, and generated brochures for the same procedures with ChatGPT-3.5 and ChatGPT-4.0 at a sixth-grade reading level using identical prompts, labeling these the “Chat-Generated” groups. Additionally, we input the original physician-written information from these online sheets into ChatGPT-3.5 and ChatGPT-4.0 to create more accurate brochures using the same prompts, labeling these the “Chat-Optimized” groups. Readability scores were analyzed using WebFx to assess Flesch-Kincaid Reading Ease, Flesch-Kincaid Grade Level, Gunning Fog Score, SMOG Index, Coleman LIAU Index, and Automated Readability Index. Statistical analysis was calculated using unpaired two-tailed t-tests, while ANOVA compared all groups. Statistical significance was determined as p < 0.05. RESULTS: ANOVA showed all four groups differed significantly across almost every metric (p<0.05), except for average syllables per word (p=0.232). Further analysis (Table 1) showed that ChatGPT-3.5 created sheets with significantly higher Flesch-Kincaid Reading Ease (p<0.05) and a lower Flesch-Kincaid Grade Level (p<0.05) compared to ChatGPT-4.0, for both the Generated and Optimized groups. Additionally, the 3.5 sheets possessed fewer sentences, words, and complex words (p<0.05) in both sets. For both versions, the Optimized sheets were written at a higher Flesch-Kincaid Grade Level (p<0.05) and included more sentences, words, and complex words (p<0.05) compared to the Generated sheets. CONCLUSION: ChatGPT-3.5 is a free, accessible tool that physicians and patients can use to create easily understandable education materials. Incorporating existing online content into ChatGPT is crucial to ensure that patients receive detailed, physician-supported information at a simplified level. With AI, we can promote health literacy, helping patients understand necessary information to make informed decisions aligned with their personal goals.

Ähnliche Arbeiten