ChatGPT‐3.5 and ‐4.0 Do Not Reliably Create Readable Patient Education Materials for Common Orthopaedic Upper‐ and Lower‐Extremity Conditions

Abstract

Purpose: To investigate whether ChatGPT-3.5 and -4.0 can serve as a viable tool to create readable patient education materials for patients with common orthopaedic upper- and lower-extremity conditions. Methods: Using ChatGPT versions 3.5 and 4.0, we asked the artificial intelligence program a series of 2 questions pertaining to patient education for 50 common orthopaedic upper-extremity pathologies and 50 common orthopaedic lower-extremity pathologies. Two templated questions were created and used for all conditions. Readability scores were calculated using the Python library Textstat. Multiple readability test scores were generated, and a consensus reading level was created taking into account the results of 8 reading tests. Results: < .001), including the Automated Readability index, Coleman-Liau index, Dale-Chall formula, Flesch-Kincaid grade, Flesch Reading Ease score, Gunning Fog score, Linsear Write Formula score, and Simple Measure of Gobbledygook index. Conclusions: Our results indicate that ChatGPT-3.5 and -4.0 unreliably created readable patient education materials for common orthopaedic upper- and lower-extremity conditions at the time of the study. Clinical Relevance: The findings of this study suggest that ChatGPT, while constantly improving as evidenced by the advances from version 3.5 to version 4.0, should not be substituted for traditional methods of patient education at this time and, in its current state, may be used as a supplemental resource at the discretion of providers.

Ähnliche Arbeiten