Physician Perspectives on ChatGPT 4.o as a Patient Resource for Abdominal Cancer Surgeries: A Cross-Sectional Survey (Preprint)

Abstract

<sec> <title>BACKGROUND</title> Artificial intelligence (AI) models are being increasingly integrated into clinical care. Moreover, the accessibility of open-AI resources makes them attractive to patients seeking clinical information. Little is known regarding the utility of large language models as patient resources for navigating major cancer diagnoses. </sec> <sec> <title>OBJECTIVE</title> This study evaluated the content, readability and safety of ChatGPT 4.o-generated responses to common perioperative queries about hepatic, pancreatic, and colon cancers. </sec> <sec> <title>METHODS</title> A 28-question survey was developed based on frequently asked surgical questions for select malignancies. Surgical oncologists rated ChatGPT 4.o-generated responses on a 5-point Likert Scale for accuracy, quality, and tangibility. Readability was assessed using the Flesch-Kincaid Reading Grade Level (FKRGL). Respondents provided free-text comments and reported comfort with patients using ChatGPT. Survey completion implied consent. </sec> <sec> <title>RESULTS</title> Seven attending surgical oncologists with a median of 7 years in practice completed the survey. Responses received mean scores of 3.5/5 for quality, 3.6/5 for accuracy, and 3.6/5 for tangibility. The responses had a median FKRGL score of 14.5. On post-hoc analysis for select questions, median FKRGL was 15.6, decreasing to 7.1 and 14.5 with prompting and rephrasing. Numerous inaccuracies and content gaps were reported, and approximately 43% of providers did not report feeling “comfortable” in having patients consult publicly available AI for medical information. </sec> <sec> <title>CONCLUSIONS</title> This study provides cautionary, yet optimistic, findings regarding the value of open-source ChatGPT as a patient resource for abdominal malignancies. Providers should be prepared to effectively counsel patients concerning use of ChatGPT to mitigate minor inaccuracies and address readability challenges. </sec>

Ähnliche Arbeiten