Accuracy of ChatGPT‐4 Plus in Providing Information on Oral Cancer Management

Abstract

OBJECTIVE: Artificial intelligence (AI)-driven large language models, such as Chat Generative Pre-Trained Transformer (ChatGPT)-4 Plus, are increasingly used for patient education and clinical decision support in oral oncology, although their accuracy in oral cancer (OC) management remains uncertain. This study evaluates the accuracy of ChatGPT-4 Plus responses to clinically relevant questions regarding OC diagnosis, treatment, recovery, and prevention. METHODS: A cross-sectional study assessed 65 clinically relevant OC-related questions using a paid ChatGPT-4 Plus subscription without modifications. Three oral medicine specialists and one radiation oncologist rated accuracy on a four-point scoring system. Interrater reliability was measured with the intraclass correlation coefficient (ICC), and chi-square tests were used for comparisons. RESULTS: Among 65 questions, 63% of responses were Score 1, with none rated as Score 4. Score 1 was most frequent in Recovery (72%), followed by Treatment (62%), Prevention (60%), and Diagnosis (55%). Scores 2 and 3 responses were highest in Diagnosis (45%). Recovery had significantly higher Score 1 responses than Diagnosis (p < 0.05), while other comparisons were not significant. ICC ranged from 0.85 to 0.93. CONCLUSIONS: ChatGPT-4 Plus provided accurate responses to clinically relevant OC-related questions, particularly regarding recovery. However, diagnostic inconsistencies highlight the need for clinician oversight before integrating AI into practice.

Ähnliche Arbeiten