Artificial intelligence in anesthesia: comparison of the utility of ChatGPT v/s google gemini large language models in pre-anesthetic education: content, readability and sentiment analysis

Abstract

BACKGROUND: Large Language Models (LLMs) such as ChatGPT and Google Gemini are increasingly explored for their potential in patient education, particularly in the perioperative setting. As text-based tools trained on extensive datasets, they can generate detailed responses to common medical queries. However, their utility in delivering reliable, understandable, and emotionally appropriate preanesthetic information remains unclear. METHODS: We conducted a prospective observational study comparing ChatGPT and Google Gemini in generating educational content for patients undergoing laparoscopic cholecystectomy. From 68 patient questions submitted by anesthesiologists, 13 high-relevance items were selected. Responses from both models were independently rated by 20 anesthesiologists using a 5-point Likert scale across four domains: accuracy, comprehensiveness, clarity, and safety. Mixed-effects ordinal regression with random intercepts for rater and question estimated odds ratios (OR) and 95% confidence intervals (CI) for ChatGPT versus Gemini. Readability was assessed using standard linguistic indices, and sentiment analysis was performed. Inter-rater reliability was evaluated using Krippendorff's α. RESULTS: ChatGPT had significantly higher odds of receiving better scores for accuracy (OR 2.32, 95% CI 1.62-3.32, p < 0.001) and comprehensiveness (OR 2.38, 95% CI 1.67-3.37, p < 0.001), with no differences for clarity (OR 1.05, 95% CI 0.75-1.47) or safety (OR 1.01, 95% CI 0.72-1.43). Gemini generated text with greater readability, demonstrated by a lower Flesch-Kincaid Grade level (p = 0.04) and higher Flesch Reading Ease score (p = 0.04). Sentiment analysis showed Gemini responses contained a wider emotional range, while ChatGPT responses were more neutral overall. Inter-rater reliability was low across domains (Krippendorff's α 0.23-0.46). CONCLUSION: ChatGPT produced more accurate and comprehensive perioperative anesthesia information, whereas Gemini offered greater readability and emotional expressiveness. Both models may serve as adjuncts in preanesthetic patient education but are not substitutes for clinician counselling. Larger, multi-center studies incorporating direct patient testing are warranted to validate these findings.

Ähnliche Arbeiten