Abstract WP19: Assessing the Ability of ChatGPT to Guide the Decision for Intravenous Thrombolysis in Patients with Acute Ischemic Strokes

Abstract

Introduction: Artificial intelligence is emerging as an adjunct promising tool in medicine. ChatGPT, an AI chatbot that uses machine learning to create human-like dialogue, has shown strong potential in the medical field, specifically aiding professionals in clinical reasoning and diagnosis. We aim to assess the ability of ChatGPT to guide the decision for intravenous thrombolysis (IVT) in patients with acute ischemic strokes (AIS). Methods: The performance of ChatGPT in clinical decision making for IVT was compared with that of a board-certified stroke neurologist using artificially created AIS scenarios, covering a broad range of indications and contraindications. ChatGPT was asked to act like an acute stroke clinical decision tool and to answer with a “yes” or “no” along with a 1-sentence justification for its answer. The accuracy and interpretation skills of ChatGPT and the stroke neurologist were analyzed by a blinded assessor, with more than 10 years of experience in stroke neurology. Results: Out of the 20 scenarios, ChatGPT’s decision to pursue or withhold IVT was deemed congruent with that of the stroke neurologist’s in 16 scenarios (80%). Based on the blinded assessor’s judgement, two clinical decisions made by ChatGPT and one clinical decision made by the stroke neurologist were rendered wrong. In one case, chatGPT’s reasoning was deemed to be incorrect but the correct clinical decision was made not to provide IVT. In another case, both the stroke neurologist and ChatGPT gave reasonable explanations despite different clinical decisions, both deemed plausible by the assessor. Overall, both chatGPT’s decision to pursue or withhold IVT and the explanation it provided were deemed accurate by the assessor in 17 scenarios (85%). Conclusion: This study shows that ChatGPT performed well in most scenarios, potentially reinforcing its ability to guide clinical decision making for IVT in AIS patients. However, ChatGPT is still prone to errors, as shown by its inability to consistently depict contraindications for IVT, and may still not be ready for independent use.

Ähnliche Arbeiten