5PSQ-129 ChatGPT’s performance and pharmaceutical intervention in dosing high-risk medications in the operating room

Abstract

<h3>Background and Importance</h3> High-risk medications can cause serious harm, more frequently when used in quick decisions environments, such as in the operating room. In this context, ChatGPT, an artificial intelligence model based on natural language processing developed by OpenAI, can assist healthcare professionals in critical situations, but there are doubts about its reliability. <h3>Aim and Objectives</h3> The objective is to evaluate the ability of ChatGPT-4.0 to answer questions about dosing high-risk medications in the operating room, comparing different techniques with or without pharmaceutical intervention. <h3>Material and Methods</h3> A list of high-risk medications used in the operating room was selected, and the responses of ChatGPT version 4.0 were evaluated as follows: Model without prompt: The standard question was: ‘What is the dosage of the drug in the operating room?’. Model with prompt: The following prompt was developed by a pharmacist ‘According to anaesthesia and pharmacy guidelines and technical data sheets, what is the drug dosage in the perioperative setting?’. Customised GPT model: The pharmacist created a high-risk medication dosing guide, which was validated by a multidisciplinary group and finally used to create a customised GPT model. The research was evaluated by two independent experts on the field in terms of: Safety: correct responses compared to summary of technical characteristics. Clarity: Writing and well-organising of information, scored as 0, 1, or 2. <h3>Accuracy: </h3> Ability to provide details related to dosing (score normalised to 1). <h3>Results</h3> A total of 40 medications were included. The expert reached consensus and the final results obtained were: <h3>Safety:</h3> Without prompt: 60% (24/40). With prompt: 70% (28/40). GPT model: 100% (40/40). <h3>Clarity:</h3> Without prompt: 58.3% (28/48). With prompt: 76.8% (43/56). GPT model: 85% (68/80). <h3>Accuracy:</h3> Without prompt: 44.8% (10.75/24). With prompt: 69.9% (19.6/28). GPT model: 99.1% (39.6/40). <h3>Conclusion and Relevance</h3> There is a need for improvement in basic ChatGPT models with or without prompts as they achieved 60 and 70% of correct responses. The valuable tool was the pharmacist customised GPT model reaching 100 % of correct answers (safety), improving clarity and accuracy. Consequently, pharmacists should work in customised models to help health professionals in clinical practice. <h3>References and/or Acknowledgements</h3> <h3>Conflict of Interest</h3> No conflict of interest

Ähnliche Arbeiten