Will Artificial Intelligence Nurse Practitioners Become True? Performance Evaluation of ChatGPT in the American Academy of Nurse Practitioners Certification Board Exam

Abstract

Nurse practitioners play a vital role in contributing to the UN’s Sustainable Development Goals, and Universal Health Coverage, especially the management of chronic noncommunicable diseases. Artificial intelligence tools such as ChatGPT are becoming promising resources for healthcare professionals. This study aimed to explore the capability of ChatGPT as a nurse practitioner by validating the performance of ChatGPT-3.5 and GPT-4 in the American Academy of Nurse Practitioners Certification Board Exam (AANPCB) practice examinations. Questions from exams for five nurse practitioner disciplines were used to evaluate the accuracy and consistency of the responses in two phases. In the first phase, the accuracy rates and concordance of answers between the two versions with the five exam sets, totaling 535 questions were analyzed. In the second phase, the consistency of ChatGPT-4 performance in six retests, each involving five random questions from each set. ChatGPT-3.5 achieved an overall accuracy rate of 80.6%, while ChatGPT-4 achieved 90.7%. ChatGPT-3.5 and ChatGPT-4 showed strong consistency within all sets, while ChatGPT-4 performed better than ChatGPT-3.5. In the retests, ChatGPT-4 provided exactly the same answers as generated initially, including the incorrect ones. In conclusion, ChatGPT demonstrated excellent performance in AANPCB practice exams, with high levels of accuracy and consistency. This suggests that ChatGPT may support nurse practitioners in making clinical decisions and improving efficiency. Further studies could explore ways to integrate artificial intelligence tools with nurse practitioner practice to enhance the advanced practice nursing workforce.

Ähnliche Arbeiten