Evaluating ChatGPT o1’s Capabilities in Peripheral Nerve Surgery: Advancing Artificial Intelligence in Clinical Practice

Abstract

OBJECTIVE: Artificial intelligence (AI) continues to advance in healthcare, offering innovative approaches to enhance clinical decision-making and patient management. Peripheral nerve surgery poses unique challenges due to the complexity of cases and the need for precise diagnostic and therapeutic strategies. This study investigates the application of OpenAI's generative AI model, o1, in assisting with intricate decision-making processes in peripheral nerve surgery. METHODS: Using advanced prompt engineering techniques, o1 was configured as a virtual medical assistant (Generative Pretrained Transformer-Nerve Surgery [GPT-NS]) to process 5 simulated clinical scenarios modeled after real-world cases. The AI guided surgeons through medical history, diagnostics, and treatment planning, culminating in case summaries. A panel of nerve surgery specialists and residents evaluated the AI's performance using a Likert scale across 7 criteria. RESULTS: GPT-NS demonstrated strong capabilities, achieving an average score of 4.3. High ratings were observed for understanding clinical issues and case presentation clarity. However, areas for improvement were noted in diagnostic sequencing and treatment recommendations. Despite a lower score indicating human evaluators' perception of their superiority over the AI in handling cases, GPT-NS showed promise as a supportive tool in clinical practice. CONCLUSIONS: As the performance of large language model AI continues to improve, it is becoming increasingly important that absolute experts assess the accuracy of the answers to ensure reliable and clinically sound integration into healthcare practices. This study underscores the potential of large language model AI in augmenting clinical decision-making in highly specialized fields like peripheral nerve surgery while demonstrating the ongoing importance of human expertise. Future research should explore ways to further refine AI capabilities and assess its integration into routine surgical workflows.

Ähnliche Arbeiten