Effect of artificial intelligence-augmented human instruction on surgical simulation performance: A randomized controlled trial

Abstract

BackgroundWith current surgical teaching models’ lack of standardization and reliance on subjective assessments by human experts rather than quantitative performance data, training novices to master surgical technical skills remains challenging. To mitigate this issue, we developed an artificial intelligence (AI) application known as the Intelligent Continuous Expertise Monitoring System (ICEMS) capable of assessing bimanual surgical skills at 0.2-second intervals and providing continuous, action-oriented verbal feedback.ObjectivesThe objective of this study is to determine the effect of AI-augmented personalized expert instruction versus AI tutor instruction alone on surgical performance, skill transfer, and affective-cognitive responses.MethodsA multi-institutional randomized controlled trial was conducted wherein medical students performed subpial brain tumour resection tasks on the NeuroVR and received real-time feedback on their performance. Students were stratified based on their year in medical school and block randomized to one of three groups. Group 1 received AI tutor instruction delivered by the ICEMS, group 2 received expert feedback in identical words to the ICEMS, and group 3 received AI data-informed personalized expert feedback. Trainees performed six practice subpial resection tasks to assess learning followed by a complex realistic brain tumour resection scenario to assess skill transfer. The ICEMS quantitatively evaluated trainee performance. Participants self-reported emotions before, during, and after training and cognitive load after training via questionnaires.ResultsEighty-seven medical students from four Quebec institutions were randomly assigned to the AI instruction (n = 30), expert instruction (n = 29), and personalized expert instruction (n = 28) groups. The ICEMS assessed and scored 522 practice resections and 87 realistic resections. During the practice tasks, personalized expert instruction resulted in significantly greater expertise scores than AI tutor instruction across several trials, including trial 5 (mean difference, 0.26 [95% CI, 0.09 to 0.43]; P = 0.01). During the realistic task, the personalized instruction group had significantly higher expertise scores than both the AI tutor instruction (mean difference, 0.20 [95% CI, 0.06 to 0.34]; P = 0.02) and expert instruction (mean difference, 0.18 [95% CI, 0.03 to 0.32]; P = 0.049) groups. The personalized expert instruction group also achieved significantly higher scores than the other two groups in certain metrics, such as bleeding and injury risk. Emotions and cognitive load demonstrated significant differences.ConclusionPersonalized expert instruction resulted in enhanced surgical performance and skill transfer compared with intelligent tutor instruction, highlighting the importance of human input and active participation in AI-based surgical training

Ähnliche Arbeiten