Evaluation of Chat Generative Pretrained Transformer’s Performance in Cosmetic Injection Consultation by Authoritative Plastic Surgeons

Abstract

Background: ChatGPT (Chat Generative Pretrained Transformer), a large language model-based artificial intelligence, simulates doctor-patient communication. This study tested its applicability in injection-based cosmetic consultations using questions selected by 3 board-certified plastic surgeons with doctoral training and more than 10 years of experience at China's leading plastic surgery hospital. Methods: Professionalism and safety ratings were independently rated by 2 doctors for the test section and a third doctor for possible argument. Each question was asked 3 times independently to assess whether ChatGPT's performance on the same question was consistently satisfactory. Descriptive statistical analysis, multiple linear regression analysis, consistency analysis, 1-way analysis of variance, and repeated-measures analysis of variance were used to evaluate ChatGPT's safety, professionalism, empathy, and performance stability in this specific field. Results: ChatGPT showed 58.3% professionalism, 94.54% safety, and only 11.48% empathy, with lowest ratings for questions related to injection effects and wrinkles and highest ratings for failed repairs. Reproducibility was 65.57%, indicating stability. Empathetic and professional answers were usually more comprehensive, whereas inconsistent ones were correct but insufficient. Conclusions: By analyzing the assessment results of ChatGPT in terms of safety, professionalism, empathy, and performance stability, it is suggested that ChatGPT could potentially serve as an auxiliary tool to help doctors and patients improve treatment outcomes; however, efficiency and patient satisfaction were not directly measured in this study and should be validated in future prospective studies involving real patients.

Ähnliche Arbeiten