D44. Artificial Intelligence In Plastic Surgery Writing: Can We Even Tell The Difference?

Abstract

PURPOSE: Artificial intelligence (AI) increasingly aids research writing. This study quantifies plastic surgeons’, trainees’, and laypeople’s ability to differentiate between AI and human-written research abstracts and evaluate the quality of writing comparatively METHODS: Ten abstracts were created using study outlines of previously accepted plastic surgery studies: 5 were generated using Pre-trained Transformer 4 (GPT-4) while 5 were written by plastic surgery trainees. Surveys were electronically distributed to participants including plastic surgeons, trainees, and laypeople who were blinded to authorship. Respondents were asked to rate 8 abstracts individually using a 12-point quality scale, and compared 4 abstracts side-by-side. Writing quality and the association between training levels and discrimination abilities were assessed using paired T-tests and regression models. RESULTS: Of 355 surveys, 174 (49%) were completed. Respondents averaged 29 years, 56% were females, 68.5% Caucasians, 51% hold/pursue doctorate degrees, and 27.3% had a bachelor’s degree or lower. Respondents in medicine constituted 63% medical students, 21% residents, and 17% attendings. Mean scores showed no significant difference between AI and human-written abstracts (9.9 vs. 10.2, P=0.11). Attendings favored AI abstracts, while other levels preferred human-written abstracts. In side-by-side comparisons, 71% selected human-written abstracts as superior. Risk-adjusted analysis showed medical students and residents more likely to prefer human-written abstracts over AI, compared to laypeople and attendings (P<0.001). CONCLUSION: GPT-4 can generate scientific abstracts comparable to human-written. Trainees exhibit better differentiation abilities and all readers discriminate better when compared side-by-side. The implications of AI in scientific writing warrant discussion due to its human-level content synthesis capability.

Ähnliche Arbeiten