Automatic Generation of Learning Material for Residential Students in ED: Prompt Engineering and Evaluation

Abstract

Emergency Department (ED) physicians must manage all types of pathologies and quickly implement the appropriate diagnostic and therapeutic pathways. Flashcards can be a valuable support and consultation tool because they are simple and effective learning methods. Accordingly, we started a project that aims to develop an app that provides flashcards to help resident students in the ED learn the correct diagnostic and therapeutic guidelines. However, developing the appropriate flashcard content is long, complex, and expensive. This research discusses how we tackled this issue through automated content generation: we first report on the prompt engineering phase and then on the evaluation of the learning material. The prompt engineering phase returned some valuable lessons learned that can be useful to researchers involved in a similar process. The (intrinsic and extrinsic) evaluation showed that the automated material was of good quality, even if the lack of in-depth contextual knowledge by the Large Language Model (LLM) of the specific domain of ED was one of the significant problems, and minor revisions remained necessary. In conclusion, the research summarized in this paper provides additional evidence about the pros and cons of using LLMs for automated content generation in the specific context of medical education.

Ähnliche Arbeiten