Extended-data for the manuscript: Preclinical Evaluation of Large Language Model-Generated Instructions to Complement Digital Prescriptions in Primary Health Care v1

Abstract

This study is a randomized, blinded experimental study utilizing prescription-inducing scenarios assigned among 62 healthcare professionals to validate instructions generated by Large Language Models (LLMs). A simulation environment was developed with a layout similar to the electronic health record (EHR) of the Unified Health System, in Brazil. Medication use instructions were generated by three models: ChatGPT-4.0, Llama3.1-8B, and Llama3.1-8B enhanced with Retrieval Augmented Generation (RAG) using drug package inserts (Llama3.1-8B-RAG). Performance metrics, including appropriateness, completeness, clarity, personalization, utility, and error rates in the instructions, were analyzed globally and specifically.

Ähnliche Arbeiten