Leveraging Artificial Intelligence Large Language Models for Writing Clinical Trial Proposals in Dermatology (Preprint)

Abstract

<sec> <title>BACKGROUND</title> Large language models (LLMs) are becoming increasingly popular in clinical trial design but have been underutilized in research proposal development. </sec> <sec> <title>OBJECTIVE</title> This study compares the performance of commonly used open access LLMs versus human proposal composition and review. </sec> <sec> <title>METHODS</title> Ten LLMs were prompted to write a research proposal. Six physicians and each of the LLMs assessed 11 blinded proposals for capabilities and limitations in accuracy and comprehensiveness. </sec> <sec> <title>RESULTS</title> Chat GPT o1 was rated the most accurate and LLaMA 3.1 the least accurate by human scorers. LLM scorers rated Chat GPT o1 and Deepseek R1 the most accurate. Chat GPT o1 was the most comprehensive and LLaMA 3.1 the least comprehensive by human and LLM scorers. LLMs performed poorly on scoring proposals, and on average rated proposals 1.9 points higher than humans for both accuracy and comprehensiveness. </sec> <sec> <title>CONCLUSIONS</title> Paid versions of ChatGPT remain the highest quality and most versatile option of available LLMs. These tools cannot replace expert input but serve as powerful assistants, streamlining the development process and enhancing productivity. </sec> <sec> <title>CLINICALTRIAL</title> n/a </sec>

Ähnliche Arbeiten