Autonomous Analysis of Curated Patient Data Using a Large Language Model–Based Multiagent Framework

Abstract

PURPOSE: Analyzing complex medical data sets is specialized and time-consuming. This study aimed to develop and evaluate a novel multiagent artificial intelligence (AI) framework for automating medical data analysis workflows and to compare its performance against nonagent-based approaches using large language models (LLMs). METHODS: A six-party AI agent system was developed using the AutoGen platform, with specialized agents for planning, data retrieval, cleaning, statistical analysis, and review, powered by OpenAI gpt-4o. This framework was applied to deidentified single patient-level data sets from 20 recent studies in the field of bone marrow transplantation (2021-2023). The primary objective was to evaluate its accuracy in replicating published primary outcomes, benchmarked against direct use of the Web site-based ChatGPT 4o. RESULTS: = .04). The agent framework's failures were predominantly due to data transformation issues (46.4%) and analysis code errors (21.4%). In contrast, ChatGPT 4o failures largely stemmed from incorrect statistical method application (38.4%) and data transformation (45.6%), often attempting to resolve code errors by switching to alternative, incorrect statistical methods. Hallucinations of variables or results were not observed in the multiagent approach. CONCLUSION: Our multiagent AI framework demonstrated superior accuracy and robustness in automating biomedical data analysis compared with a generalized LLM.

Ähnliche Arbeiten