RoBIn Chatbot: Leveraging LLMs for Automated Risk of Bias Assessment in Clinical Studies

Abstract

The Risk of Bias (RoB) assessment is an essential instrument for evaluating the reliability of clinical studies and identifying any systematic error that can occur. This task is traditionally performed by humans, and only a few works tried to automate it using machine learning. Recent advances in large language models (LLMs) have revolutionized natural language processing and information retrieval, allowing us to build applications that can chat with documents and perform the most diverse tasks. In this work, we propose RoBIn chatbot, an LLM application able to receive clinical studies as input and classify their RoB. RoBIn chatbot uses a model trained on data derived from the Cochrane Database of Systematic Reviews and is able to perform inference for six bias types. To prevent the LLM from generating misleading conclusions, it relies on retrieval-augmented generation on the submitted file to extract the piece of evidence and send it to a pretrained model responsible for performing RoB inference.

Ähnliche Arbeiten