Evaluation of ChatGPT and Gemini Large Language Models for Generating Pharmacokinetic Models with SimBiology

Abstract

Generative Artificial Intelligence (AI) chatbots like ChatGPT or Gemini rely on Large Language Models (LLMs), which are deep-learning algorithms capable of recognizing, generating, summarizing, translating, and predicting content using large data sets. They are yielding a strong boost to technological innovation in all fields, including drug discovery and development, in which in-silico models are often used to describe the behavior of biological systems and study their pharmacokinetic/pharmacodynamic (PK/PD) properties. This study aimed to evaluate how Generative AI tools, such as ChatGPT and Gemini, among the widely accessible LLMs, can support the PK/PD modeling process. It evaluated the potential of these LLMs in generating instructions within SimBiology, a MATLAB tool dedicated to modeling, simulation and analysis of biological systems. Four case studies were considered, the first two aimed at teaching SimBiology fundamentals and at providing basic model examples, and the second two directly related to the creation of a PK/PD model. Each output was evaluated based on the instructions provided, the differences between the two LLMs' answers, the errors made, and the ability of the tool to correct them. The results showed that ChatGPT offered greater accuracy and flexibility in code generation than Gemini, since it is able to better correct its errors, although both presented structural and syntactic errors, limiting the fully automated modeling tasks. In conclusion, ChatGPT and Gemini are promising tools for building in-silico models, especially in the early stages of model development, but at present, they require human supervision and expertise to correct errors and improve reliability.

Ähnliche Arbeiten