Potential Meets Practicality: <scp>AI</scp>'s Current Impact on the Evidence Generation and Synthesis Pipeline in Health Economics

Abstract

Artificial intelligence has already shown great promise in health economics and outcomes research, activities essential to inform healthcare policy and reimbursement. Artificial intelligence can increase efficiency in core research activities, e.g., streamlining literature reviews, processing complex unstructured data, and enhancing predictive analytics. However, its acceptance faces hurdles, including sensitive-data use and security, regulatory constraints, and health technology assessment agency requirements. Future adoption of artificial intelligence will require continued demonstration of value, enabling trust-building across diverse stakeholders. Health economics and outcomes research (HEOR) plays a key but often underappreciated role in drug development, providing essential evidence to inform healthcare policy and reimbursement by global payors. While there is broad enthusiasm around the potential of artificial intelligence (AI) among HEOR researchers [1], there are challenges related to its widespread adoption. First, there are a variety of approaches that fall under the AI umbrella, including generative AI based on natural language processing (NLP), large language models (LLMs), and machine learning (ML) methods that classify, cluster, and predict outcomes. Each has potential distinct roles in HEOR, but there are few published evaluations of how these technologies can be practically applied across a breadth of research areas. Second, stakeholder perspectives may impact the value of AI. For example, within the pharmaceutical industry, AI end-users may be interested in more efficient execution of rote tasks, while regulatory bodies and health technology assessment (HTA) agencies, with a wider-ranging public health purview, advise a cautious approach that upholds established scientific practices while ensuring compliance with legal, ethical, data protection requirements and quality standards [2]. Herein, we examine the current practical and potential future applications of AI across activities critical to payor reimbursement in HEOR. Figure 1 highlights the connectivity of three major HEOR research tools (evidence synthesis, economic modeling and real-world evidence (RWE) generation and evaluation), while also giving an overview of how AI is currently being applied in these fields. Evidence synthesis refers broadly to activities related to review, curation, and extraction of information from published literature. Within this space, including informal targeted and formal systematic literature reviews (SLRs), AI has been applied to facilitate more efficient and thorough processes via automation, enhanced precision, and the management of large volumes of data while saving time, including: NLP algorithms can assist in the initial stages of SLRs by scanning large datasets of publications to identify relevant studies using a variety of schemes, including Generative Pre-Trained Transformer 4 (GPT-4) and other AI classifiers [3]. Once relevant studies are identified, AI tools can automatically extract key data points such as study characteristics and outcomes, ensuring consistency across reviews, improving efficiency, and saving time [3]. ML models can help automate the quality assessment of included studies, determining their reliability and validity with high accuracy [3]. AI can automate the generation of attrition flow diagrams required for SLRs to show the screening and selection process of studies [3]. AI tools can help identify gaps in existing research and evidence, offering guidance on future research directions [3]. AI can summarize individual papers or groups of papers, highlighting key findings and drafting entire sections of manuscripts. These tools also can aggregate and synthesize data, such as effect sizes, and perform statistical analyses. While the above applications clearly lift the burden of traditionally labor-intensive processes, there exists an unrealized potential of AI to also facilitate activities downstream of SLRs. For example, comparative effectiveness research via indirect treatment comparisons is typically based on network meta-analyses (NMA) synthesizing evidence from RCTs collected from SLRs; AI-based methods may improve the speed at which previously described methods integrate effect estimates from an NMA with observational real-world data (RWD) [4], allowing for more thorough consideration of the impact of individual patient characteristics and improved control for sources of bias [1]. AI has the potential to transform how researchers analyze RWD and improve their efficiency in generating insights from RWE. AI excels at processing unstructured data, such as text from clinical notes, medical images, and social media posts, using tools like NLP and image recognition [5]. This has enabled the assessment of the patient journey, treatment patterns, and resource utilization, and can also be used to monitor pharmacovigilance or flu dynamics. ML algorithms, particularly deep learning models, can handle high-dimensional data (e.g., genomics, imaging, and multi-omics data) with millions of variables, and multimodal data (e.g., electronic health record [EHR] data, genomic data, imaging, and sensor data) can be integrated and learned from to find patterns across different types of inputs. As such, these data sources can be used to automatically detect patterns without the need for pre-specifying relationships, which can lead to better predictive power and insights in disease diagnosis and is useful for identifying patient subgroups or clusters with distinct features, such as fast progressors or high responders [6]. AI models such as neural networks and decision trees can model complex non-linear relationships and interactions between variables, without assuming a pre-specified functional form. This enables precision medicine approaches including the prediction of treatment effects and optimal drug sequencing on an individual patient level, and making forecasts about disease progression and patient readmission. Although some of the aforementioned applications can also be addressed by more traditional statistical methods, they may struggle when there are many predictors relative to observations and requirements to specify relationships between variables upfront, making it difficult to capture non-linearities and higher-order interactions without significant manual intervention. Researchers have used AI to ameliorate such issues [7]. These technical and capability improvements arising from the use of AI to analyze and interpret RWD will impact the way health economic modeling is executed: HE models can more easily incorporate insights from RWE, such as the sizing of the target population for a specific compound or characterizing patient subgroups with distinct disease progression patterns. Most ML applications in the field of HEOR through 2022 have focused on cohort selection, feature selection, and predictive analytics, with fewer applications within economic modeling [1]. However, recent evidence has shown the potential application of LLMs in constructing cost-effectiveness modeling code [8]. GPT-4 was used with a structured prompt coding framework, which fed the LLM information on model assumptions, methods, and parameter values, calling for fully programmed model R scripts to be outputted. The authors found most tests contained either no errors or a single minor error, highlighting the potential time savings in using AI in the construction of cost-effectiveness models [8]. Additionally, a GPT LLM is also being used within R packages to allow for visual representation of how model functions work together, aiding in debugging processes and enabling more robust cost-effectiveness modelling [9]. These approaches allow for calls directly to LLMs within the model scripts, sending up and pulling down coding prompts and outputs, respectively, through Application Programming Interfaces. However, linking modeling code to LLMs created by private companies means employing “black box” algorithms that may lack transparency; associated risks could be mitigated by including regular checks and error flags. Additionally, the more LLMs are used in health economic modeling, the more they will learn, with error rates for model building already seemingly low. LLMs owned by private, for-profit companies may increase prices for use (e.g., cost per token), an issue if model building and use workflow depend on such LLMs. However, multiple large-scale LLMs, some open-access, are now available, with similar prompt engineering needs, hopefully reducing over-reliance on one singular external company. It is important for users to understand differing data usage and storage policies of the LLM they intend to use to reduce information security issues. Current ML and NLP capability could allow for learning from larger, linked healthcare and economic datasets to optimize parameter estimation, such as costs of care and productivity impacts, meaning AI integration at all points of the economic modeling pipeline is feasible. Irrespective of the state of the art of AI in each of the reviewed disciplines, research to date clearly shows the far-reaching implications of AI to alter the basic needs and skill sets required of industry HEOR researchers. In modeling, AI applications in script-based modeling can reduce the need for researchers to construct models from scratch in MS Excel. For RWE, while a deep understanding and knowledge of existing real-world data sources will continue to be crucial for the use of AI, scientists proficient in applying NLP and/or ML methods and interpreting their results are likely to be increasingly required. Similarly, for evidence synthesis and modeling, it is unlikely that AI tools will fully obviate the need for human involvement for processes like validation and screening conflict resolution due to regulatory and HTA requirements; however, sponsors will certainly realize time and cost savings, potentially increasing budgets to fund research to support additional activities to further bolster an asset's economic value story. Applications of AI across the HEOR space have shown demonstrable ability to improve operational processes and consequently reduce associated time and cost. In the evidence synthesis and the real-world evidence spaces, the value of AI has been relatively robustly evaluated, while in HEOR modeling the implementation and evaluation of AI applications are in relative infancy. While the usage of AI/ML in HTA remains limited to date, the advent of guidelines from regulators and HTA agencies setting requirements on justification and transparency when using AI in submissions is likely to rapidly increase its acceptance and impact [2, 10]. AI will soon be able to go beyond current applications and bridge the data-divide highlighted in Figure 2. Generating hypotheses, constructing and evaluating patient-specific treatment-pathways, estimating real-time costs and outcomes based on data linkage across systems and types should be feasible on a larger scale thanks to AI. The fast rate of change in this field requires open, shared-learning across sectors and agents, with many day-to-day tasks of HEOR researchers likely to change as human-time spent on evidence synthesis, RWE analytics, and modeling is transferred to AI. Though we summarize key potential future and current use cases of AI in HEOR, there is a need for more published evidence to enable a better benefit–risk assessment and to build increased trust in AI methods from all stakeholder perspectives. The authors declare no conflicts of interest.

Ähnliche Arbeiten