AI in Research Assessment, A Global Symposium

Abstract

The Research Evaluation Group (INORMS) and the REACH Network are pleased to invite you to a Global Symposium on “AI in Research Assessment”, taking place 11 February 2026. This event brings together leading voices in research policy and evaluation, to explore one of the most urgent questions facing the scholarly community: **Can AI assess research?** Artificial intelligence promises to reshape how research is evaluated. Emerging AI tools can generate detailed, human-like assessment - opening exciting opportunities to reduce administrative burden, support evaluation at scale, and uncover new insights. At the same time, these technologies raise concerns around accuracy, transparency, equity, and the protection of research integrity. The symposium arrives at a moment when the global research community is already rethinking evaluation practices. As assessment evolves beyond narrow bibliometric indicators, researchers and institutions increasingly recognize the importance of mentorship, open science, societal and economic contributions, and collaborative work. How might AI support these responsible approaches - and under what safeguards? Our aim is to engage critically with both the promise and the pitfalls of AI-assisted evaluation, asking: • When can AI meaningfully contribute to research assessment? • What standards, governance, and oversight are required? • Which tasks should remain firmly in the hands of peer-review? **AI in Research Assessment: Friend or Foe for Research Integrity?** Dr. Maura Hiney The interplay between research assessment and research integrity is complex. Robust research integrity practices ensure trust and quality in the direct products of research (outputs) and the subsequent use and effects of these outputs (outcomes and impacts). In such an environment, honest and fair assessment is supported. However, when research integrity fails, assessors may be basing their judgements on biased, poor-quality, or untrustworthy outputs and outcomes. In addition, assessing individuals whose research practices are poor or who are willing to cheat to advance their careers may skew the assessment system in their favour and disadvantage those striving for quality in their research. Adding AI tools to an already complex relationship raises the question of whether using such tools in research assessment will help or hinder research integrity practices. **Can Large Language Models Be Used Responsibly in Research Assessment?** Dr. Mike Thelwall Large Language Models (LLMs) are already being used in research assessment by at least one funder, as well as by some conferences and journals. In all cases they support but do not replace human judgment. Recent evidence also suggests that LLMs are moderately good at assessing the quality of published academic research, even based only on titles and abstracts. Given the huge burden of various forms of peer and expert review in academia, it is tempting to suggest that shortcuts and timesaving must be possible. This is already happening because LLMs are secretly used by many reviewers to replace their opinion, even if the journal, conference, or funder tells them not to. This practice has probably increased the risk that flawed research passes peer review and good research is blocked. More generally, all uses of LLMs in research assessment must be considered carefully from various perspectives, including biases, perverse incentives, and the practical outcomes of greater reliance on AI. For example, if AI becomes more important, will authors prioritize studies that they expect ChatGPT to like, such as those singing the praises of AI or with exaggerated value claims? **Bridging Research and AI for Impact Narratives** Dr. Diego Emilio Lozano Qualitative peer review for researchers is inherently time-consuming, demanding substantial effort for reviewer training and process standardization to ensure robust and unbiased evaluations. To address this, the University of Monterrey (UDEM) conducted a control exercise: 80 researchers were evaluated by five independent peer reviewers to mitigate human bias and establish a reliable average score. Subsequently, we tested an AI tool as a potential replacement for multiple human evaluators. While the AI excelled at generating critical textual analysis, it was ultimately found to be incapable of delivering a reliable, direct quantitative assessment. Our final, successful approach redefined the AI’s role: the tool now rapidly analyzes complex research documents and generates structured, focused validation questions for the human evaluator. This transformation shifts the evaluator's task from deep review to efficient verification. This AI-augmented workflow significantly accelerates the joint validation of impact narratives. The conclusive model positions the AI not as a substitute, but as a critical support mechanism, projecting a sustainable reduction in required human evaluators from five to just two. **Panel Session Includes** Dr. Elizabeth (Lizzie) Gadd, Loughborough University Dr. Julia Melkers, Arizona State University

Ähnliche Arbeiten