Will AI revolutionize literature reviews?

Abstract

The study by Passby et al. on artificial intelligence (AI)-generated dermatology literature reviews provides valuable insights into the potential of AI to transform scientific publishing, particularly in literature reviews.1 Their evaluation of three AI tools—The Literature, Microsoft's Copilot and Google's Gemini—across five dermatology topics demonstrated that AI-generated reviews could serve as a useful resource for clinicians and patients. Notably, they highlighted Gemini's consistently superior performance in evidence analysis, conclusions and reference accuracy. However, the authors concluded that, despite its promise, even the best AI-generated reviews fall short compared to human-written systematic reviews underscoring both the potential and the current limitations of AI in this field. The potential of AI in encoding scientific knowledge is considerable.2 Rapidly advancing foundation models, such as OpenAI's GPT-4, Meta's LLaMA or Google's Gemini, have the potential to significantly accelerate evidence synthesis.3 This could help address the time-consuming nature of systematic reviews, facilitating more frequent updates to clinical guidelines and reducing publication delays. However, as highlighted by Passby et al., there are critical challenges. Much like AI-powered diagnostic tools, a reality check reveals certain shortcomings and limitations that currently hinder the broader applicability of AI in generating literature reviews.4 AI tools often missed key references and occasionally strayed off-topic, raising concerns about their reliability and comprehensiveness and their inability to access non-open access papers potentially skew results and misses crucial data. Moreover, it remains unclear whether current AI models can provide the nuanced interpretation and critical analysis that human experts bring to systematic reviews. Human-authored literature reviews offer unique perspectives and insights that transcend mere summarization. They provide critical analysis, identify trends and controversies and offer viewpoints that can shape future research directions. The human touch in framing arguments, drawing connections between disparate findings and providing context often makes a literature review truly impactful. Furthermore, the process of conducting a literature search and writing a review article is an invaluable learning experience for researchers. It allows researchers to immerse themselves deeply in their field, developing a nuanced understanding of the current state of knowledge, methodologies and ongoing debates. The act of writing a review article requires researchers to articulate complex ideas clearly, construct logical arguments and frame the current state of knowledge within a broader context. This process often leads to the identification of knowledge gaps and the formulation of new research questions, sparking innovation and driving the field forward. The benefit of literature reviews is not only consumed by those who read them but also profoundly impacts those who conduct them. This process serves as a crucial educational experience for researchers, particularly those early in their careers. There is also a risk that AI models could exacerbate the ‘rich get richer’ phenomenon in academic citations. If AI tools primarily draw from mainstream thinking and frequently cited papers, they could further marginalize critical or alternative viewpoints, potentially stifling scientific debate and innovation. Training researchers, editors and reviewers to effectively use and critically evaluate AI-generated content is crucial. A recent study revealed that while researchers possess different levels of familiarity with artificial intelligence, its application in scientific research remains in its early stages.5 Moving forward, the scientific community must thoughtfully integrate AI into the publishing process, leveraging its strengths while preserving the critical human elements of expertise, judgement and creativity that drive scientific progress. The future of scientific publishing likely lies in a synergistic relationship between human intellect and artificial intelligence, rather than a wholesale replacement of traditional methods. None. Data sharing not applicable to this article as no datasets were generated or analysed during the current study.

Ähnliche Arbeiten