Revealing the Paper Mill Iceberg: AI-Based Screening of Cancer Research Publications

Abstract

Abstract Objectives To train and validate a machine learning model to distinguish paper mill publications from genuine cancer research articles, and to screen the cancer research literature to assess the prevalence of papers that have textual similarities to paper mill papers. Design Methodological study applying a BERT-based text classification model to article titles and abstracts. Setting Retracted paper mill publications listed in the Retraction Watch database were used for model training. The cancer research corpus was screened by the model, using the PubMed database restricted to original cancer research articles published between 1999 and 2024. Participants The model was trained on 2,202 retracted paper mill papers and validated on independent data collected by image integrity experts. A total of 2.6 million cancer research papers were screened. Main outcome measures Classification performance of the model. Prevalence of papers flagged as similar to retracted paper mill publications with 95% confidence intervals and their distribution over time, by country, publisher, cancer type, research area, and within high-impact journals (Decile 1). Results The model achieved an accuracy of 0.91. When applied to the cancer research literature, it flagged 9.87% (95% CI 9.83 to 9.90) of papers and revealed a large increase in flagged papers from 1999 to 2024, both across the entire corpus and in the top 10% of journals by impact factor. Over 170,000 papers affiliated with Chinese institutions were flagged, accounting for 35% of Chinese cancer research articles. Most publishers had published substantial numbers of flagged papers. Flagged papers were overrepresented in fundamental research and in gastric, bone, and liver cancer. Conclusions Paper mills are a large and growing problem in the cancer literature and are not restricted to low impact journals. Collective awareness and action will be crucial to address the problem of paper mill publications.

Ähnliche Arbeiten