AI vs Humans: The Future of Academic Peer Review in Public Administration

Abstract

<title>Abstract</title> In the ever-evolving landscape of academia, artificial intelligence (AI) presents promising opportunities for enhancing the academic review process. In this study, we evaluated the proficiency of Bard and GPT-4, two of the most advanced AI models, in conducting academic reviews. Bard and GPT-4 were compared to human reviewers, highlighting their capabilities and potential areas for improvement. Through a mixed-methods approach of quantitative scoring and qualitative thematic analysis, we observed a consistent performance of the AI models surpassing human reviewers in comprehensibility, clarity of review, the relevance of feedback, and accuracy of technical assessments. Qualitative analysis revealed nuanced proficiency in evaluating structure, readability, argumentation, narrative coherence, attention to detail, data analysis, and implications assessment. While Bard exhibited exemplary performance in basic comprehension and feedback relevance, GPT-4 stood out in detailed analysis, showcasing impressive attention to minor discrepancies and meticulous scrutiny. The results underscore the potential of AI as an invaluable tool in the academic review process, capable of complementing human reviewers to improve the quality, efficiency, and effectiveness of reviews. However, we also identified areas where human reviewers excel, particularly in understanding complex academic language and intricate logical progressions, offering crucial insights for future AI model training and development.

Ähnliche Arbeiten