Machine learning-assisted abstract screening on learning analytics: a step-by-step tutorial

Abstract

Systematic reviews are crucial for synthesizing evidence, but their manual processes, particularly abstract screening, are labor-intensive and prone to error. Advances in machine learning (ML) offer solutions to enhance efficiency and accuracy. Using a learning analytics (LA) in higher education review as a case study, this tutorial provides a step-by-step guide to implementing two ML solutions to streamline abstract screening. One solution is ASReview, an active learning-based ML framework. We detailed data preparation, ASReview setup, and its active learning capabilities, which significantly reduce manual workloads while maintaining high recall rates. The second solution is ChatGPT, a GPT-4 powered large language model (LLM), to demonstrate optimizing prompts and parameters in Python's Google Colab environment for accurate and consistent screening results. We present performance metrics, including sensitivity, specificity, and accuracy, to evaluate each tool's strengths and limitations. ASReview excels in handling large datasets, while ChatGPT enhances screening precision with well-designed prompts. This tutorial empowers researchers to integrate ML into systematic reviews, ensuring rigor, transparency, and efficiency while addressing the growing complexity of evidence synthesis.

Ähnliche Arbeiten