Auto-assessment of assessment: A human-in-the-loop AI framework addressing policy gaps in academic assessment

Abstract

Education is being transformed by rapid advances in Artificial Intelligence (AI), including emerging Generative Artificial Intelligence (GAI). Such technology can significantly support academics and students by automating monotonous tasks and making personalised suggestions. However, despite the potential of the technology, there are significant concerns regarding AI misuse, particularly by students in assessments. There are two schools of thought: one advocates for a complete ban on it, while the other views it as a valuable educational tool, provided it is governed by a robust usage policy. This contradiction clearly indicates a major policy gap in academic practices, and new policies are required to uphold academic standards while enabling staff and students to benefit from technological advancements. We surveyed 117 academics from three countries (the UK, the UAE, and Iraq) and identified that most academics retain positive opinions regarding AI in education. For example, the majority of experienced academics do not favour complete bans, and they see potential benefits of AI for students, teaching staff, and academic institutions. Importantly, academics specifically identified the particular benefits of AI for autonomous assessment (71.79% of respondents agreed). Therefore, for the first time, we introduce a novel AI framework for evaluating students' work (e.g., reports, coursework, etc.) within a human-in-the-loop assessment process, in which automated grade suggestions are generated and subsequently reviewed by qualified instructors based on students' knowledge and in-depth understanding of the submitted content. The survey results and evaluation outcomes further highlight a significant lack of awareness of modern AI-based tools among experienced academics, a gap that must be addressed to uphold educational standards.

Ähnliche Arbeiten