The Use of Generative AI Tools in Assessing Openended Text-based Examinations

Abstract

With the increasing demand and enrollment of students in higher education, academics must spend significant time providing feedback and evaluations. This study attempts to study the use of generative artificial intelligence (GenAI) tools in assessment. The primary aim is to examine how effective GenAI tools are in evaluating, especially for open-ended short text-based questions. A case study was conducted using students who follow a foundation program in computing at a leading university in Sri Lanka. A proctored online examination was set up in a controlled environment, and answers were evaluated using ChatGPT, Gemini, and manual methods. The evaluation was carried out using common rubrics in all three methods mentioned above, and the marks generated and given were compared using all three evaluation methods. Interestingly, the results show that compared to ChatGPT and Gemini, the evaluation results are much closer to the marks manually given by using manual marking. The ChatGPT and Gemini as GenAI tools generated answers that are also closer to each other. The findings prove that we can use GenAI tools effectively to evaluate structured examinations and provide results based on pre-defined rubrics. This will reduce the significant time taken for the evaluation and improve the productivity of academics.

Ähnliche Arbeiten