Generative AI tools in designing MCQs for English language examinations: Insights from Lecturers

Abstract

This study examines ChatGPT's ability to generate syntax-based sentence transformation multiple-choice questions (MCQs) using the syntactic types listed in the ReParaphrased classification framework. These include: Negation Switching (NS), Diathesis Alternation (DA), Subordination and Nesting Changes (SNC), Coordination Changes (CC), and Ellipsis (Ell). Using a quantitative approach, the researchers aim to provide personal insights into designing exam questions based on the content of B1 Empower. A statistical analysis of 120 AI-generated test items was conducted to identify the frequency and distribution of each syntactic transformation type, highlighting the favored patterns in the generated dataset. The findings suggest that ChatGPT tended to create test items using tactics with a clear pattern of transformation, such as SNC and NS, while showing less favor for tactics that require more nuanced contextual understanding, such as Ell and CC. In addition, AI could create questions quickly and effectively; however, some problems remained, including semantic distortions and awkward forms, such as double negatives or passives. This result highlights the crucial role of human intervention in proofreading and refining AI-generated questions to ensure the accuracy and relevance of the dataset items.

Ähnliche Arbeiten