The Influence of ChatGPT’s Questions and Answers on Student Learning in Software Design Patterns

Abstract

This paper explores the dual pedagogical role of ChatGPT in computer engineering education, focusing on its capacity to both generate questions and provide answers that support the learning of software design patterns. Our study is motivated by the increasing use of AI-powered tools as virtual tutors, yet little is known about their ability to foster higher-order thinking in technical domains. We conducted a two-phase study to evaluate ChatGPT’s contribution to deep learning through Bloom’s Taxonomy and Socratic Questioning frameworks. In Phase 1, we applied a reverse Socratic framework to analyze 50 questions generated by ChatGPT across ten software design patterns. Each question was coded according to Bloom’s Taxonomy and Paul & Elder’s Socratic categories. Results showed that while most questions fell into the lower-to-middle cognitive levels, ChatGPT demonstrated potential for generating higher-order prompts, especially when guided by intentional prompts. In Phase 2, we experimentally assessed how answer quality influences students’ follow-up questions. Two groups of senior computer engineering students received either high-quality (cognitively rich) or low-quality (superficial) answers from ChatGPT. We found that students exposed to high-quality answers posed significantly more Higher-Order Thinking (HOT) questions and demonstrated broader Socratic coverage. Statistical analysis confirmed a strong relationship between answer quality and the cognitive depth of student inquiry. These findings suggest that ChatGPT can actively support instructional scaffolding and critical thinking when used strategically. Unlike prior work that treats AI as a reactive information source, our study highlights its proactive role in shaping how students think and inquire. We conclude with practical implications for integrating AI-generated questioning and feedback into software engineering curricula, including prompt engineering strategies and instructional scaffolding design.

Ähnliche Arbeiten