Generative AI in graduate bioprocess engineering exams: Is attention all students need?

Abstract

State-of-the-art large language models (LLMs) can now answer conceptual textbook questions with near-perfect accuracy and perform complex equation derivations, raising significant concerns for higher education. This study evaluates the performance of LLMs on graduate-level bioprocess engineering exams, which include multiple-choice, short-answer, and long-form questions requiring calculations. First, allowing students to use LLMs led to a 36% average score increase compared to exams taken with only textbooks and notes. Second, as students became more familiar with LLMs, their performance improved further, particularly among students with disabilities. Third, under optimized conditions on two exams, OpenAI’s GPT-4o scored approximately 70 out of 100, while more advanced models such as OpenAI o1, o3, GPT-4.5, Qwen3-235B-A22B, and DeepSeek R1 scored above 85, outperforming 96% of human test-takers. This indicates that students with access to more capable AI tools may gain an unfair advantage. Fourth, we propose guidelines for developing exam questions that are less susceptible to LLM-generated solutions. These include tasks such as interpreting graphical biological pathways, answering negatively worded conceptual questions, performing complex numerical calculations and optimizations, and solving open-ended research problems that demand critical thinking. This article calls for urgent reforms to bioprocess engineering education, advocating for the integration of LLM literacy through hands-on activities that address both practical applications and ethical considerations. ● LLMs increased student exam scores by 36%, and students improved LLM usage skills over time. ● OpenAI o1 and DeepSeek R1 surpass 96% of human test-takers. ● Advanced LLMs require subscription or expensive hardware, creating inequality. ● This article provides an AI-literacy course module.

Ähnliche Arbeiten