Evaluating the Performance of AI-Powered Browser Extensions in Online Pharmacy Exams

Abstract

OBJECTIVE: Artificial intelligence (AI) tools present both opportunities and challenges in health professions education, particularly regarding academic integrity during online assessments. This study evaluated the functionality and performance of AI-powered browser extensions on remotely proctored pharmacy exams. METHODS: Three AI-powered browser extensions were tested on 24 required PharmD online exams administered during Fall 2024. Extensions were selected based on cost, advertised accuracy, item-type compatibility, and integration with the Canvas learning management system. Exams were completed under both proctored and unproctored conditions, depending on extension compatibility with Proctorio. AI-generated responses were graded using the same criteria applied to students. Analyses compared extension performance overall, against each other, against student averages, and by item type. RESULTS: Only 1 extension functioned in the proctored environment during midterms before being blocked by a subsequent update. Across all 24 exams, the 3 AI extensions achieved passing scores on 61.5% to 84.6% of midterm exams and 81.8% to 100% of finals, with significant differences between extensions on select midterms. Compared to students, AI scores were similar or higher on 20 of 24 (83.3%) exams. Accuracy was highest for nonreference multiple-choice items (85.6%) and lowest for reference-based short-answer questions (34.6%). CONCLUSIONS: AI browser extensions can achieve a passing score on the majority of pharmacy exams and can perform comparably to or better than students, including within some proctored environments. These findings underscore the need for assessment strategies that emphasize higher-order thinking and contextual reasoning to preserve academic integrity in the AI era.

Ähnliche Arbeiten