Letter to the Editor: Addressing test security issues pertaining to reuse of closed-book summative assessment items possibly compromised during the COVID-19 pandemic

Abstract

Letter to the EditorLetter to the Editor: Addressing test security issues pertaining to reuse of closed-book summative assessment items possibly compromised during the COVID-19 pandemicElapulli Sankaranarayanan Prakash and Balint KacsohElapulli Sankaranarayanan PrakashDepartment of Biomedical Sciences, Mercer University School of Medicine, Macon, Georgia and Balint KacsohDepartment of Biomedical Sciences, Mercer University School of Medicine, Macon, GeorgiaPublished Online:22 Jun 2020https://doi.org/10.1152/advan.00096.2020MoreSectionsPDF (38 KB)Download PDF ToolsExport citationAdd to favoritesGet permissionsTrack citations to the editor: Our discussions in the medical education community have revealed that some medical schools have administered multiple-choice (MC) summative assessments of competence unproctored, or with web-based remote proctoring not involving high-security proctoring protocols during the COVID-19 pandemic. While this may be justified under these unprecedented circumstances, a key question is what do schools that typically reuse summative test items across cohorts do to minimize threats to validity of test scores from future administrations of summative assessments of competence with possibly compromised items? The beneficiaries would take the exam under standard conditions in the future, yet their exam would be suspect. As it is rarely possible to replenish an entire summative MC test form with valid high-quality items on a yearly basis, maintaining a secure item database is a key measure (1). This is also an issue that directly relates to public accountability of medical schools.A legal defense of the validity of unproctored high-stakes online testing in the face of arguably unknowable effects of cheating would be problematic (2). As documented earlier (3), it is possible to obtain screenshots of test items with smart phones or other devices, potentially compromising test item security. Even with web-based remote monitoring, these actions may escape detection.We suggest that, during the pandemic (while test item security is potentially compromised), faculty committees select 50–70% of the questions of a given exam from the proven items (with acceptable item performance data) in their secure database, with new items to make up the remainder. Additionally, a transient 15–25% reduction in the number of summative MC items for a course held during the pandemic might be considered, if it would still allow for an adequate sampling of learning from across various aspects of a course and enable valid pass-fail adjudications in each course/program.After the pandemic, the same exam might be composed of ~70% protected, uncompromised, proven items from the secure database and ~30% newly written ones. This approach would allow comparison of cohorts of examinees, validate the exam, and maximize exam security, but might be wasteful and would not enable the institution to ascertain the necessity to discard valuable test items.Alternatively, ~25% of test items used during the pandemic under nonstandard proctoring conditions would be reused, with ~50% of items drawn from the secure database, and newly written items making up the rest. This approach enables a statistical screen for a "boost" in scores suggestive of significant irregular behavior connected to unprotected (potentially compromised) reused items at the level of a cohort of examinees, or potentially at the level of individual examinees. To be clear, this letter does not pertain to deliberately designed, authentic summative assessments of competence intended to be of the "open-book" type.DISCLOSURESNo conflicts of interest, financial or otherwise, are declared by the authors.DISCLAIMERSThe views expressed herein are strictly the personal views of the authors.AUTHOR CONTRIBUTIONSE.S.P. and B.K. conceived and designed research; E.S.P. drafted manuscript; E.S.P. and B.K. edited and revised manuscript; E.S.P. and B.K. approved final version of manuscript.REFERENCES1. Kibble JD. Best practices in summative assessment. Adv Physiol Educ 41: 110–119, 2017. doi:10.1152/advan.00116.2016. Link | ISI | Google Scholar2. Pearlman K. Unproctored internet testing: practical, legal, and ethical concerns. Ind Organ Psychol 2: 14–19, 2009. doi:10.1111/j.1754-9434.2008.01099.x.Crossref | ISI | Google Scholar3. Tonkin AL. "Lifting the carpet" on cheating in medical school exams. BMJ 351: h4014, 2015. doi:10.1136/bmj.h4014. Crossref | PubMed | Google ScholarAUTHOR NOTESCorrespondence: E. S. Prakash ([email protected]edu). Download PDF Previous Back to Top FiguresReferencesRelatedInformation CollectionsAPS Cross-Journal CollectionsCoronavirus-Related PapersWe at APS recognize that our readers and authors are working diligently to respond to the COVID-19 pandemic. Thus, articles discussing COVID-19, coronavirus, SARS, and other related research are being made freely accessible in this cross-journal collection. This collection will continue to grow as relevant content is identified and new contributions on these topics are accepted. More from this issue > Volume 44Issue 3September 2020Pages 362-362 Copyright & PermissionsCopyright © 2020 the American Physiological Societyhttps://doi.org/10.1152/advan.00096.2020PubMed32568003History Received 29 May 2020 Accepted 29 May 2020 Published online 22 June 2020 Published in print 1 September 2020 Metrics

Ähnliche Arbeiten