37. The Use of Artificial Intelligence in Academic Applicant Screening: A Systematic Review

Abstract

BACKGROUND: The increasing volume of applications to various higher education programs, including residency programs, has intensified the demand for efficient, unbiased screening methods. With the recent growth of artificial-intelligence (AI), these tools have emerged as a promising tool. We conducted a systematic review to assess the current landscape of AI technologies used in applicant selection, focusing on its role in screening and interview selection processes. METHODS: We conducted a systematic review using the PubMed, Medline, Embase, Scopus, and Cochrane databases. Inclusion criteria included articles with original data that utilized AI-driven methods for applicant screening or interview selection in higher education and medical training admissions. Data extracted include methodology, decision stages (e.g., initial screening, interview selection), evaluation metrics, accuracy of the AI tool, and ethical considerations. RESULTS: Of the 18 studies, 9 (50%) assessed use of AI in residency admissions, 4 (22.2%) in graduate school admissions, 3 (16.7%) in undergraduate admissions, and 2 (11.1%) in medical school admissions. A total of 61,327 applicants were analyzed. Machine learning algorithms were used to rank or score applicants based on specified criteria (n=12, 66.7%) and natural language processing programs (n=6, 33.3%) were used to assess letters of recommendation or essays. AI accuracy averaged an area under the receiver operating characteristic (AUROC) curve value of 0.84 and an under the precision-recall curve (AUPRC) value of 0.64, indicating effective performance. The studies varied in terms of AI maturity, with 11 (61.1%) in pilot stages with limited empirical validation and 7 (38.9%) reporting use of the AI tool in admissions cycles to either screen applicants or use in conjunction with human holistic review. Concerns about potential biases in algorithm design and data inputs were noted in half of the studies. CONCLUSIONS: AI is promising in high-volume applicant screening. However, the variability in AI methods and the limited transparency in design raise ethical concerns, highlighting the need for rigorous validation and standardization. Further research should establish clear guidelines for AI use in admissions to optimize fairness and efficacy while mitigating bias risks.

Ähnliche Arbeiten