ICRE Top Research Abstracts 2023

Abstract

Editor’s Note: The following are the Top Research in Residency Education abstracts selected by the Journal of Graduate Medical Education and the Royal College of Physicians and Surgeons of Canada for the 2023 International Conference on Residency Education (ICRE).Introduction: Trainee development relies on supervisor narrative comments, yet these comments are not routinely measured for quality. The Quality of Assessment for Learning (QuAL) score has validity evidence for measuring quality in this context, but it is time intensive to score the large volume of these comments generated in medical education assessment programs. Natural language processing (NLP) models have the ability to rapidly analyze and categorize human text. We set out to develop an NLP model for applying the QuAL score to narrative supervisor comments.Methods: One thousand two-hundred and fifty EPA assessments were randomly extracted and de-identified from both McMaster’s and Saskatchewan’s emergency medicine (EM) residency training programs. These comments were put into 25 unique 100 comment surveys for rating. Twenty-five EM faculty members and 25 EM residents each filled out a survey rating comments with the QuAL score. Discrepant QuAL scores were resolved by 2 of the study authors. Eighty percent of the data was used as the training data set and 20% for the validation set. A transformer model technique was used to determine overall QuAL score as well as QuAL score sub-components.Results: All 50 raters completed the rating exercise; imperfect agreement on QuAL score were resolved by 2 study authors. The 3 QuAL score sub-components had a balanced accuracy of 0.615, 0.85, and 0.902. For the overall QuAL score, the NLP model had a balanced accuracy and top 2 accuracy of 0.52 and 0.83, respectively. The NLP model can be viewed at www.commentquality.ca.Conclusions: We have developed an NLP model for rating the quality of narrative supervisor comments in competency-based medical education (CBME) using the QuAL score. This can serve as a tool for nudging in real time, audit and feedback in faculty development initiatives, and an outcome measure for overall program evaluation in CBME.M. Spadafore1; Y. Yilmaz2; V. Rally3; M. Russell3; B. Thoma3; A. Pardhan2; L. Martin3; T. Chan2; S. Monteiro2; S. Singh4; R. Woods31University of Michigan, Ann Arbor, Michigan, United States of America; 2McMaster University, Hamilton, Ontario, Canada; 3University of Saskatchewan, Saskatoon, Saskatchewan, Canada; 4University of Ottawa, Ottawa, Ontario, CanadaIntroduction: Although postgraduate medical education programs across North America have committed increasing resources to equity, diversity, and inclusion (EDI) issues, trainees continue to experience discrimination and harassment. Excavating the mechanisms for how discrimination is perpetuated is critical to meaningfully addressing these EDI issues and aligning institutional priorities, learner experiences, and patient care. We undertook an intersectional qualitative thematic analysis exploring resident experiences of inclusion, exclusion, and discrimination in paediatrics, neurosurgery, and plastic surgery postgraduate residency training programs.Methods: Participants were recruited using purposive and snowball sampling. We conducted semi-structured interviews that explored broadly participant identities, their perceived relevance to training, and experiences of inclusion, exclusion, and discrimination. Interviews were conducted until saturation was reached. Initial coding was completed by 2 research team members with access to all transcripts. An anonymized coding summary was then shared with the research team for a second layer of deductive analysis. We used intersectionality theory as an analytic lens to identify how different social and professional identities related to the awareness, experiences, and mechanisms of discrimination.Results: Twelve participants were interviewed. Exclusion experiences were related to degree of discordance between the participant’s identities and the dominant identities in their program. For example, women of color from non-dominant religious backgrounds reported exclusion experiences along all 3 identity axes. Participants identified that their exclusion experiences stemmed from implicit assumptions and biases grounded in a patriarchal, Eurocentric medical culture, which manifested as unintentional microaggressions rather than explicitly exclusionary discourse and practice. Resident experiences were misaligned with programs’ formal EDI commitments. Many participants reported that their marginalized identities advantaged them in caring for patients with similar identities.Conclusions: Aversive racism, through social dominance, implicit bias, and in-group favoritism, mechanistically accounts for ongoing trainee discrimination experiences. Addressing these mechanisms is important for improving equity and inclusion in learning environments.J. Lam1; H. Yan2; R. Giroux3; A. Atkinson1; C. Forrest1; A. Kulkarni1; T. Martimianakis21The Hospital for Sick Children, Toronto, Ontario, Canada; 2University of Toronto, Toronto, Ontario, Canada; 3St. Michael’s Hospital, Toronto, Ontario, Canada; 3The Hospital for Sick Children, Toronto, Ontario, CanadaIntroduction: The modern-day medical education professionalism movement depends on virtues to undergo and sustain it. Competency-based curricula require that trainees exhibit qualities such as integrity, compassion, and humility in behavioral terms. Humility may be the most challenging virtue to understand and practice. This may be due to contrasting conceptualizations of humility and its role in medicine. This study aims to develop a cohesive conceptualization of physician humility in medical practice and define its functions and implications within the field.Methods: This integrative review followed Whittemore and Knafl methodology. The authors searched PubMed, Ovid MEDLINE, Web of Science, EMBASE, ERIC, and PyscINFO from database inception through July 7, 2022. English-language empirical studies, perspectives, and editorials pertaining to the conceptual aim of this investigation were included. An applied thematic analysis was conducted. Articles were uploaded to Quirkos, a computer software program to support qualitative data analysis, and analyzed using open and axial coding. Authors organized themes, identified relationships among themes, and refined them through group discussion.Results: Of 958 potential articles, 49 met the inclusion criteria. The authors identified themes within the categories of the definition of humility, functions of humility, implications of humility, and fostering of humility within the medical practice. Within the theme of the definition of humility, the authors identified what humility is (eg, honest self-disclosure, unpretentious openness, low self-focus) and what humility is not (eg, low self-esteem, self-deprecation, meekness) to provide a nuanced conceptualization. Humility is seen as integral for clinical practice, learning and curiosity, and interpersonal competence, and it may be cultivated by exposure to positive role models.Conclusions: Humility in medicine is a rich, multidimensional construct with numerous positive implications within medical practice. These findings contribute to discourse in medical education and professional development on humility in clinical practice.J. Ratelle; E. Usher; D. Suarez; A. Leep Hunderfund; A. Sawatsky; C. Matchett; A. Aragon SierraMayo Clinic, Rochester, Minnesota, United States of AmericaEditor’s Note: The following are the Top Resident Research abstracts selected by the Journal of Graduate Medical Education and the Royal College of Physicians and Surgeons of Canada for the 2023 International Conference on Residency Education (ICRE).Introduction: Climate change is a public health emergency. Health care providers need to adapt care to this reality and contribute to mitigating our carbon footprint. For many residents, education in planetary health does not exist. To understand how to incorporate planetary health into residencies, we performed a scoping review to explore the inclusion of planetary health in medical education.Methods: A search strategy developed with a health sciences librarian was run on 6 databases from inception to February 2022: Medline, Embase, APA PsycINFO, CINAHL, Global Health, and Scopus. The framework outlined by Arksey and O’Malley (2005) was implemented to select studies that described the implementation of planetary health within undergraduate and postgraduate medical education. Commentaries were included if they outlined what a potential curriculum would entail. Extracted data was grouped thematically based on competencies described, key considerations for curricular development, and anticipated barriers.Results: After screening a total of 2564 studies, 43 studies were included, of which 11 were observational studies, 21 were commentaries, 3 were reviews, and 8 were qualitative studies. Twenty-seven studies involved medical education, while 14 discussed multidisciplinary education including veterinary medicine and nursing education. Two studies discussed planetary heath education for staff physicians. Reported competencies were varied but included the dissemination of climate health literacy and its application to clinical practice. Key considerations for curricular development included longitudinal implementation, interprofessional collaboration, and experiential learning through case-based discussions or quality improvement projects. Barriers to implementation included time constraints and the lack of knowledgeable educators.Conclusions: Our scoping review highlights themes to help faculties and accreditation bodies implement and advocate for planetary health within resident education. We recommend all residency programs incorporate sustainability (ie, triple bottom line) into quality improvement projects. Opportunities for longitudinal integration into residency curriculums should also be identified.N. Séguin1; R. Selvam2; L. Zhang1; A. Lacaille-Ranger1; L. Sikora2; D. McIsaac1; H. Moloo11The Ottawa Hospital, Hospital, Ontario, Canada; 2University of Ottawa, Ottawa, Ontario, CanadaIntroduction: There is increasing evidence to suggest that supervisors exhibit different assessment behaviors when supervising off-service versus on-service residents. As programs of assessment rely on collecting robust performance data to inform high-stakes decisions about resident progress and promotion, it is important to examine the quality of such inputs. Therefore, this study compared the quality of entrustable professional activity (EPA) assessments generated for residents on-service versus off-service.Methods: This retrospective database study compared the quality of on- and off-service EPA assessments as measured by the Quality of Assessment of Learning (QuAL) score, a previously published measure of EPA quality that has demonstrated strong psychometric characteristics including reliable scores and the ability to discriminate assessments based on utility. Fifty EPA assessments (25 on- and 25 off-service) were rated for 5 different EPAs (3 non-procedural and 2 procedural) by 3 blinded raters. QuAL scores were analyzed using a factorial ANOVA.Results: Mean QuAL scores for EPAs completed on-service were significantly higher than those completed off-service with (3.57±1.07 vs 2.67±1.01, P<.001). Post-hoc analysis demonstrated that this was true for the following EPAs: resuscitating/coordinating care (P=.003); airway management/ventilation (P<.001); and managing emergency mental health conditions (P=.007).Conclusions: This study provides insights for educators about how data from on- versus off-service environments differ, which potentially can impact downstream, high-stakes decisions made by competence committees. Our study suggests that currently on-service EPA assessments are of higher quality. Future work should explore whether or not faculty development initiatives could improve assessment quality completed by physicians in other specialties. If not, consideration could also be given to the use of other assessment tools for when residents are off service.K. Endres; W. Cheung; N. Dudek; T. WoodUniversity of Ottawa, Ottawa, Ontario, CanadaIntroduction: Competency-based medical education (CBME) requires ongoing formative assessments and feedback on learners’ performance. We assessed the evolution and quality of narrative feedback provided to residents in the Surgical Foundations curriculum over a 5-year period of CBME.Methods: We performed a retrospective cross-sectional study of assessments of entrustable professional activities (EPAs) in the Surgical Foundations curriculum at Queen’s University from 2017 to 2022. We collected variables pertaining to the assessor, resident, and assessment. Two raters independently evaluated the quality of narrative feedback using the Quality of Assessment of Learning (QuAL) score (0-5).Results: A total of 3900 assessments were completed by 2108 (54.1%) attendings, 1570 (40.3%) residents/fellows, 130 (3.3%) allied health care, 42 (1.1%) medical students, and 15 (0.4%) unidentified. Most assessments were triggered by residents (3248 of 3850, 84.4%) with 509 of 3850 (13.2%) triggered by programs and only 93 of 3850 (2.4%) triggered by the assessor. Out of 3900, 2229 (57%) assessments had narrative feedback documented with a mean±SD QuAL score of 2.16±1.49. Of these 2229 assessments, 1614 (72.4%) provided evidence about the resident’s performance, 951 (42.7%) provided suggestions for improvement, and 499 of 2229 (22.4%) connected suggestions to the evidence. The narrative feedback quality improved over 5 years from 1.89±1.52 in 2017 to 2.25±1.49 in 2022 (R=0.003; P<.05). The quality of narrative feedback provided by attendings was significantly lower (2.02±1.47) than those provided by trainees (2.32±1.51) and allied health care (2.42±1.29) (P<.001). The quality of narrative feedback was higher when the resident was not entrusted (2.35±1.66 vs 2.13±1.45, P<.05), when they were assessed in an acute care (2.15±1.53) or procedural setting (2.23±1.43), and if they were being assessed on clinical management (2.52±1.70) and discharge planning decisions (2.23±1.19).Conclusions: The frequency and quality of narrative feedback on EPA assessments in Surgical Foundations curriculum was fair. These findings can facilitate faculty development.I. de Vries; S. Mann; L. McEwen; T. Phillips; B. Zevin; R. D. FernandesQueen’s University, Kingston, Ontario, CanadaIntroduction: Harassment in surgery is pervasive. Previous literature has consistently reported greater harassment for women surgeons and surgical trainees. In Canada, the intersection between gender, career stage, specialty, and experiences of harassment is less understood. The objective of our study was to survey Canadian surgeons and surgical trainees to gain a greater understanding of experiences of harassment across genders, career stages, and multiple specialties.Methods: A cross-sectional, online survey was distributed to Canadian residents, fellows, and practicing surgeons in general surgery, plastic surgery, and neurosurgery through their national society email lists, and social media posts. There were 202 survey respondents (129 staff, 60 residents, 11 fellows, 2 retirees). Chi-square tests were employed to assess differences in incidence, type, and perpetration of harassment experiences between categorical groups. The odds of experiencing harassment were calculated using logistic regression analysis.Results: Women were 3.5 times more likely to report being harassed compared to men (95% CI 1.82-6.78, P<.001). Women were more likely to be harassed by colleagues (P=.02) and patients/families (P<.001) compared to men. Residents were 2 times more likely to report being harassed compared to fellows/staff (95% CI 1.01-3.97, P=.04). There were no significant differences in self-reported harassment between the 3 specialties. There was no significant difference between current residents’ reported incidences of harassment (62.5%) and fellow/staff recollections of their experiences of harassment during residency (59.2%).Conclusions: Despite successful efforts to increase the recruitment of women in surgery, the incidence of gender-based harassment remains high. Further, present harassment prevalence remains largely unchanged from staff recollections from their residency. Despite the limitations of possible recall and non-response bias, our findings highlight the continuing need to implement systemic changes to improve surgical culture to continue to attract the best and brightest to the field.K. Jariwala1; C. Wilson2; J. Davidson2; C. Symonette3; S. De Ribaupierre3; N. Seemann21Schulich School of Medicine and Dentistry, London, Ontario, Canada; 2Children’s Hospital - LHSC, London, Ontario; 3Victoria Hospital - LHSC, London, Ontario, CanadaIntroduction: Learning and growth in residency education often require vulnerability, defined as openness to uncertainty, risk, and emotional exposure. However, expressing vulnerability can threaten a resident’s credibility and professional identity. Despite this important tension, studies examining vulnerability in residency education are limited. As such, this study aims to understand residents’ experiences of vulnerability and identify the factors that influence vulnerability in residency education.Methods: Using a constructivist grounded theory approach, individual semi-structured interviews were conducted with 15 residents from 10 different specialties. Themes were identified and their relationships were examined using constant comparative analysis.Results: Three themes represented participants’ experiences of vulnerability in residency: (1) Courage in the face of uncertainty, (2) Worth the risk?, and (3) A double-edged sword. Vulnerability was described as a relational process shaped by personal, social, and contextual influences. Residents viewed vulnerability as a valuable yet challenging experience. Given the risks and uncertainties involved, residents engaged in reflective processes before expressing vulnerability. The sociocultural context of residency represented a key modifier for engaging in vulnerability. For instance, tensions existed between expressing vulnerability to foster learning and demonstrating competence to achieve entrustment. Social agents, such as clinical teachers and peers, shaped both the experience of, and outcomes derived from, vulnerability. The influence of vulnerability on professional and personal outcomes was thus contingent on the nature of the task, and the social and environmental contexts in which it was experienced.Conclusions: Our theory represents residents as active agents in their learning who reflect on the potential benefits and risks of vulnerability. The results can inform how experiences of vulnerability can be understood and studied in residency education. These findings capture the nuance and complexity of vulnerability in residency and the need to create supportive learning environments that leverage the value of vulnerability while acknowledging its risks.H. Nichol; J. Turnnidge; N. Dalgarno; J. TrierQueen’s University, Kingston, Ontario, Canada

Ähnliche Arbeiten