Composition of the panel of reference for concordance tests: Do teaching functions have an impact on examinees’ ranks and absolute scores?

Abstract

BACKGROUND: Concordance tests are designed to assess the component of uncertainty of clinical reasoning. Scoring is based on a comparison of examinees' answers with those of a panel of reference, including their variability. This allows construction of tests that are close to real clinical life, with all its complexity and ambiguity. AIM: This study was carried out to determine the effect of teaching functions of members composing the reference panels on students' scores and ranking. METHODS: A group of 80 residents in family medicine from a French University (Bobigny) completed a 72-item concordance test. The answers of two panels, each made up of 29 family physicians (teaching function versus non-teaching function), were used to generate the correction keys. RESULTS: Correlation between the sets of data obtained with the two panels is high (ICC = 0.98). Concordance scores obtained from the teaching-function panel are higher than scores obtained from the non-teaching-function panel (72.0 versus 76.3; p < 0.001). Ranking provided by the two panels was very similar. CONCLUSIONS: This legitimizes the use of non-teaching physicians on panels. Panel composition influenced absolute score values: Residents showed more concordance with their academic trainers than with community-based physicians.

Ähnliche Arbeiten