Who Made Thee a Judge? Selecting and Training Participants for Standard Setting

Abstract

This chapter does not attempt to address participant selection and training for each standard-setting method and context, but provides a general framework for selection and training that should apply to a variety of test-centered and examinee-centered standard-setting methods. To provide a concrete frame of reference, much of the discussion is specific to the Angoff method. Although the shortcomings of the Angoff method have drawn increased criticism in recent years, it has been the most frequently used method in standard setting (Berk, 1986; Impara & Plake, 1997; Kane, 1994; Maurer, Alexander, Callahan, Bailey, & Dambrot, 1991; Sireci & Biskin, 1992; R.L.Smith & J.K.Smith, 1988). Its familiarity makes it a convenient example with which to illustrate the systematic approach to the selection and training of participants proposed in this chapter. To facilitate generalization to different contexts, the authors discuss examples of participant selection and training in two very different standard setting contexts: public education, as illustrated by the National Assessment of Educational Progress (NAEP), and professional credentialing, as illustrated by a certification program administered by the American Registry of Radiologic Technologists (ARRT).

Ähnliche Arbeiten