James C. Impara
Amanda A. Wolkowitz
Chad W. Buckendahl
Closing the Loop: Providing Test Developers with Performance Level Descriptors so Standard Setters Can Do Their Job
Standard setting panels are often in a quandary when recommending one performance standard for an exam, and even more challenged when classifying examinees into more than two performance levels. When using test-centered methods such as a Modified Angoff (e.g., the Yes/No modification, Impara and Plake, 1997) or Bookmark (Mitzel et al., 2001) panelists must have sufficient numbers of items that can be answered correctly at each performance level to support interpretation of the classification. If there are no, or too few, items that can be answered correctly by the target examinee, the test may not accurately differentiate between examinees at the different levels. The limited number of items at these levels will also reduce classification consistency evidence. It is therefore necessary to provide test developers with the performance level descriptors early in the process and direct them to ensure that they attempt to write test items that can be answered by examinees at each of the performance levels. Research in progress will be reported with examples illustrating intended versus actual item level performance when considering standard setting results.