CREATE Open Seminar - Assessing the Measurement Equivalence of a set of Items: Item-specific Diagnostics on an Interpretable Metric.

The characteristic of 'measuring the same thing repeatedly in the same way' makes psychological tests with equivalent items an attractive choice for one-off assessments and progress monitoring. A typical example can be found in oral reading fluency tests, where kids would typically read aloud multiple equivalently-designed short texts and words correct per minute on a text is the measured outcome. A hierarchy of measurement models formalizes the global equivalence of the item set. The traditional model comparison approach provides a binary statistical significance decision about the global level of equivalence in the item set, but interpretable local diagnostics to assess the degree of equivalence for specific items on a meaningful metric are not yet available. We introduce such item-specific effect-size diagnostics through smart use of the effect-coding identification rule to set the to-be-measured latent variable’s scale.