Comparing multidimensional to unidimensional computerized adaptive testing under two empirical scenarios: The impact of design factors

Muirne Paap, Sebastian Born, & Johan Braeken

Session 5B, 13:00 - 14:30, VIA

Research has shown the benefits of taking into account the correlation among dimensions when estimating latent trait scores in computerized adaptive tests (CATs). Multidimensional CATs (MCATs) could further improve measurement precision/decrease test length as compared to using separate unidimensional CATs for each domain, especially if domains are highly correlated.

In this study, we systematically evaluate the impact of a number of important design factors on CAT performance, using realistic example item banks. Two main scenarios are compared: health assessment (polytomous items, small to medium item bank sizes, high discrimination parameters) and educational testing (dichotomous items, large item banks, small to medium-sized discrimination parameters). Measurement efficiency is evaluated for both between-item multidimensional CATs (MCAT conditions) and separate unidimensional CATs for each latent dimension (UCAT condition). We focus on fixed-precision CATs since it is both feasible and desirable in health settings; but to date most research regarding CAT has focused on fixed-length testing. This study shows that the benefits associated with fixed-precision multidimensional CAT hold under a wide variety of circumstances.

MCAT has great potential when it comes to reducing test length and improving accuracy and precision of latent trait scores, both in health and educational measurement. We will discuss how the incremental value of MCAT depends on factors like adequate targeting, the size of the correlations, item bank size, and item parameters.

Published Sep. 5, 2018 1:53 PM - Last modified Sep. 5, 2018 1:53 PM