Nonlinear item-level moderation in measurement models: Exploring the relationship between product and process data

Maria Bolsinova & Dylan Molenaar

Session 2A, 12:45 - 14:15, HAGEN 2

When educational tests are presented in a computerised form, it is feasible to not only record the product of the response process (i.e., response accuracy or response choice), but also the characteristics of the process itself. For each combination of the person and the item different values of many additional variables could be recorded: response times, confidence ratings, verbally reported response processes, number of actions in interactive items, number of item clicks, number of eye fixations on the areas of interest, inspection times, response changes, certainty scores, or physiological measures. These variables can be included as moderators in the measurement models for the ability of interest such that one can investigate whether the probability of a correct response is related to the value of the moderator and whether there is an interaction effect between the measured ability and the moderator. For moderators that vary across persons but not across items (e.g., traditional moderators like age or SES) there is a wide variety of multi-group, linear, nonlinear and nonparametric methods for investigating these effects. Item-level moderators have received much less attention in the latent variable model literature. The development of methods to test for interactions between item-level moderators and the ability has recently started to evolve across similar lines as in traditional moderation models. That is, approaches have been proposed that require categorisation (Partchev & De Boeck, 2012) of the item-level moderators and models have been proposed by specifying linear functions between the intercept and slope parameter of the measurement model and the item-level moderator (Bolsinova, Tijmstra, & Molenaar, 2017; Goldhammer, Steinwascher, Kroehne,& Naumann, 2017).

However, parametric nonlinear and nonparametric models for indicator-level moderation are lacking while such approaches are valuable in exploring the exact form of the relationship between the moderator and the parameters of the model. The assumption of linearity of item-level moderation might be violated in practice, and using linear models might lead to invalid conclusions about the relationship between the parameters of the measurement model and the item-specific moderator. For instance, one might conclude that the intercept increases with the values of the moderator (e.g., that slower responses on a science test are more often correct), while it might be that it increases only up to some value of the moderator and decreases after that value, or that the increase is not linear. Therefore, we propose to model the relationship between the item-specific moderator and the parameters of the measurement model in a more flexible way. In this presentation, parametric nonlinear and nonparametric item-level moderation methods are developed. In a simulation study we demonstrate the viability of these methods. In addition, the methods are applied to a real dataset pertaining to arithmetic ability in which the main and interaction effects of response time are investigated.

Published Sep. 5, 2018 1:39 PM - Last modified Sep. 5, 2018 1:39 PM