CREATE Seminar: Causal Inference and Differential Effect Analysis in (Covariate-adjusted) Group Comparisons

Evaluation studies of educational interventions often examine average effect estimates, which offer insights into the overall effectiveness of, for instance, language or math training on enhancing learning or improving achievement. The focus on average group comparisons can, however, mask important heterogeneity in intervention outcomes. Investigating differential effects for specific subgroups or even individuals offers a more nuanced understanding of who benefits most from an intervention and enables more targeted treatment decisions. To support such analyses, I will present a psychometric framework and the corresponding R package EffectLiteR (e.g., Mayer et al., 2016; Sengewald & Mayer, 2024), which addresses various threats to causal inference in group comparisons. Most notably, EffectLiteR can adjust for baseline group differences and incorporates different latent variable models in the effect analysis to correct for measurement error. The pre-specified or user-defined effect estimates provide insights into the variation of intervention outcomes in relation to participant characteristics (e.g., prior achievements or individual predispositions). In addition to summarizing the psychometric framework, the talk will demonstrate the implementation of the approach in an empirical example with large-scale assessment data of the German National Education Panel Study (NEPS; Blossfeld & Roßbach, 2019). Specifically, an analysis of differential effects for evaluating the impact of private tutoring on reading literacy will be presented (Heyne, Gnambs, & Sengewald, 2024). The demonstration will showcase the implementation of the approach with the user-friendly graphical interface of EffectLiteR and the interpretation of different effect estimates.