CREATE Open Seminar: Benefits of Process Data for Evaluating the Differential Effectiveness of App-Based Treatments

With the growing use of digital treatments, such as learning apps, process data are increasingly available that offer valuable insights into how participants engage with these interventions. Building on this potential, our study bridges psychometric approaches and evaluation research by integrating response time modeling to refine the measurement of latent abilities, alongside differential effect analyses to explore heterogeneity in treatment outcomes. We compare different modeling strategies using data from Torkildsen et al. (2022), who developed an app-based morphological training program and evaluated its effectiveness in a randomized controlled trial involving 717 second-grade students. Our findings highlight that disengagement indicators not only enhance data quality but also provide a better understanding of who benefits most from the treatment.