Ronny Scherer

• Tondeur, Jo; Howard, Sarah K.; Bucchiarone, Antonio; Scherer, Ronny & Braak, Johan van (2026). How we study what we study: A methodological review of 50 years of educational technology research. Computers & Education. ISSN 0360-1315. doi: 10.1016/j.compedu.2026.105760. Full text in Research Archive Show summary

  Research in educational technology has been extensively mapped in terms of what has been studied, but far less attention has been paid to how it has been studied. This paper reports a methodological review of fifty years of research in Computers & Education, covering a corpus of 5,516 articles published between 1976 and 2026, with the principal findings based on the post-1996 full-text corpus. Using an AI-assisted analysis, applied to full texts and abstracts, we traced the evolution of research paradigms, approaches, and methods across the journal's full publication history. The principal findings are based on the post-1996 full-text corpus. Pre-1996, full-text machine-readable files were not available for analysis. Analysis of pre-1996 abstracts was possible and are included to provide context for trends observed post-1996. The findings show that quantitative approaches and questionnaire-based data collection have constituted the field's sustained methodological centre of gravity. Within this dominant pattern, two shifts stand out: 1) the rise of systematic reviews as a recognised approach, and 2) a computational turn. The range of methods used in research has grown steadily, suggesting a gradual broadening of the field's repertoire. These patterns are only visible across multiple decades. Findings reveal that quantitative dominance is not a recent development but a persistent structural feature, and that the computational surge is better understood as a second wave following a peak in the mid-2000s. These findings have implications for what kinds of knowledge educational technology research is equipped to produce. The study argues that understanding a field's methodological history is a prerequisite for extending it.

• Patsis, Panagiotis; Campos, Diego G. & Scherer, Ronny (2026). Educational inequality across countries: An integrative data analysis of TIMSS 2023 data. Studies in Educational Evaluation. ISSN 0191-491X. 91. doi: 10.1016/j.stueduc.2026.101683. Full text in Research Archive Show summary

  Socioeconomic disparities in educational achievement remain a central challenge for education systems. This study examines cross-national variation in the relation between socioeconomic status (SES) and Grade 8 mathematics achievement, using data from TIMSS 2023. Employing an integrative data analysis approach, we estimated within-school, between-school, and contextual SES effects for 43 countries and applied random-effects meta-regression to examine system-level moderators. Results show that SES is a consistent predictor of achievement within schools, while between-school SES effects were substantially larger, indicating pronounced compositional inequalities. The pooled contextual SES effect was large (ES2 = 0.85, 95% CI [0.74, 0.95]) and highly heterogeneous across countries (τ² = 0.11; Q(42) = 642.40, p < .001), ranging from negligible in Nordic systems to very large in more stratified systems. Random-effects meta-regression analyses revealed that income inequality and between-school segregation strongly predicted contextual SES effects, whereas higher levels of democratic governance were associated with weaker effects.

• Teo, Timothy; Scherer, Ronny; Fung, Artemis Sze Kok & Fung, Cynthia Sze Ling (2026). Factors Associated with Students’ Adoption of Artificial Intelligence Technology in Tertiary Education: A Meta-Analytic Review. Educational Research Review. ISSN 1747-938X. doi: 10.1016/j.edurev.2026.100804. Full text in Research Archive Show summary

  This meta-analytic review synthesizes evidence on factors shaping post-secondary students’ adoption of artificial intelligence (AI) through a multilevel meta-analysis of 233 correlations from 32 studies (N = 16,977). A three-level model with correlated effects was identified as the optimal approach, yielding moderate positive weighted average correlations for individual (r = .57), contextual (r = .53), and technological factors (r = .50). No single category dominates adoption. Among specific predictors, usage intentions emerged as the strongest predictor (r = .64). Interestingly, perceived risks and trust showed weaker associations. Within the three broad categories of factors, the correlations exhibited large overall heterogeneity (I-squared >98%), which could be partly explained by the diversity of constructs and the students’ experience with AI. For practice, integration of AI should target specific constructs (e.g., learning objectives, intentions) and adapt to students’ experience levels. For research, the field over-relies on traditional technology acceptance models (e.g., TAM, UTAUT) and neglects AI-specific factors (e.g., anthropomorphism, ethics). Addressing this gap is important for advancing theory and evidence-based policy.

• Pietsch, Marcus; Eryilmaz, Nurullah; Strietholt, Rolf; Scherer, Ronny; Nascimento, Matheus Monteiro & Abels, Simone [Show all 7 contributors for this article] (2026). Endangering Just Sustainabilities: An IPD Meta-Analysis on the Social Stratification of Youth Environmental Attitudes in 65 Countries. Journal of Environmental Psychology. ISSN 0272-4944. 112. doi: 10.1016/j.jenvp.2026.103085. Full text in Research Archive Show summary

  This study examines global socioeconomic inequality in environmental attitudes among young people. Using novel data from an international study of nearly 700,000 individuals across 65 countries, we combine a two-stage individual participant data (IPD) meta-analysis with a machine learning (Meta-CART) approach to examine cross-national variation in inequality in environmental attitudes and its associations with the macrostructural dimensions of capacity, distribution, and institutional openness. Higher socioeconomic status (SES) is consistently associated with stronger environmental preservation attitudes in both primary and secondary school. However, the strength of this relationship varies across countries. Revealing a structural paradox, press freedom and gross domestic product (GDP) interact to moderate SES disparities: the largest gaps appear in nations with both high institutional openness and affluence. Contrary to expectations, GDP and academic freedom do not exert consistent linear effects. The results suggest that national conditions shape the social distribution of environmental attitudes and that institutional enablers of knowledge may reinforce environmental engagement among advantaged youth, challenging the equity aspirations of sustainability education.

• Fütterer, Tim; König, Lars; Campos, Diego Gonzalez; Scherer, Ronny; Zitzmann, Steffen & Hecht, Martin (2026). Fast-Track Your Abstract Screening: Mastering ASReview for Accelerating Abstract Screening and Evaluating Decisions From Automatic-Screening Methods. Advances in Methods and Practices in Psychological Science (AMPPS). ISSN 2515-2459. 9(2), p. 1–36. doi: 10.1177/25152459261442150. Full text in Research Archive Show summary

  Research syntheses, such as systematic reviews and meta-analyses, are crucial for synthesizing research to support evidence-based decision-making. However, the abstract-screening phase, during which researchers evaluate titles and abstracts for inclusion, is highly time-consuming and often results in cognitive biases and fatigue. To address these challenges, machine-learning-assisted tools, particularly those using active learning, have gained prominence. One such tool is Active Screening Review (ASReview), an open-source software for semiautomating title and abstract screening in systematic reviews. ASReview incorporates user feedback to prioritize relevant studies, reducing screening time and improving efficiency. Despite its potential, many researchers remain uncertain about integrating ASReview into their workflows and making evidence-based decisions regarding the tool’s configuration, training, and stopping criteria. In this tutorial, we provide a step-by-step guide to using ASReview, including practical examples from psychological research. We demonstrate the software’s application in two use cases: screening unlabeled abstracts using active learning and verifying results from automated-screening methods. In the tutorial, we also offer evidence-based recommendations for selecting stopping rules to balance sensitivity and efficiency. We also outline strategies for prescreening, data-set preparation, model setup, and progress monitoring to ensure that researchers can maximize the tool’s benefits while maintaining scientific rigor. By offering evidence-based guidance at each stage of the process for practitioners without coding skills, in this tutorial, we aim to help researchers harness artificial-intelligence-aided screening to enhance the quality and efficiency of research syntheses across disciplines.

• Scherer, Ronny; Campos, Diego G.; Fütterer, Tim & Siddiq, Fazilat (2026). Unraveling gender disparities in teachers’ technological pedagogical and content knowledge: A large-scale meta-analytic review. Educational Research Review. ISSN 1747-938X. 50. doi: 10.1016/j.edurev.2026.100765. Full text in Research Archive Show summary

  This large-scale meta-analytic review investigates gender disparities in pre- and in-service K-12 teachers' Technological Pedagogical and Content Knowledge (TPACK) self-efficacy and performance. Synthesizing 915 effect sizes from 102 primary studies and 420 teacher samples (N = 681,745 teachers) in 70 countries via multilevel meta-analyses, we found small but significant gender differences favoring male teachers in TPACK self-efficacy (d = −0.17), particularly in technological knowledge (d = −0.31) and technological pedagogical knowledge (d = −0.14). However, no significant gender disparities were observed in TPACK performance (d = −0.08). Our findings revealed considerable heterogeneity in these effects across samples, studies, countries, measures, and over time, which could be partly explained by factors such as educational level, country-level gender inequality, and economic development. These results do not support male teachers’ advantage in all aspects of technology-related teaching competencies. Our meta-analytic review further implies that gender disparities in TPACK vary across contexts or over time.

• Fütterer, Tim; Campos, Diego Gonzalez; Gfrörer, Thomas; Lavelle-Hill, Rosa; Murayama, Kou & Scherer, Ronny (2025). AI tools for systematic literature reviews and meta-analyses in educational psychology: An overview and a practical guide. Learning and Individual Differences. ISSN 1041-6080. 126. doi: 10.1016/j.lindif.2025.102849. Full text in Research Archive Show summary

  The rapid growth of research literature has made systematic reviews and meta-analyses increasingly time-consuming, limiting their utility in fast-evolving fields such as educational psychology. Artificial intelligence (AI) tools have enormous potential to streamline these processes, yet their adoption remains limited due to usability issues and a lack of systematic guidance. Out of 282 tools that we compiled from overviews that listed AI tools for research syntheses, we filtered a subset of 7 AI tools that met quality standards, such as transparency and accessibility. These tools were evaluated for their potential to support systematic reviews and meta-analyses in educational psychology. Our review highlights the tools' strengths, limitations, and ethical considerations for their responsible use by providing practical guidance and coding information.

• Rohatgi, Anubha; Scherer, Ronny & Hatlevik, Ove Edvard (2025). Digitale forskjeller i elevers digitale kompetanse og algoritmiske tenkning. In Rohatgi, Anubha & Hatlevik, Ove Edvard (Ed.), Søkelys på digital læring: Sekundæranalyser fra ICILS 2023. Cappelen Damm Forskning/NOASP. ISSN 9788202905682. p. 141–174. doi: 10.23865/cdf.271.ch5. Full text in Research Archive Show summary

  I mange land utgjør gapet mellom elever med og elever uten tilgang til teknologi en betydelig utfordring som forsterker sosiale ulikheter. Selv om dette ikke er like fremtredende i Norge, finnes det fortsatt digitale skiller, og disse har ikke blitt vesentlig redusert de siste ti årene. Begrepet digitale forskjeller omhandler forskjeller i tilgang til, bruk av og ferdigheter i digital teknologi. I dette kapittelet utforsker vi digitale forskjeller ved å undersøke sammenheng mellom elevers bakgrunn og elevenes prøveresultater i digital kompetanse (DK) og algoritmisk tenkning (AT). Det er en rekke variabler som har betydning for digitale skiller, for eksempel elevens bakgrunn, inkludert kjønn og språk som snakkes hjemme, familiens status, som foreldrenes utdanningsnivå og yrkesstatus, samt tilgang til digitale ressurser. Hver enkelt variabel har betydning for elevprestasjonen, men sammen kan de også forsterke forskjellene mellom elevenes resultater. Vi undersøker styrken i sammenhengene mellom de fem ulike variablene og utforsker hvorfor spesifikke elevgrupper har ulike resultater i DK og AT. Vi benytter flernivåanalyser som synliggjør interseksjonalitet som kjennetegner elever som risikerer å bli digitalt ekskluderte. Våre analyser av ICILS 2023-dataene viser signifikante relasjoner mellom de ulike variablene, som kjønn, språklig bakgrunn, familiens status og tilgang til digitale ressurser og elevenes prestasjoner i DK og AT. For de norske dataene ble det også avdekket tegn på interseksjonalitet. Elever som ikke snakker norsk hjemme, som mangler stabil internettforbindelse hjemme og har foreldre med lavt utdanningsnivå og yrker av lav status identifiseres som digitalt sårbare.

• Campos, Diego Gonzalez; Koskinen, Antti; Munkácsy, Balázs; Rolfe, Victoria; Patsis, Panagiotis & Tóth, Edit [Show all 12 contributors for this article] (2025). Educational inequalities in Europe: A scoping review of longitudinal studies in K-12 education. Studies in Educational Evaluation. ISSN 0191-491X. 87. doi: 10.1016/j.stueduc.2025.101523. Full text in Research Archive Show summary

  Educational inequalities are a critical challenge. Understanding how these disparities emerge, evolve, and can be addressed requires longitudinal evidence. However, most existing research syntheses rely on cross-sectional data. In this scoping review, we map longitudinal research on educational inequalities among K–12 students in Europe. Drawing on 161 studies published between 1986 and 2024, we examined how disparities related to socioeconomic status, migration or ethnic background, gender, geographical location, or their intersections have been studied over time. Academic achievement and socioeconomic status were the most studied outcome and inequality dimensions, while inequalities in socio-emotional outcomes and geographical location received less attention. Four research clusters were also identified in the literature: individual psychological determinants, mechanisms of inequality, developmental patterns of inequality, and structural determinants of educational transitions. Overall, the findings underscore the heterogeneity of empirical approaches to longitudinal research on educational inequality in Europe and highlight gaps for future research.

• Aydın, Burak; Fokkema, Marjolein; Eryilmaz, Nurullah; Muijs, Daniel; Scherer, Ronny & Pietsch, Marcus (2025). How school leadership and innovation shape instructional pathways to student achievement across nations: Evidence from multilevel structural equation modeling and decision tree analysis. Studies in Educational Evaluation. ISSN 0191-491X. 87. doi: 10.1016/j.stueduc.2025.101521. Full text in Research Archive Show summary

  Educational leadership, innovation and teaching play essential roles in shaping student achievement. However, extant literature primarily has relied on linear modelling approaches and has not focused on substantively testing a theory. The present study employs multilevel structural equation modelling (ML-SEM) and multilevel decision trees (MLM trees) to investigate associations between school leadership, team innovation, cognitive activation and student achievement using PISA-TALIS 2018 linked data across seven countries: Australia; Colombia; Czech Republic; Denmark; Georgia; Malta; and Türkiye. The ML-SEM findings indicated no significant indirect effects from leadership on achievement. The MLM trees highlighted country-specific patterns in associations between school leadership, innovation and student achievement, revealing potential nonlinear relationships. These findings suggest that the relationship between leadership, instructional practices and achievement is highly context-dependent. The study contributes to the literature by offering a comparative analysis of ML-SEM and MLM trees, highlighting traditional linear models’ limitations in educational research.

• Backfisch, Iris; Scherer, Ronny; Schneider, Jürgen & Lachner, Andreas (2025). How Valid, Really? A Meta-Analysis of the Validity Evidence of Technological Pedagogical and Content Knowledge (TPACK) Self-Report Assessments. Educational Research Review. ISSN 1747-938X. 49. doi: 10.1016/j.edurev.2025.100718. Full text in Research Archive Show summary

  Technological Pedagogical and Content Knowledge (TPACK) plays a central role for teacher training and measuring teachers’ preparedness for technology-enhanced teaching. Hence, crafting a validity argument of the popular TPACK self-assessments is key to facilitating evidence-based practices in these areas. However, the existing body of validity evidence abounds in divergent findings, for instance on the factor structure of TPACK self-reports across pre- and in-service teacher samples. To synthesize this evidence and capture its heterogeneity, we conducted a systematic review with a meta-analysis. First, we examined the internal validity of TPACK by analyzing its factor structure using meta-analytic structural equation and network modeling. Second, we compared two theoretical models describing the relations among the TPACK dimension—the transformative and integrative models—against the data. Third, we assessed the external validity of self-reported TPCK by examining its correlation with actual technology use in teaching. Our findings revealed two distinguishable dimensions of TPACK for both pre- and in-service teacher samples, which represented the technological and non-technological dimensions. The integrative model provided an overall better fit to the data. Moreover, TPACK self-reports were moderately correlated with technology use (r̄ s = 0.23–0.49) and exhibited substantial heterogeneity. Our results underscore the importance of generating and considering both internal and external validity evidence when applying TPACK assessments to generate implications for educational research and practice.

• Maehler, Débora B.; Martin, Silke; Gorges, Julia & Scherer, Ronny (2025). Determinants of cognitive skills in adulthood: Age cohort patterns. International Journal of Lifelong Education. ISSN 0260-1370. 44(2), p. 131–152. doi: 10.1080/02601370.2024.2390070. Full text in Research Archive Show summary

  The current study examines change and stability of cognitive skills across life stages and delves into the factors contributing to changes in cognitive skills. Specifically, we examine the associations between cognitive skills and individual characteristics and contextual factors in a large sample of German adults aged 16–65 years (N = 2,430; PIAAC-L data). Across all age cohorts, cognitive skills were predicted mainly by a person’s educational background, but they were also associated with factors related to socialisation, lifestyle and personality. The findings indicate that specific factors influence cognitive skills at different life stages. For example, personality was related to cognitive skills solely in the two younger age cohorts, whereas the impacts of contextual factors were most pronounced in the middle cohort.

• Emslander, Valentin; Holzberger, Doris; Ofstad, Sverre Berg; Fischbach, Antoine & Scherer, Ronny (2025). Teacher-Student Relationships and Student Outcomes: A Systematic Second-Order Meta-Analytic Review. Psychological bulletin. ISSN 0033-2909. 151(3), p. 365–397. doi: 10.1037/bul0000461. Full text in Research Archive Show summary

  Teacher–student relationships (TSRs) play a vital role in establishing a positive classroom climate and promoting positive student outcomes. Several meta-analyses have suggested significant correlations between positive TSRs and, for example, academic achievement, motivation, executive functions, and well-being, as well as between negative TSRs that result in behavior problems or bullying. These meta-analyses have differed substantially in TSR–outcome relationships, moderators, and methodological quality, thus complicating the interpretation of these findings. In this preregistered systematic review of meta-analyses plus original second-order meta-analyses (SOMAs), we aimed to (a) synthesize the meta-analytic evidence on relations between TSRs and student outcomes, (b) map influential moderators of these relations, and (c) assess the methodological quality of the meta-analyses. We synthesized over 70 years of educational research across 26 meta-analyses encompassing 119 meta-analytic effect sizes based on approximately 2.64 million prekindergarten and K–12 students. We conducted several three-level SOMAs and found that TSRs had similar large significant relations with eight clusters of student outcomes: academic achievement, academic emotions, appropriate student behavior, behavior problems, executive functions and self-control, motivation, school belonging and engagement, and well-being. The link with bullying was only marginally significant. Our moderator analyses suggested a larger TSR–outcome link for middle and high school students. Although more recent meta-analyses fulfilled more methodological quality criteria, these differences were not associated with TSR–outcome relations. These results map the field of TSR research; present their relations, moderators, and methodological quality in meta-analyses; and show how TSRs are equally important for a wide range of student outcomes and samples.

• Simonseier, Bianca A.; Kampmann, Kristin; Staub, Jacqueline & Scherer, Ronny (2025). The effects of programming interventions in early childhood: A meta-analysis. Learning and Individual Differences. ISSN 1041-6080. 120. doi: 10.1016/j.lindif.2025.102699. Full text in Research Archive Show summary

  While recent meta-analyses confirm the positive effects of programming interventions in school and university students, it is still unclear whether teaching programming in early childhood settings is similarly effective. Our meta-analysis clarifies this issue by synthesizing the effects of programming interventions for 3- to 7-year-old children. Moreover, we propose and employ a taxonomy for instructional approaches commonly used in programming interventions to systematically analyze the relevance of instructional design. The results from 25 early childhood studies and 82 effect sizes showed a medium overall effect of d = +0.73, 95 % CI [0.51, 0.96]. Programming interventions effectively enhanced programming knowledge, computational thinking, mathematics knowledge, and cognitive skills. Our findings highlight the effectiveness of unplugged over screen-based activities and interventions, including robotics, over those using other program outputs. The results provide educators with valuable information about instructional methods. The taxonomy can guide researchers and educators in implementing instructional strategies when teaching programming.

• Scherer, Ronny & Emslander, Valentin (2025). Utilizing primary study quality in meta-analyses in psychology: A step-by-step tutorial. Psychological methods. ISSN 1082-989X. doi: 10.1037/met0000751. Full text in Research Archive Show summary

  Evaluating the quality of primary studies is a key step in meta-analyses in psychology. This step is aimed at reducing the risk of bias and establishing the validity of the inferences drawn from the meta-analytic findings. However, the extant body of research offers little guidance on how to represent and incorporate primary study quality (PSQ) in meta-analyses, and some common procedures, such as creating sum scores from a set of quality indicators, often lack the backing from measurement models. Addressing these issues, we present a tutorial that guides researchers in their analytic decisions and approaches to represent and incorporate PSQ in their meta-analyses. Specifically, we describe, review, and illustrate approaches to (a) represent PSQ by single or multiple quality indicators or aggregated scores; (b) examine the moderator effects of PSQ; and (c) test the sensitivity of moderator effects to PSQ. We illustrate these approaches and present a step-by-step tutorial with analytic code for researchers’ guidance. We also encourage meta-analysts to take a measurement perspective on representing PSQ if multiple quality indicators are aggregated into a quality score. Moreover, we argue for conducting moderator sensitivity analyses to obtain more evidence on the impact of PSQ in a meta-analysis.

• Veletic, Jelena & Scherer, Ronny (2025). The Nordic perspective on the association between distributed leadership and teacher job satisfaction: A multilevel SEM approach. International Journal of Educational Research. ISSN 0883-0355. 130. doi: 10.1016/j.ijer.2025.102559. Full text in Research Archive Show summary

  In this study, we address the lack of robust empirical evidence on the association between distributed leadership and teacher job satisfaction with the work environment in the Nordic countries by examining this relationship at the teacher and school levels, for both teachers and principals. We also explore how teachers’ shared perceptions of distributed leadership are associated with individual job satisfaction (i.e., contextual effects), surpassing the limitations of the single-level analyses. Using data from 11,788 teachers in 654 schools who participated in the Teaching and Learning International Survey (TALIS) 2018 in Denmark, Finland, Norway, and Sweden, we found a strong positive association between teacher job satisfaction with the work environment and teacher-reported distributed leadership at the school level, and a moderately strong association at the teacher level. In contrast, we found a negative contextual effect which suggested that stronger distributed leadership at the school level was associated with lower satisfaction of individual teacher in Norway and Finland. Additionally, principal-reported distributed leadership was weakly, positively associated with teacher job satisfaction with the work environment only in Sweden. This study shows mixed results: at the teacher level, stronger distributed leadership has potential to enhance job satisfaction, while at the school level, it can be linked to lower individual teacher satisfaction as shown in Norway and Finland. Our study further highlights the distinction between teacher- and principal-reported distributed leadership, as well as individual- and shared-teacher perceptions, and their potential differential effects on job satisfaction.

• Scherer, Ronny & Campos, Diego Gonzalez (2025). Meta-analyzing correlation matrices in the presence of hierarchical effect size multiplicity. Research Synthesis Methods. ISSN 1759-2879. 16(6), p. 828–858. doi: 10.1017/rsm.2025.10027. Full text in Research Archive Show summary

  To synthesize evidence on the relations among multiple constructs, measures, or concepts, meta-analyzing correlation matrices across primary studies has become a crucial analytic approach. Common meta-analytic approaches employ univariate or multivariate models to estimate a pooled correlation matrix, which is subjected to further analyses, such as structural equation modeling. In practice, meta-analysts often extract multiple correlation matrices per study from various samples, study sites, labs, or countries, thus introducing hierarchical effect size multiplicity into the meta-analytic data. However, this feature has largely been ignored when pooling correlation matrices for meta-analysis. To contribute to the methodological development in this area, we describe a multilevel, multivariate, and random-effects modeling approach, which pools correlation matrices meta-analytically and, at the same time, addresses hierarchical effect size multiplicity. Specifically, it allows meta-analysts to test various assumptions on the dependencies among random effects, aiding the selection of a meta-analytic baseline model. We describe this approach, present four working models within it, and illustrate them with an example and the corresponding R code.

• Adnan, Müge; Tondeur, Jo; Scherer, Ronny & Siddiq, Fazilat (2024). Profiling teacher educators: ready to prepare the next generation for educational technology use? Technology, Pedagogy and Education. ISSN 1475-939X. 33(4), p. 527–544. doi: 10.1080/1475939X.2024.2322481. Full text in Research Archive
• Acar, Selcuk; Lee, Lindsay Ellis & Scherer, Ronny (2024). A Reliability Generalization of the Torrance Tests of Creative Thinking-Figural. European Journal of Psychological Assessment. ISSN 1015-5759. doi: 10.1027/1015-5759/a000819. Full text in Research Archive
• Scherer, Ronny; Siddiq, Fazilat & Nilsen, Trude (2024). The potential of international large-scale assessments for meta-analyses in education. Large-scale Assessments in Education. 12(1). doi: 10.1186/s40536-024-00191-1. Full text in Research Archive Show summary

  Meta-analyses and international large-scale assessments (ILSA) are key sources for informing educational policy, research, and practice. While many critical research questions could be addressed by drawing evidence from both of these sources, meta- analysts seldom integrate ILSAs, and current integration practices lack methodological guidance. The aim of this methodological review is therefore to synthesize and illus- trate the principles and practices of including ILSA data in meta-analyses. Specifically, we (a) review four ILSA data inclusion approaches (analytic steps, potential, challenges); (b) examine whether and how existing meta-analyses included ILSA data; and (c) provide a hands-on illustrative example of how to implement the four approaches. Seeing the need for meta-analyses on educational inequalities, we situated the review and illustration in the context of gender differences and socioeconomic gaps in stu- dent achievement. Ultimately, we outline the steps meta-analysts could take to utilize the potential and address the challenges of ILSA data for meta-analyses in education.

• König, Lars; Zitzmann, Steffen; Fütterer, Tim; Campos, Diego Gonzalez; Scherer, Ronny & Hecht, Martin (2024). An evaluation of the performance of stopping rules in AI-aided screening for psychological meta-analytical research. Research Synthesis Methods. ISSN 1759-2879. doi: 10.1002/jrsm.1762. Full text in Research Archive Show summary

  Several AI-aided screening tools have emerged to tackle the ever-expanding body of literature. These tools employ active learning, where algorithms sort abstracts based on human feedback. However, researchers using these tools face a crucial dilemma: When should they stop screening without knowing the proportion of relevant studies? Although numerous stopping rules have been proposed to guide users in this decision, they have yet to undergo comprehen- sive evaluation. In this study, we evaluated the performance of three stopping rules: the knee method, a data-driven heuristic, and a prevalence estimation technique. We measured performance via sensitivity, specificity, and screening cost and explored the influence of the prevalence of relevant studies and the choice of the learning algorithm. We curated a dataset of abstract collections from meta-analyses across five psychological research domains. Our findings revealed performance differences between stopping rules regarding all perfor- mance measures and variations in the performance of stopping rules across different prevalence ratios. Moreover, despite the relatively minor impact of the learning algorithm, we found that specific combinations of stopping rules and learning algorithms were most effective for certain prevalence ratios of rel- evant abstracts. Based on these results, we derived practical recommendations for users of AI-aided screening tools. Furthermore, we discuss possible impli- cations and offer suggestions for future research.

• Ye, Wangqiong; Scherer, Ronny & Blömeke, Sigrid (2024). Teachers’ and principals’ perceptions of school emphasis on academic success: measurement invariance, agreement, and relations to student achievement. Large-scale Assessments in Education. 12. doi: 10.1186/s40536-024-00207-w. Full text in Research Archive Show summary

  A school’s emphasis on academic success (SEAS) is a crucial aspect of school climate that can be rated by different informants, such as teachers or principals, and with respect to their perception of different groups’ emphasis on academic success, such as parents or students. We know little about whether there is agreement between these ratings. Utilizing Israeli Trends in International Mathematics and Science Study (TIMSS) 2019 data and multilevel structural equation modeling, we examined how teachers and principals rate teachers’, students’, and parents’ emphasis on academic success and how these ratings are related to student achievement. We found that (a) ratings are invariant across informants; (b) teachers and principals agree on parents’ and students’ yet not teachers’ emphasis; (c) teacher ratings explain more variation in student achievement than principal ratings. Since SEAS ratings diverge by informants and relations to student achievement, researchers need to craft validity arguments in their studies.

• Amland, Tonje; Grande, German; Scherer, Ronny; Lervåg, Arne & Melby-Lervåg, Monica (2024). Cognitive Factors Underlying Mathematical Skills: A Systematic Review and Meta-Analysis. Psychological bulletin. ISSN 0033-2909. doi: 10.1037/bul0000457. Full text in Research Archive Show summary

  In understanding the nature of mathematical skills, the most influential theories suggest that mathematical cognition draws on different systems: numerical, linguistic, spatial, and general cognitive skills. Studies show that skills in these areas are highly predictive of outcomes in mathematics. Nonetheless, the strength of these relations with mathematical achievement varies, and little is known about the moderators or relative importance of each predictor. Based on 269 concurrent and 174 longitudinal studies comprising 2,696 correlations, this meta-analysis summarizes the evidence on cognitive predictors of mathematical skills in children and adolescents. The results showed that nonsymbolic number skills (often labeled approximate number sense) correlate significantly less with mathematical achievement than symbolic number skills and that various aspects of language relate differently to mathematical outcomes. We observed differential predictive patterns for arithmetic and word problems, and these patterns only partly supported the theory of three pathways—quantitative, linguistic, and spatial—for mathematical skills. Concurrently, nonsymbolic number and phonological skills were weak but exclusive predictors of arithmetic skills, whereas nonverbal intelligence quotient (IQ) predicted word problems only. Only symbolic number skills predicted both arithmetic and word problems concurrently. Longitudinally, symbolic number skills, spatial ability, and nonverbal IQ predicted both arithmetic and word problems, whereas language comprehension was important for word problem solving only. As in the concurrent data, nonsymbolic number skill was a weak longitudinal predictor of arithmetic skills. We conclude that the candidates to target in intervention studies are symbolic number skills and language comprehension. It is uncertain whether the two other important predictors, nonverbal IQ and spatial skills, are actually malleable.

• Campos, Diego Gonzalez; Fütterer, Tim; Gfrörer, Thomas; Lavelle-Hill, Rosa; Murayama, Kou & König, Lars [Show all 9 contributors for this article] (2024). Screening Smarter, Not Harder: A Comparative Analysis of Machine Learning Screening Algorithms and Heuristic Stopping Criteria for Systematic Reviews in Educational Research. Educational Psychology Review. ISSN 1040-726X. 36(1). doi: 10.1007/s10648-024-09862-5. Full text in Research Archive Show summary

  Abstract Systematic reviews and meta-analyses are crucial for advancing research, yet they are time-consuming and resource-demanding. Although machine learning and natural language processing algorithms may reduce this time and these resources, their performance has not been tested in education and educational psychology, and there is a lack of clear information on when researchers should stop the reviewing process. In this study, we conducted a retrospective screening simulation using 27 systematic reviews in education and educational psychology. We evaluated the sensitivity, specificity, and estimated time savings of several learning algorithms and heuristic stopping criteria. The results showed, on average, a 58% ( SD = 19%) reduction in the screening workload of irrelevant records when using learning algorithms for abstract screening and an estimated time savings of 1.66 days ( SD = 1.80). The learning algorithm random forests with sentence bidirectional encoder representations from transformers outperformed other algorithms. This finding emphasizes the importance of incorporating semantic and contextual information during feature extraction and modeling in the screening process. Furthermore, we found that 95% of all relevant abstracts within a given dataset can be retrieved using heuristic stopping rules. Specifically, an approach that stops the screening process after classifying 20% of records and consecutively classifying 5% of irrelevant papers yielded the most significant gains in terms of specificity ( M = 42%, SD = 28%). However, the performance of the heuristic stopping criteria depended on the learning algorithm used and the length and proportion of relevant papers in an abstract collection. Our study provides empirical evidence on the performance of machine learning screening algorithms for abstract screening in systematic reviews in education and educational psychology.

• Scherer, Ronny (2024). Is the Technology Acceptance Model just old wine in new wineskins? Exploring issues for further model development. Journal of University Teaching and Learning Practice. doi: 10.53761/var55x96. Full text in Research Archive Show summary

  For about three decades, the Technology Acceptance Model (TAM) has been a crucial source of information for understanding the factors and processes that influence the integration of technology. The TAM has been extensively studied in various educational settings, involving students and teachers across different academic levels and specific technologies. The diverse range of applications has generated a vast body of evidence affirming the model’s relevance in education. However, despite its success, the research on the TAM still has some unresolved issues and limitations. In this commentary, I address these existing concerns and describe further areas for improvement, including the model itself, the measurement of TAM constructs, and existing gaps and conflicting findings in the current body of evidence. By shedding light on these issues, I hope to guide future research endeavors that build upon and refine the TAM.

• Hübner, Nicolas; Fahrbach, Thorsten; Lachner, Andreas & Scherer, Ronny (2023). What predicts students’ future ICT literacy? Evidence from a large-scale study conducted in different stages of secondary school. Computers & Education. ISSN 0360-1315. 203. doi: 10.1016/j.compedu.2023.104847. Full text in Research Archive
• Cabellos, Beatriz; Siddiq, Fazilat & Scherer, Ronny (2023). The moderating role of school facilitating conditions and attitudes towards ICT on teachers' ICT use and emphasis on developing students’ digital skills. Computers in Human Behavior. ISSN 0747-5632. 150. doi: 10.1016/j.chb.2023.107994. Full text in Research Archive
• Scherer, Ronny; Siddiq, Fazilat; Howard, Sarah K. & Tondeur, Jo (2023). Gender divides in teachers' readiness for online teaching and learning in higher education: Do women and men consider themselves equally prepared? Computers & Education. ISSN 0360-1315. 199, p. 1–16. doi: 10.1016/j.compedu.2023.104774. Full text in Research Archive Show summary

  During the last years, the “Great Online Transition” has brought to light large variation in teachers' readiness for online teaching and learning (OTL). Drawing from an international sample of 731 higher-education teachers, we examined gender differences in OTL readiness as a source of this variation. Currently, in the field of OTL, better evidence is needed to understand the associated dimensions and effects of gender on teachers’ experiences and perceptions of readiness, to provide better support and professional learning opportunities in transitioning to an online and blended practice. To provide such evidence, we first evaluated the measurement bias in the readiness measures and found support for strong gender invariance. Second, we quantified the gender differences in readiness levels: Women reported higher readiness for cognitive activation practices (d = +0.15); men reported higher self-efficacy in technological content knowledge (d = −0.20). These gender differences were small, varied across readiness constructs, and were due to a gender gap in OTL experience. Third, construct associations involving perceived institutional support were weaker for women. To improve the quality, robustness, and validity of the respective evidence, we argue that studying gender divides in OTL readiness needs to consider measurement bias, OTL experience, and construct associations.

• Scherer, Ronny; Siddiq, Fazilat; Howard, Sarah K. & Tondeur, Jo (2023). The more experienced, the better prepared? New evidence on the relation between teachers’ experience and their readiness for online teaching and learning. Computers in Human Behavior. ISSN 0747-5632. 139. doi: 10.1016/j.chb.2022.107530. Full text in Research Archive Show summary

  In the present study, we tested the common assumption that teachers with more experience consider themselves better prepared for online teaching and learning (OTL). Utilizing the data from a survey of 366 higher-education teachers from Portugal at the beginning of the COVID-19 pandemic in 2020, we performed structural equation modeling to quantify the experience-readiness relationship. The survey contained an assessment of teachers' OTL readiness which was measured by their perceptions of the institutional support, online teaching presence, and TPACK self-efficacy. In contrast to the linearity assumption “the more experienced, the better prepared”, we found robust evidence for a curvilinear relationship. Teachers’ readiness for OTL increased first and then decreased with more experience—this applied especially to the self-efficacy dimension of readiness. Further analyses suggested that the experience-readiness relationship does not only exist at the level of aggregated constructs but also at the level of indicators, that is, specific areas of knowledge, teaching, and support. We argue that both novice and experienced teachers in higher education could benefit from experience-appropriate, pedagogical, and content-related support programs for OTL.

• Fütterer, Tim; Scherer, Ronny; Scheiter, Katharina; Stürmer, Kathleen & Lachner, Andreas (2023). Will, skills, or conscientiousness: What predicts teachers' intentions to participate in technology-related professional development? Computers & Education. ISSN 0360-1315. 198, p. 1–17. doi: 10.1016/j.compedu.2023.104756. Full text in Research Archive Show summary

  To unfold the potential of learning with technology in classrooms, teachers must be prepared to integrate it meaningfully and with high instructional quality. Professional development (PD) is an important avenue for preparing in-service teachers to integrate technology into their teaching. Whereas existing research indicates that teachers’ motivation predicts classroom technology use and participation in PD, knowledge about other predictors (e.g., technology-related knowledge and personality) is scarce. This scarcity is surprising because technology-related prior knowledge and, for example, conscientiousness are especially important in systems in which teachers choose PD voluntarily. In this study, we analyzed unique data from 321 in-service teachers whose schools were randomized to receive one-to-one technology. Using sequential linear and nonlinear regression, we examined the extent to which teachers’ will, skills, and conscientiousness simultaneously (rather than separately) predicted their intentions to participate in technology-related PD. Controlling for important personal and contextual characteristics, we found robust evidence that the perceived utility of technology in classrooms was a stronger predictor of participation intentions than technology-related knowledge and conscientiousness. Contrary to existing assumptions, our findings illustrate that teachers should still be made aware of the utility of technology for instruction to strengthen their intentions to participate in technology-related PD.

• Karimi, Roya; Mallah, Narmeen; Scherer, Ronny; Rodríguez-Cano, Rubén & Takkouche, Bahi (2023). Sleep quality as a mediator of the relation between depression and chronic pain: a systematic review and meta-analysis. British Journal of Anaesthesia. ISSN 0007-0912. doi: 10.1016/j.bja.2023.02.036. Full text in Research Archive
• Tondeur, Jo; Howard, Sarah K.; Scherer, Ronny & Siddiq, Fazilat (2023). Untangling the Great Online Transition: A network model of teachers’ experiences with online practices. Computers & Education. ISSN 0360-1315. 203. doi: 10.1016/j.compedu.2023.104866. Full text in Research Archive Show summary

  The Covid-19 pandemic provided a critical opportunity to understand how teachers experienced and perceived online practices, as they transitioned to emergency remote teaching. However, how these experiences relate, and how they should be addressed together to support teachers and schools, remains unclear. Therefore, the main aim of the current study is to examine associations among teachers’ experiences and perceptions of online teaching and learning. The data were captured at the beginning of the COVID pandemic in 2020 through an Journal Pre-proof online questionnaire, when teachers were transitioning to emergency remote teaching. Data included in the analysis comprises 222 secondary school teachers' responses to open-ended questions, analysed through a combined method of thematic and network analysis. The results indicate that perceptions and experiences must be studied from a system of variables that interact in determining the success or failure of online teaching and learning. Findings can contribute to decision making about how these variables assemble and relate in different ways to design more targeted support for teachers' future online and blended teaching practices.

• Campos, Diego Gonzalez; Cheung, M. W. & Scherer, Ronny (2023). A Primer on Synthesizing Individual Participant Data Obtained From Complex Sampling Surveys: A Two-Stage IPD Meta-Analysis Approach. Psychological methods. ISSN 1082-989X. doi: 10.1037/met0000539. Full text in Research Archive Show summary

  The increasing availability of individual participant data (IPD) in the social sciences offers new possibilities to synthesize research evidence across primary studies. Two-stage IPD meta-analysis represents a framework that can utilize these possibilities. While most of the methodological research on two-stage IPD meta-analysis focused on its performance compared with other approaches, dealing with the complexities of the primary and meta-analytic data has received little attention, particularly when IPD are drawn from complex sampling surveys. Complex sampling surveys often feature clustering, stratification, and multistage sampling to obtain nationally or internationally representative data from a target population. Furthermore, IPD from these studies is likely to provide more than one effect size. To address these complexities, we propose a two-stage meta-analytic approach that generates model-based effect sizes in Stage 1 and synthesizes them in Stage 2. We present a sequence of steps, illustrate their implementation, and discuss the methodological decisions and options within. Given its flexibility to deal with the complex nature of the primary and meta-analytic data and its ability to combine multiple IPD sets or IPD with aggregated data, the proposed two-stage approach opens up new analytic possibilities for synthesizing knowledge from complex sampling surveys.

• Siddiq, Fazilat; Røkenes, Fredrik Mørk; Lund, Andreas & Scherer, Ronny (2023). New kid on the block? a conceptual systematic review of digital agency. Education and Information Technologies: The Official Journal of the IFIP Technical Committee on Education. ISSN 1360-2357. doi: 10.1007/s10639-023-12038-3. Full text in Research Archive Show summary

  The digitalisation of education has been emphasised over the last two decades, affecting and developing both the theories and practices of teaching and learning. Considering these developments, the need for addressing teachers’ and students’ digital competence has gained ground. In the last few years, the concept of digital agency has been noted in empirical and theoretical research – broadly focusing on the integration of information and communication technologies (ICT) in educa- tion. Nonetheless, how digital agency is understood and how it relates to the more established concept of digital competence remains unclear. To address this gap, we conducted a conceptual systematic review and examined the current state of the knowledge on digital agency. Specifically, we review the conceptualisations of digital agency, its underlying theoretical frameworks, and how it relates to digital competence and similar concepts. Database searches resulted in 32 publications, with the majority published in education and design journals and stemming from Scandinavian countries. Our findings show that out of 32 studies, only one aimed at defining digital agency explicitly. Nevertheless, for the last three years, digital agency has been more frequently used in the body of literature, emphasising the design and transformation of teaching and learning with technology towards a ‘new normal’ considering the post-pandemic era and lessons learned. Reviewing the ex- tant body of knowledge on digital agency, we review the strengths and weaknesses of the concept and compare it to the more established notion of digital competence. Finally, we discuss implications for policy, research, and practice in education.

• Campos, Diego Gonzalez & Scherer, Ronny (2023). Digital gender gaps in Students’ knowledge, attitudes and skills: an integrative data analysis across 32 Countries. Education and Information Technologies: The Official Journal of the IFIP Technical Committee on Education. ISSN 1360-2357. 29, p. 655–693. doi: 10.1007/s10639-023-12272-9. Full text in Research Archive
• Scherer, Ronny & Nilsen, Trude (2022). Accountability for the future? International testing and future directions. In Tierney, Robert J.; Rizvi, Fazal & Erkican, Kadriye (Ed.), International Encyclopedia of Education (4th Edition). Elsevier. ISSN 9780128186299. p. 324–340. doi: 10.1016/B978-0-12-818630-5.09060-6. Full text in Research Archive
• Talić, Irma; Scherer, Ronny; Marsh, Herbert W.; Greiff, Samuel; Möller, Jens & Niepel, Christoph (2022). Uncovering everyday dynamics in students' perceptions of instructional quality with experience sampling. Learning and Instruction. ISSN 0959-4752. 81. doi: 10.1016/j.learninstruc.2022.101594. Full text in Research Archive
• Scherer, Ronny (2022). Analyzing International Large-Scale Assessment Data with a Hierarchical Approach. In Nilsen, Trude; Stancel-Piątak, Agnes & Gustafsson, Jan-Eric (Ed.), International Handbook of Comparative Large-Scale Studies in Education. Springer. ISSN 9783030382988. doi: 10.1007/978-3-030-38298-8_59-1. Full text in Research Archive Show summary

  International large-scale assessments in education (ILSAs) follow complex sampling designs and ultimately create hierarchical data structures with students nested in classrooms, classrooms in schools, schools in regions, etc. To describe adequately key issues in education, such as socioeconomic gaps in academic achievement or the relations among school characteristics and student achievement using ILSA data, researchers need to consider the hierarchical data structure in statistical models. Multilevel modeling is one approach to account for such hierarchies and consider variables at different levels of analysis. This chapter provides an overview of the prominent multilevel modeling approaches to analyzing ILSA data, illustrates and discusses their strengths and weaknesses, and highlights the key methodological decisions researchers have to take in this context. The first part reviews the current practices of multilevel modeling in secondary analyses of ILSA data. This rapid systematic review is followed by a second part in which we present, illustrate, and discuss multilevel modeling approaches, including multilevel regression, multilevel structural equation models, and multilevel mixture models. Next to model estimation and fit evaluation, we review key issues associated with the multilevel modeling of ILSA data and focus on handling plausible values, multigroup and incidental multilevel data structures, and weighting. Our chapter provides worked examples showcasing the potential of multilevel modeling for ILSA data analysis.

• Lopez-Pedersen, Anita; Mononen, Riikka; Aunio, Pirjo Annika; Scherer, Ronny & Melby-Lervåg, Monica (2022). Improving Numeracy Skills in First Graders with low performance in early numeracy: A Randomized Controlled Trial. Remedial and special education. ISSN 0741-9325. doi: 10.1177/07419325221102537. Full text in Research Archive Show summary

  Children with low performance in early numeracy are at risk of facing learning difficulties in mathematics, but few trials have examined how this can be ameliorated. A total of 120 first-grade children (Mage = 6.4 years) were randomly assigned to an intervention or a control condition. The 14-week intervention targeted early numeracy skills and was delivered in small groups three times a week. Immediately after the initial 8-week intervention phase, moderate and positive effects were found on early numeracy (d = 0.19), word problem solving (d = 0.41), and approximate number sense (d = 0.35). However, only the effects on word problems were significant, and all effects disappeared after the children undertook a second 6-week intervention phase. Overall, results indicate that (a) early numeracy skills are malleable in low-performing children, but (b) frequent and long-term interventions are needed for the positive effects to last.

• Lotz, Christin; Scherer, Ronny; Greiff, Samuel & Sparfeldt, J. R. (2022). g's little helpers – VOTAT and NOTAT mediate the relation between intelligence and complex problem solving. Intelligence. ISSN 0160-2896. 95. doi: 10.1016/j.intell.2022.101685. Full text in Research Archive
• Teig, Nani; Scherer, Ronny & Olsen, Rolf Vegar (2022). A systematic review of studies investigating science teaching and learning: over two decades of TIMSS and PISA. International Journal of Science Education. ISSN 0950-0693. 44(12), p. 2035–2058. doi: 10.1080/09500693.2022.2109075. Full text in Research Archive Show summary

  A great number of studies have investigated science teaching and learning (STL) using data from the Trends in International Mathematics and Science Study (TIMSS) and Programme for International Student Assessment (PISA). Nevertheless, there is little effort to synthesise these studies. Reviewing the status of research on STL, including the trends, approaches, and findings is crucial for identifying research gaps that require greater attention. Consequently, this review synthesises empirical studies investigating STL using TIMSS and PISA. First, we mapped their characteristics based on the aims, data, STL measures, and research approaches. Second, patterns of the findings were summarized by focusing on the (1) relationships between STL and student outcomes; (2) factors that explain STL variation; and (3) patterns of STL. Most studies examined STL related to inquiry activities and their relationships to student outcomes. Their findings were largely inconsistent and could be traced back to different ways STL was conceptualized and measured. This review calls for the studies examining TIMSS and PISA data to enhance the quality of research approaches and clarity in reporting them. It also encourages researchers to reflect upon the knowledge gained from harnessing these data to examine STL and discusses the challenges and opportunities that lie ahead.

• Emslander, Valentin & Scherer, Ronny (2022). The Relation Between Executive Functions and Math Intelligence in Preschool Children: A Systematic Review and Meta-Analysis. Psychological bulletin. ISSN 0033-2909. 148(5-6), p. 337–369. doi: 10.1037/bul0000369. Full text in Research Archive
• Mittal, Oleksandra; Scherer, Ronny & Nilsen, Trude (2022). Assessing the evidence for the comparability of socioeconomic status between students with and without immigrant background in Norway and Sweden. Large-scale Assessments in Education. 10(1). doi: 10.1186/s40536-022-00132-w. Full text in Research Archive Show summary

  The prerequisite for meaningful comparisons of educational inequality indicators across immigration status is the comparability of socioeconomic status (SES) measures. The Programme for International Student Assessment (PISA) uses its index of economic, social, and cultural status (ESCS) to provide insights into the problems of inequality across students’ socioeconomic and immigration statuses. However, missing evidence regarding the comparability of the ESCS index or its components across students with and without immigrant background challenges the accuracy of empirical inferences. Our study sheds light on the comparability of the index of household possessions (HOMEPOS) across immigration status in Norway and Sweden—two countries that continue to be two largest recipients of immigration flows among their Nordic neighbours. We tested the PISA 2018 HOMEPOS scale for the overall measurement invariance and possible differential item functioning (DIF) across three student groups with first-generation, second-generation, or no immigrant background. Several HOMEPOS items exhibited DIF within these countries. Moreover, we examined how four strategies to deal with DIF items may affect the inferences regarding educational inequalities across immigration status. The strength of the HOMEPOS–achievement association was sensitive to the choice of approach for 15-year-old immigrant students, while it remained stable and moderate for native students. Our findings encourage researchers using the HOMEPOS scale to consider the invariance testing to avoid measurement bias and provide robust evidence characterizing immigrant achievement gaps.

• Scherer, Ronny & Campos, Diego Gonzalez (2022). Measuring those who have their minds set: An item-level meta-analysis of the implicit theories of intelligence scale in education. Educational Research Review. ISSN 1747-938X. 37. doi: 10.1016/j.edurev.2022.100479. Full text in Research Archive Show summary

  Fixed and growth mindsets represent implicit theories about the nature of one's abilities or traits. The existing body of research on academic achievement and the effectiveness of mindset interventions for student learning largely relies on the premise that fixed and growth mindsets are mutually exclusive. This premise has led to the common practice in which measures of one mindset are reversed and then assumed to represent the other mindset. Focusing on K-12 and university students (N = 27328), we tested the validity of this practice via a comprehensive item-level meta-analysis of the Implicit Theories of Intelligence Scale (ITIS). By means of meta-analytic structural equation modeling and network analysis, we examined (a) the ITIS item-item correlations and their heterogeneity across 32 primary studies; (b) the factor structure of the ITIS, including the distinction between fixed and growth mindset; and (c) moderator effects of sample, study, and measurement characteristics. We found positive item-item correlations within the sets of fixed and growth mindset items, with substantial between-study heterogeneity. The ITIS factor structure comprised two moderately correlated mindset factors ( = 0.63–0.65), even after reversing one mindset scale. This structure was moderated by the educational level and origin of the student sample, the assessment mode, and scale modifications. Overall, we argue that fixed and growth mindsets are not mutually exclusive but correlated constructs. We discuss the implications for the assessment of implicit theories of intelligence in education.

• Emslander, Valentin; Levy, Jessica; Scherer, Ronny & Fischbach, Antoine (2022). Value-added Scores Show Limited Stability over Time in Primary School. PLOS ONE. 17(12). doi: 10.1371/journal.pone.0279255. Full text in Research Archive Show summary

  Value-added (VA) models are used for accountability purposes and quantify the value a teacher or a school adds to their students’ achievement. If VA scores lack stability over time and vary across outcome domains (e.g., mathematics and language learning), their use for high-stakes decision making is in question and could have detrimental real-life implications: teachers could lose their jobs, or a school might receive less funding. However, school-level stability over time and variation across domains have rarely been studied together. In the present study, we examined the stability of VA scores over time for mathematics and language learning, drawing on representative, large-scale, and longitudinal data from two cohorts of standardized achievement tests in Luxembourg ( N = 7,016 students in 151 schools). We found that only 34–38% of the schools showed stable VA scores over time with moderate rank correlations of VA scores from 2017 to 2019 of r = .34 for mathematics and r = .37 for language learning. Although they showed insufficient stability over time for high-stakes decision making, school VA scores could be employed to identify teaching or school practices that are genuinely effective—especially in heterogeneous student populations.

• Konstantinidou, Evi & Scherer, Ronny (2022). Teaching with technology: A large-scale, international, and multilevel study of the roles of teacher and school characteristics. Computers & Education. ISSN 0360-1315. 179. doi: 10.1016/j.compedu.2021.104424. Full text in Research Archive
• Stedal, Kristin; Broomfield, Catherine; Hay, Phillipa; Touyz, Stephen & Scherer, Ronny (2021). Neuropsychological functioning in adult anorexia nervosa: A meta-analysis. Neuroscience and Biobehavioral Reviews. ISSN 0149-7634. 130, p. 214–226. doi: 10.1016/j.neubiorev.2021.08.021. Full text in Research Archive Show summary

  Several studies have conceptualized neuropsychological dysfunction as part of the core pathology and defining behaviors seen in the eating disorder anorexia nervosa (AN). The aim of the current review was to synthesize the differences in neuropsychological test performance between individuals with AN and healthy controls, quantify and explain their heterogeneity. The search and screening procedures resulted in fifty studies that comprised 186 neuropsychological test results. Utilizing random-effects meta-analyses, the results revealed evidence for significant, moderate underperformance in people with AN in overall neuropsychological functioning (g = -0.43, 95 % CI [-0.50, -0.36]). Weighted mean effect sizes ranged from g = -0.53 for visuospatial abilities to g = -0.10 for planning. Study and participant characteristics, including body mass index (BMI) and age, had significant moderator effects, especially on executive function, memory, and visuospatial abilities. The findings from the current study provide an extensive and comprehensive overview of the possible impairments in neuropsychological functioning in adult patients diagnosed with AN.

• Stedal, Kristin; Scherer, Ronny; Touyz, Stephen; Hay, Phillipa & Broomfield, Catherine (2021). Neuropsychological functioning in young anorexia nervosa: A meta-analysis. Journal of Child Psychology and Psychiatry. ISSN 0021-9630. p. 1–10. doi: 10.1111/jcpp.13562. Full text in Research Archive Show summary

  Background Cognitive inflexibility and compulsive behaviours are commonly observed in patients with anorexia nervosa (AN) and are often considered to be caused by impairments in executive functioning and visuospatial processing. Despite an increasing number of young individuals presenting with AN, there is a lack of meta-analytic evidence on the neuropsychological functioning in this population. Our primary aim was to review and synthesize the differences in neuropsychological test performance between young people with AN and healthy controls, and to explore potential moderators. Methods A database search following PRISMA guidelines of MEDLINE, PsycINFO, ISI Web of Science and Epistemonikos was conducted. Hedges' g served as an effect size indicating the standardized mean differences. We utilized mixed-effects meta-regression and machine learning meta-analyses to identify relevant moderators. Results Sixteen studies met the inclusion criteria with a total of 1333 participants (665 with AN) and 59 effect sizes. Random-effects meta-analyses demonstrated a small and insignificant difference between young individuals with AN and controls (urn:x-wiley:00219630:media:jcpp13562:jcpp13562-math-0001 = −0.144, 95% CI [−0.328, 0.041]) in overall neuropsychological functioning. However, the difference was significant for the cognitive domains of memory, working memory and visuospatial abilities. Moderator and machine-learning analyses revealed a stronger negative effect in older participants and moderator effects of country and assessment quality. Conclusions Our findings highlight the significant impact of age on neuropsychological test performance in patients with AN. There is a need for a more widespread inclusion of potentially confounding variables in primary studies as well as instruments specifically targeted at younger populations.

• Qazi, Atika; Hasan, Najmul; Abayomi-Alli, Olusola; Hardaker, Glenn; Scherer, Ronny & Sarker, Yeahia [Show all 8 contributors for this article] (2021). Gender differences in information and communication technology use & skills: a systematic review and meta-analysis. Education and Information Technologies: The Official Journal of the IFIP Technical Committee on Education. ISSN 1360-2357. doi: 10.1007/s10639-021-10775-x. Full text in Research Archive
• Greiff, Samuel; Graesser, Arthur C.; Iliescu, Dragos; Rouet, Jean-François; Scheiter, Katharina & Scherer, Ronny [Show all 8 contributors for this article] (2021). PIAAC Cycle 2 assessment framework: Adaptive problem solving, The Assessment Frameworks for Cycle 2 of the Programme for the International Assessment of Adult Competencies. OECD Publishing. ISSN 9789264946064. p. 155–203. doi: 10.1787/4bc2342d-en. Full text in Research Archive
• Kampa, Nele; Scherer, Ronny; Saß, Steffani & Schipolowski, Stefan (2021). The relation between science achievement and general cognitive abilities in large-scale assessments. Intelligence. ISSN 0160-2896. 86. doi: 10.1016/j.intell.2021.101529. Full text in Research Archive Show summary

  Although large-scale assessments (LSA) of school achievement claim to measure domain-specific achievement, they have been criticized for primarily measuring domain-general abilities. Numerous studies provide evidence that LSA of mathematical achievement as well as verbal achievement cover both general cognitive abilities (GCA) and domain-specific achievement dimensions. We extend previous research by analyzing a standards-oriented and literacy-oriented LSA in the domain of science to determine the relation of these two assessment types with domain-general abilities. While literacy-oriented assessments focus on the knowledge and skills students need to meet the demands of modern societies, standards-oriented assessments focus on national educational standards and curricula. A sample of 1722 students worked on three assessments: (a) the PISA scientific literacy assessment; (b) a standards-oriented assessment based on the German National Educational Standards in biology, chemistry, and physics developed by the Institute for Educational Quality Improvement (IQB); and (c) a GCA test. Comparisons of competing structural models showed that models differentiating between domain-specific achievement and GCA best represented the structure of the assessments. Furthermore, standards-oriented and literacy-oriented LSAs in science shared common variance with GCA but also comprised specific variance. In addition to a factor representing students' GCA, we identified a science literacy-oriented and two standards-oriented factors. Relations with school grades in various STEM and non-STEM subjects were mixed and only partly provided evidence for the specificity of science LSAs. Our findings are important for understanding and interpreting results of LSAs in the contexts of GCA and science. We discuss our outcomes with respect to educational monitoring practices.

• Backfisch, Iris; Scherer, Ronny; Siddiq, Fazilat; Lachner, Andreas & Scheiter, Katharina (2021). Teachers' technology use for teaching: Comparing two explanatory mechanisms. Teaching and Teacher Education : An International Journal of Research and Studies. ISSN 0742-051X. 104. doi: 10.1016/j.tate.2021.103390. Full text in Research Archive
• Teo, Timothy; Unwin, Siobhan; Scherer, Ronny & Gardiner, Veronica (2021). Initial teacher training for twenty-first century skills in the Fourth Industrial Revolution (IR 4.0): A scoping review. Computers & Education. ISSN 0360-1315. 170. doi: 10.1016/j.compedu.2021.104223. Full text in Research Archive
• Dörendahl, Jan; Scherer, Ronny; Greiff, Samuel; Martin, Romain & Niepel, Christoph (2021). Dimensional Comparisons in the Formation of Domain-Specific Achievement Goals. Motivation Science. ISSN 2333-8113. 7(3), p. 306–318. doi: 10.1037/mot0000203. Full text in Research Archive
• Damsa, Crina I.; Langford, Malcolm; Scherer, Ronny & Uehara, Dan (2021). Teachers’ agency and online education in times of crisis. Computers in Human Behavior. ISSN 0747-5632. 121(106793). doi: 10.1016/j.chb.2021.106793. Full text in Research Archive Show summary

  This study examines academic teachers' agency and emergency responses, prompted by the physical closure of universities and university colleges due to the COVID-19 crisis. The pandemic-related lockdown accelerated the digitalization of education and forced teachers to adjust their teaching. A theoretical model is elucidated, in which teachers' agency is understood as the willingness to engage in iterational, practical-evaluative, projective, and transformative action despite the existence of practical, personal, and institutional constraints. We explored the nature and degree of this agency through a survey of university teachers in Norway in the first month of the lockdown. Teachers attempted to create learning environments that facilitated knowledge transfer and interaction and sought to solve problems through self-help and support from colleagues and network, although many struggled with insufficiently developed digital competence and institutional support. Latent profile and qualitative analyses revealed different clusters of teacher responses, from strong resistance to online teaching through to transformation of teaching practices. Qualitative analyses unveiled different expressions of teachers' agency, both ostensible and occlusive, whereby action was shaped by constraining circumstances. These findings can inform future studies of online teaching, indicate the conditions for development of teachers’ digital competence, and illustrate the challenges brought about by crises.

• Nilsen, Trude; Scherer, Ronny & Blömeke, Sigrid (2021). Hva fremmer et innovativt miljø i skolen, og hvilken betydning har det for læreres undervisning? In Björnsson, Julius Kristjan (Eds.), Hva kan vi lære av TALIS 2018? Gode relasjoner som grunnlag for læring. Cappelen Damm Akademisk. ISSN 9788202718800. p. 35–55. doi: 10.23865/noasp.123. Full text in Research Archive Show summary

  What encourages an innovative environment in schools and what does that imply for teaching? An innovative environment is important for a school’s ability to adapt to societal changes and for students’ ability to cope with future society. It is therefore crucial to find out what promotes an innovative environment in a school and what consequences an innovative environment has for teaching and learning. Certain aspects of teaching are especially important for students to learn how to cope with a society where, for instance, critical thinking and problem solving are required and where diversity is increasing. Such teaching practices include cognitive activation and inclusive practices. Using TALIS 2018, we employ two-level (teachers and schools) structural equation modelling to examine both direct and indirect effects of distributed leadership on teaching practices via an innovative environment. Results show that distributed leadership has a medium strong relationship to an innovative environment and that an innovative environment has a medium strong relationship to teaching practices. An innovative environment mediates the effect of distributed leadership on teaching practices. Overall, our study has implications for policy and practice as it finds that school leaders who employ distributed leadership can help promote an innovative environment and that an innovative environment in turn may encourage the type of teaching students need to acquire skills that are important in the 21st century.

• Scherer, Ronny; Siddiq, Fazilat & Sánchez-Scherer, Bárbara (2021). Some Evidence on the Cognitive Benefits of Learning to Code. Frontiers in Psychology. 12. doi: 10.3389/fpsyg.2021.559424. Full text in Research Archive Show summary

  Computer coding—an activity that involves the creation, modification, and implementation of computer code and exposes students to computational thinking—is an integral part of today's education in science, technology, engineering, and mathematics (STEM) (Grover and Pea, 2013). As technology is advancing, coding is becoming a necessary process and much-needed skill to solve complex scientific problems efficiently and reproducibly, ultimately elevating the careers of those who master the skill. With many countries around the world launching coding initiatives and integrating computational thinking into the curricula of higher education, secondary education, primary education, and kindergarten, the question arises, what lies behind this enthusiasm for learning to code? Part of the reasoning is that learning to code may ultimately aid students' learning and acquiring of skills in domains other than coding. Researchers, policy-makers, and leaders in the field of computer science and education have made ample use of this argument to attract students into computer science, bring to attention the need for skilled programmers, and make coding compulsory for students. Bill Gates once stated that “[l]earning to write programs stretches your mind, and helps you think better, creates a way of thinking about things that I think is helpful in all domains” (2013). Similar to the claims surrounding chess instruction, learning Latin, video gaming, and brain training (Sala and Gobet, 2017), this so-called “transfer effect” assumes that students learn a set of skills during coding instruction that are also relevant for solving problems in mathematics, science, and other contexts. Despite this assumption and the claims surrounding transfer effects, the evidence backing them seems to stand on shaky legs—a recently published paper even claimed that such evidence does not exist at all (Denning, 2017), yet without reviewing the extant body of empirical studies on the matter. Moreover, simply teaching coding does not ensure that students are able to transfer the knowledge and skills they have gained to other situations and contexts—in fact, instruction needs to be designed for fostering this transfer (Grover and Pea, 2018). In this opinion paper, we (a) argue that learning to code involves thinking processes similar to those in other domains, such as mathematical modeling and creative problem solving, (b) highlight the empirical evidence on the cognitive benefits of learning computer coding that has bearing on this long-standing debate, and (c) describe several criteria for documenting these benefits (i.e., transfer effects). Despite the positive evidence suggesting that these benefits may exist, we argue that the transfer debate has not yet to be settled.

• Philipsen, Brent; Tondeur, Jo; Scherer, Ronny; Pynoo, Bram & Zhu, Chang (2021). Measuring institutional support for online and blended learning professional development: validating an instrument that examines teachers’ perceptions. International Journal of Research & Method in Education. ISSN 1743-727X. doi: 10.1080/1743727X.2021.1926973. Full text in Research Archive
• Blömeke, Sigrid; Nilsen, Trude & Scherer, Ronny (2021). School Innovativeness Is Associated With Enhanced Teacher Collaboration, Innovative Classroom Practices, and Job Satisfaction. Journal of Educational Psychology. ISSN 0022-0663. doi: 10.1037/edu0000668. Full text in Research Archive
• Chen, Chia-Wen; Wang, Wen-Chung; Mok, Magdalena Mo Ching & Scherer, Ronny (2021). A Lognormal Ipsative Model for Multidimensional Compositional Items. Frontiers in Psychology. 12, p. 1–19. doi: 10.3389/fpsyg.2021.573252. Full text in Research Archive Show summary

  Compositional items – a form of forced-choice items – require respondents to allocate a fixed total number of points to a set of statements. To describe the responses to these items, the Thurstonian item response theory (IRT) model was developed. Despite its prominence, the model requires that items composed of parts of statements result in a factor loading matrix with full rank. Without this requirement, the model cannot be identified, and the latent trait estimates would be seriously biased. Besides, the estimation of the Thurstonian IRT model often results in convergence problems. To address these issues, this study developed a new version of the Thurstonian IRT model for analyzing compositional items – the lognormal ipsative model (LIM) – that would be sufficient for tests using items with all statements positively phrased and with equal factor loadings. We developed an online value test following Schwartz’s values theory using compositional items and collected response data from a sample size of N = 512 participants with ages from 13 to 51 years. The results showed that our LIM had an acceptable fit to the data, and that the reliabilities exceeded 0.85. A simulation study resulted in good parameter recovery, high convergence rate, and the sufficient precision of estimation in the various conditions of covariance matrices between traits, test lengths and sample sizes. Overall, our results indicate that the proposed model can overcome the problems of the Thurstonian IRT model when all statements are positively phrased and factor loadings are similar.

• Scherer, Ronny (2020). The case for good discipline? Evidence on the interplay between disciplinary climate, socioeconomic status, and science achievement from PISA 2015. In Frønes, Tove Stjern; Pettersen, Andreas; Radišić, Jelena & Buchholtz, Nils (Ed.), Equity, Equality and Diversity in the Nordic Model of Education. Springer. ISSN 9783030616472. p. 197–224. doi: 10.1007/978-3-030-61648-9_8. Full text in Research Archive Show summary

  In both educational and psychological research, the relation between socioeconomic status (SES) and academic achievement is the most widely examined contextual effect. While several research syntheses have reported evidence of positive and significant SES–achievement relations (i.e., higher SES is associated with better academic achievement in several domains), they also reported substantial variation across educational contexts, such as classrooms, schools, and educational systems, and proposed mechanisms underlying these relations. This chapter addressed this variation and tested three hypotheses on the interplay between socioeconomic status, the disciplinary climate in science lessons, and science achievement—the compensation hypothesis, the mediation hypothesis, and the moderation hypothesis. Utilizing the Programme for International Student Assessment (PISA) 2015 data from the Nordic countries (Denmark, Finland, Iceland, Norway, and Sweden), multilevel structural equation modeling provided evidence to test the contextual, indirect, and cross-level interaction effects. While evidence for the compensation hypothesis existed in most Nordic countries, evidence supporting the mediating and moderating roles of the disciplinary climate for the SES–achievement relation was sparse.

• Nilsen, Trude; Scherer, Ronny; Gustafsson, Jan-Eric; Teig, Nani & Kaarstein, Hege (2020). Teachers’ role in enhancing equity: A multilevel structural equation modeling with mediated moderation. In Frønes, Tove Stjern; Pettersen, Andreas; Radišić, Jelena & Buchholtz, Nils (Ed.), Equity, Equality and Diversity in the Nordic Model of Education. Springer. ISSN 9783030616472. p. 173–196. doi: 10.1007/978-3-030-61648-9_7. Full text in Research Archive Show summary

  Even though equity is an important aim for the Nordic countries, for many of these countries, the effect of a student’s home background on their achievement seems to increase over time. If the aim is to reduce the effect of SES (socioeconomic status) on student outcomes, there is a need to identify the factors that moderate this relation. One such factor could be teachers and their instruction because they have been found to be key to student outcomes. However, few have linked teachers and their instruction to equity, and fewer still have made this link in Nordic countries. The aim of the present study is to identify the aspects of teacher quality and their instruction that may reduce the relationship between SES and student achievement in the Nordic countries. Eighth-grade students from the only two Nordic countries participating in TIMSS 2015 (Norway and Sweden) were selected. Multigroup, multilevel (students and classes) structural equation models with random slopes were employed to investigate which aspects of teacher quality moderate the relation between SES and student science achievement via instructional quality. The findings show that teacher professional development and specialisation reduce the relation between SES and science achievement via instructional quality in Sweden, while there were no significant findings for Norway. This study contributes to the fields of equity and teacher effectiveness, demonstrating that teachers may make a difference in reducing inequity through their competence and instruction.

• Huang, Fang; Teo, Timothy & Scherer, Ronny (2020). Investigating the antecedents of university students’ perceived ease of using the Internet for learning. Interactive Learning Environments. ISSN 1049-4820. doi: 10.1080/10494820.2019.1710540. Full text in Research Archive
• Teig, Nani; Scherer, Ronny & Kjærnsli, Marit (2020). Identifying patterns of students' performance on simulated inquiry tasks using PISA 2015 log‐file data. Journal of Research in Science Teaching. ISSN 0022-4308. 57(9), p. 1400–1429. doi: 10.1002/tea.21657. Full text in Research Archive Show summary

  Previous research has demonstrated the potential of examining log‐file data from computer‐based assessments to understand student interactions with complex inquiry tasks. Rather than solely providing information about what has been achieved or the accuracy of student responses (product data), students' log files offer additional insights into how the responses were produced (process data). In this study, we examined students' log files to detect patterns of students' interactions with computer‐based assessment and to determine whether unique characteristics of these interactions emerge as distinct profiles of inquiry performance. Knowledge about the characteristics of these profiles can shed light on why some students are more successful at solving simulated inquiry tasks than others and how to support student understanding of scientific inquiry through computer‐based environments. We analyzed the Norwegian PISA 2015 log‐file data, science performance as well as background questionnaire (N = 1,222 students) by focusing on two inquiry tasks, which required scientific reasoning skills: coordinating the effects of multiple variables and coordinating theory and evidence. Using a mixture modeling approach, we identified three distinct profiles of students' inquiry performance: strategic, emergent, and disengaged. These profiles revealed different characteristics of students' exploration behavior, inquiry strategy, time‐on‐task, and item accuracy. Further analyses showed that students' assignment to these profiles varied according to their demographic characteristics (gender, socio‐economic status, and language at home), attitudes (enjoyment in science, self‐efficacy, and test anxiety), and science achievement. Although students' profiles on the two inquiry tasks were significantly related, we also found some variations in the proportion of students' transitions between profiles. Our study contributes to understanding how students interact with complex simulated inquiry tasks and showcases how log‐file data from PISA 2015 can aid this understanding.

• Scherer, Ronny; Siddiq, Fazilat & Viveros, Bárbara Sánchez (2020). A meta-analysis of teaching and learning computer programming: Effective instructional approaches and conditions. Computers in Human Behavior. ISSN 0747-5632. 109, p. 1–18. doi: 10.1016/j.chb.2020.106349. Full text in Research Archive Show summary

  This meta-analysis maps the evidence on the effectiveness of instructional approaches and conditions for learning computer programming under three study conditions: (a) Studies focusing on the effectiveness of programming interventions per se, (b) studies focusing on the effectiveness of visualization and physicality, and (c) studies focusing on the effectiveness of dominant instructional approaches. Utilizing the data from 139 interventions and 375 effect sizes, we found (a) a strong effect of learning computer programming per se (Hedges’g  = 0.81, 95% CI [0.42, 1.21]), (b) moderate to large effect sizes of visualization (g = 0.44, 95% CI [0.29, 0.58]) and physicality interventions (g = 0.72, 95% CI [0.23, 1.21]), and (c) moderate to large effect sizes for studies focusing on dominant instructional approaches (s = 0.49–1.02). Moderator analyses indicated that the effect sizes differed only marginally between the instructional approaches and conditions—however, collaboration in metacognition instruction, problem solving instruction outside of regular lessons, short-term interventions focusing on physicality, and interventions focusing on visualization through Scratch were especially effective. Our meta-analysis synthesizes the existing research evidence on the effectiveness of computer programming instruction and, ultimately, provides references with which the effects of future studies could be compared.

• Scherer, Ronny; Howard, Sarah K.; Tondeur, Jo & Siddiq, Fazilat (2020). Profiling Teachers' Readiness for Online Teaching and Learning in Higher Education: Who's Ready? Computers in Human Behavior. ISSN 0747-5632. 118. doi: 10.1016/j.chb.2020.106675. Full text in Research Archive Show summary

  The COVID-19 pandemic has forced a shift to online teaching and learning (OTL) in colleges and universities across the globe, requiring teachers to adapt their teaching in a very short time—independent of whether they were prepared. Drawing from an international sample of N = 739 higher education teachers in 58 countries, the present study sheds light on teachers' readiness for OTL at the time of the pandemic by (a) identifying teacher profiles based on a set of key dimensions of readiness; (b) explaining profile membership by individual teacher characteristics, contextual aspects of the shift to OTL, and country-level indicators representing educational innovation and cultural orientation. We conducted latent profile analysis and identified three teacher profiles with consistently high or low readiness or an inconsistent readiness profile—hence, teachers in higher education are not a homogeneous group. Importantly, key individual and contextual variables, such as teachers’ gender and prior OTL experience, the context of the OTL shift, the innovation potential in education, and cultural orientation, explained profile membership. We discuss these findings with respect to the nature of the profiles, how they can be understood with respect to key determinants, and their implications for OTL in higher education.

• Scherer, Ronny (2020). Analysing PIAAC Data with Structural Equation Modelling in Mplus. In Maehler, Débora B. & Rammstedt, Beatrice (Ed.), Large-Scale Cognitive Assessment: Analyzing PIAAC Data. Springer Nature. ISSN 9783030475154. p. 165–208. doi: 10.1007/978-3-030-47515-4_8. Full text in Research Archive Show summary

  Structural equation modelling (SEM) has become one of the most prominent approaches to testing substantive theories about the relations among observed and/or unobserved variables. Applying this multivariate procedure, researchers are faced with several methodological decisions, including the treatment of indicator variables (e.g. categorical vs. continuous treatment), the handling of missing data, and the selection of an appropriate level of analysis. The PIAAC data pose additional issues, such as the clustering of individual-level data, the large number of participating countries, the representation of performance scores by a set of plausible values, and the differences in the selection probabilities. Therefore, a flexible software package is required to handle them. This chapter introduces readers to analysing PIAAC data with SEM in the software Mplus by (a) presenting the key concepts behind SEM, (b) discussing the complexities of the PIAAC data and their possible handling, (c) illustrating the specification and evaluation of measurement and structural models, and (d) pointing to current developments in the areas of measurement invariance testing and multilevel SEM. Sample input and output files are provided.

• Hjetland, Hanne Næss; Brinchmann, Ellen Iren; Scherer, Ronny; Hulme, Charles & Melby-Lervåg, Monica (2020). Preschool pathways to reading comprehension: A systematic meta-analytic review. Educational Research Review. ISSN 1747-938X. 30:100323, p. 1–23. doi: 10.1016/j.edurev.2020.100323. Full text in Research Archive Show summary

  The ability to construct meaning from texts is the core of reading. We report a meta-analysis and a systematic review of 64 longitudinal studies tracing the development of reading comprehension from preschool. Previous research showed that linguistic comprehension and code-related abilities in preschool correlate moderately with reading comprehension, but the results across studies are inconsistent. Meta-analytic structural equation modelling showed two distinct but related pathways from preschool linguistic comprehension abilities to reading comprehension in school. One pathway consists of code-related skills (letter knowledge and phonological awareness), and it affects reading comprehension through word recognition. A second pathway consists of linguistic comprehension skills (vocabulary and grammar), and it has a direct influence on reading comprehension. Early interventions to facilitate language development appear to provide a promising approach to facilitate the later development of reading comprehension skills.

• Rohatgi, Anubha & Scherer, Ronny (2020). Identifying profiles of students' school climate perceptions using PISA 2015 data. Large-scale Assessments in Education. 8(1). doi: 10.1186/s40536-020-00083-0. Full text in Research Archive Show summary

  The concept of school climate has received much attention as a predictor of educational outcomes, including students’ well-being, academic achievement, and motivation. To measure this concept, international large-scale assessments often rely on students’ perceptions of its different dimensions, such as their sense of belonging, teacher support, and disciplinary climate. However, students may perceive these dimensions differently and, ultimately, create inter-individual variation—a variation that has been explained only to a limited degree in the current body of literature. The present study explores this variation for the Norwegian PISA 2015 data (N = 5313). Using the person-centered approach of latent profile analysis, we found evidence for the existence of three student profiles: (1) students with consistently positive perceptions, (2) students with moderately negative perceptions, and (3) students with extremely negative perceptions, especially concerning teachers’ fairness and bullying. These results support the hypothesis of individual differences in school climate perceptions.

• Howard, Sarah K.; Tondeur, Jo; Siddiq, Fazilat & Scherer, Ronny (2020). Ready, set, go! Profiling teachers' readiness for online teaching in secondary education. Technology, Pedagogy and Education. ISSN 1475-939X. doi: 10.1080/1475939X.2020.1839543. Full text in Research Archive Show summary

  In a very short time, secondary school education across the globe transitioned to online learning and teaching, in response to the Covid-19 pandemic. This study aims at identifying teacher profiles in secondary education to better understand perceptions of both individual and institutional readiness to transition to online teaching. To do this, the current study grouped teachers on the basis of their TPACK self-efficacy beliefs, online presence and perceived institutional support for online teaching. To date, data have been collected from teachers (N = 222) from 20 countries. The data were submitted to latent profile analysis to identify readiness profiles. The added value of the current study lies in the combined view of individual and institutional readiness and the uniqueness of the dataset. It provides a large-scale international perspective and a wide range of possible experiences. Findings inform how education institutions can personalise and support transitions to online teaching.

• Scherer, Ronny; Siddiq, Fazilat & Tondeur, Jo (2020). All the same or different? Revisiting measures of teachers' technology acceptance. Computers & Education. ISSN 0360-1315. 143. doi: 10.1016/j.compedu.2019.103656. Full text in Research Archive Show summary

  Technology acceptance has been widely discussed and examined in educational contexts. Despite the variety of models and measures describing teachers' technology acceptance, two key assumptions persist in the existing body of literature: First, the technology acceptance construct can be represented by a set of diverse, yet correlated attitudes and beliefs. Second, the effects of technology acceptance on the intentions to use technology and technology use—two commonly studied outcome variables—follow a cascade. The existing evidence backing these assumptions is, however, diverse, as the considerable between-study variation in the relations between the technology acceptance and outcome variables shows. This variation remained largely unexplained, and the present study offers an explanation from the perspective of construct validity. Analyzing a large meta-analytic sample (N = 37211 teachers) obtained from a previous meta-analysis, we synthesized the correlations among measures of teachers' technology acceptance and found support for the existence of a common trait that underlies all technology acceptance variables. This finding remained even after distinguishing between different teacher samples (i.e., pre-vs. in-service teachers) and types of technology (i.e., technology in general vs. specific technologies). There was no convincing evidence for the hypothesized cascade of effects, due to a weak and insignificant link between usage intentions and technology use. Our findings provide evidence for the representation of teachers' technology acceptance as a single latent variable and consequently offer a way to describe its relations to usage intentions and technology use without compromising the validity of the inferences drawn from them.

• Scherer, Ronny & Teo, Timothy (2020). A tutorial on the meta-analytic structural equation modeling of reliability coefficients. Psychological methods. ISSN 1082-989X. 25(6), p. 747–775. doi: 10.1037/met0000261. Full text in Research Archive Show summary

  Reporting the reliability of the scores obtained from a scale or test is part of the standard repertoire of empirical studies in psychology. With reliability being a key concept in psychometrics, researchers have become more and more interested in evaluating reliability coefficients across studies and, ultimately, quantify and explain possible between-study variation. This approach—commonly known as “reliability generalization”—can be specified within the framework of meta-analysis. The existing procedures of reliability generalization, however, have several methodological issues: (a) unrealistic and often untested assumptions on the measurement model underlying the reliability coefficients (e.g., essential τ-equivalence for Cronbach’s α); (b) the use of univariate approaches to synthesizing reliabilities of total and subscale scores; (c) the lack of comparability across different types of reliability coefficients. However, these issues can be addressed directly through meta-analytic structural equation modeling (MASEM)—a method that combines meta-analysis with structural equation modeling through synthesizing either correlation matrices or model parameters across studies. The primary objective of this article is to present the potential MASEM has for the meta-analysis of reliability coefficients. We review the extant body of literature on the use of reliability generalization, discuss and illustrate two MASEM approaches (i.e., correlation-based and parameter-based MASEM), and propose some practical guidelines. Future directions for utilizing MASEM for reliability generalization are discussed.

• Goldhammer, Frank; Scherer, Ronny & Greiff, Samuel (2019). Advancements in Technology-Based Assessment: Emerging Item Formats, Test Designs, and Data Sources. Frontiers in Psychology. doi: 10.3389/fpsyg.2019.03047. Full text in Research Archive Show summary

  Technology has become an indispensable tool for educational and psychological assessment in today’s world. Researchers and large-scale assessment programs alike are increasingly using digital technology (e.g., laptops, tablets, and smartphones) to collect behavioral data beyond the mere idea of responses as correct. Along these lines, technology innovates and enhances assessments in terms of item and test design, methods of test delivery, data collection and analysis, as well as the reporting of test results. The aim of this Research Topic is to present recent advancements in technology-based assessment. Our focus is on cognitive assessments, including the measurement of abilities, competencies, knowledge, and skills but may also include non-cognitive aspects of the assessment. In the area of (cognitive) assessments the innovations driven by technology are manifold: Digital assessments facilitate the creation of new types of stimuli and response formats that were out of reach for assessments using paper; for instance, interactive simulations including multimedia elements, as well as virtual or augmented realities which serve as the task environment. Moreover, technology allows the automated generation of items based on specific item models. Such items can be assembled into tests in a more flexible way than that offered by paper-and-pencil tests and could even be created on the fly; for instance, tailoring item difficulty to individual ability (adaptive testing), while assuring that multiple content constraints are met. As a requirement for adaptive testing or to lower the burden of raters coding item responses manually, computers enable the automatic scoring of constructed responses; for instance, text responses can be scored automatically by using natural language processing and text mining. Technology-based assessments provide not only response data (e.g., correct vs. incorrect responses) but also process data (e.g., frequencies and sequences of test-taking strategies, including navigation behavior) which reflects the course of solving a test item. Process data has been used successfully, among others, to evaluate the data quality, to define process-oriented constructs, to improve measurement precision, and to address substantial research questions. We expect the contributions of this Research Topic to build on this research by considering how technology can further improve, and enhance, educational and psychological assessment. Regarding educational testing, both research papers on the assessment of learning (e.g., summative assessment of learning outcomes) and on the assessment for learning (e.g., formative assessment to support the learning process) are welcome. We expect submissions of empirical papers that present and evaluate innovative technology-based assessment approaches, as well as new applications or illustrations of already existing approaches. We are also interested in papers addressing the validity of test scores and other indicators obtained from innovative assessment procedures. Please note that this description serves as a starting point to familiarize potential contributors with the scope of this Research Topic. Thus, there may also be contributions that fit the Research Topic that are not mentioned explicitly here. If you have any questions about the scope of our Research Topic, feel free to contact the editors of this Research Topic. Keywords: technology-based assessment, item design, test design, automatic scoring, process data

• Siddiq, Fazilat & Scherer, Ronny (2019). Is there a gender gap? A meta-analysis of the gender differences in students' ICT literacy. Educational Research Review. ISSN 1747-938X. 27, p. 205–217. doi: 10.1016/j.edurev.2019.03.007. Full text in Research Archive Show summary

  The study of gender differences in academic achievement has been one of the core topics in education, especially because it may uncover possible gaps and inequalities in certain domains. Whereas these differences have largely been examined in traditional domains, such as mathematics, reading, and science, the existing body of empirical studies in the domain of ICT literacy is considerably smaller, yet abounds in diverse findings. One of the persistent findings however is that boys consider their ICT literacy to be higher than that of girls. This meta-analysis tests whether the same pattern holds for students’ actual performance on ICT literacy tasks, as measured by performance-based assessments. In total, 46 effect sizes were extracted from 23 empirical studies using a random-effects model. Overall, the gender differences in ICT literacy were significant, positive, and favored girls (g = + 0.12, 95 % CI = [0.08, 0.16]). This effect varied between studies, and moderation analyses indicated that the grade level students were taught at moderated its magnitude—effect sizes were larger in primary school as compared to secondary school. In conclusion, our findings contrast those obtained from previous meta-analyses that were based on self-reported ICT literacy and suggest that the ICT gender gap may not be as severe as it had been claimed to be.

• Tondeur, Jo; Scherer, Ronny; Siddiq, Fazilat & Baran, Evrim (2019). Enhancing pre-service teachers’ technological pedagogical content knowledge (TPACK): a mixed-method study. Educational Technology Research and Development. ISSN 1042-1629. doi: 10.1007/s11423-019-09692-1. Full text in Research Archive Show summary

  The main aim of this two-step mixed-method study was to explore the effectiveness of the strategies used to prepare pre-service teachers for technological pedagogical content knowledge (TPACK). Specifically, we focused on the strategies included in the synthesis of qualitative evidence (SQD) model: (1) using teacher educators as role models, (2) reflecting on the role of technology in education, (3) learning how to use technology by design, (4) collaboration with peers, (5) scaffolding authentic technology experiences, and (6) providing continuous feedback. To explore the relation between the perceived occurrences of the SQD-strategies and TPACK (controlled for pre-service teachers’ general attitudes towards technology), survey data were collected from a sample of 688 final-year pre-service teachers in Belgium. In a next step, 16 telephone interviews and 6 in-depth interviews were conducted to gain a more in-depth insight into the nature of the 6 strategies and their influences on TPACK. The quantitative analyses indicated positive correlations between the SQD-strategies and TPACK, controlled for general attitudes towards technology. The findings from the qualitative analyses showed that teachers acknowledged the importance of the six strategies. However, the respondents emphasized that some of the six strategies are often underutilized. Based on the quantitative and qualitative results, the discussion provides recommendations to improve the potential of pre-service training to enhance future teachers’ TPACK.

• Tondeur, Jo; Scherer, Ronny; Baran, Evrim; Siddiq, Fazilat; Valtonen, Teemu & Sointu, Erkko (2019). Teacher educators as gatekeepers: Preparing the next generation of teachers for technology integration in education. British Journal of Educational Technology (BJET). ISSN 0007-1013. 50(3), p. 1189–1209. doi: 10.1111/bjet.12748. Full text in Research Archive
• Scherer, Ronny & Teo, Timothy (2019). Editorial to the special section—Technology acceptance models: What we know and what we (still) do not know. British Journal of Educational Technology (BJET). ISSN 0007-1013. 50(5). doi: 10.1111/bjet.12866. Full text in Research Archive Show summary

  The rapid technological advancements and the digitalization in almost all areas of our lives, including education, have turned the attention of researchers to the factors that explain a person's technology acceptance. This attention resulted in several theoretical models that describe both the behavioral intention and the use of technologies, such as the technology acceptance model (TAM) and the unified theory of acceptance and use of technology (UTAUT). Over the last three decades, the body of empirical research on these models has increased, yet abounded in contradictory findings, in particular on the generalizability and comparability of these models (Nistor, 2014; Scherer, Siddiq, & Tondeur, 2019). Some reasons for the divergent findings may lie in the cultural specificity of the factors hypothesized to explain technology acceptance and adoption, the validity of measures used to represent them and the specificity to certain technologies (Marangunić & Granić, 2015; Scherer & Teo, 2019). Given the enormous influence TAMs have and will have on the design and distribution of almost any technology in education, including learning analytics tools and collaborative environments, it is critical to bring to attention the current issues and challenges surrounding them in order to identify future needs and research directions. These goals lie in the heart of this special section—it highlights persistent findings on technology acceptance across samples, domains, technologies, countries and other contexts, identifies commonalities in and differences between TAMs and reviews the contributions of these models to teaching and learning. Specifically, the authors present empirical studies and theoretical reviews in order to (a) classify and extend the set of constructs and models of technology acceptance, (b) classify and extend samples of students and teachers, (c) explain contrary findings and (d) review overarching issues in TAMs and research (see Table 1).

• Scherer, Ronny; Siddiq, Fazilat & Viveros, Bárbara Sánchez (2019). The Cognitive Benefits of Learning Computer Programming: A Meta-Analysis of Transfer Effects. Journal of Educational Psychology. ISSN 0022-0663. 111(5), p. 764–792. doi: 10.1037/edu0000314. Full text in Research Archive Show summary

  Does computer programming teach students how to think? Learning to program computers has gained considerable popularity, and educational systems around the world are encouraging students in schools and even children in kindergartens to engage in programming activities. This popularity is based on the claim that learning computer programming improves cognitive skills, including creativity, reasoning, and mathematical skills. In this meta-analysis, we tested this claim performing a 3-level, random-effects meta-analysis on a sample of 105 studies and 539 effect sizes. We found evidence for a moderate, overall transfer effect (g = 0.49, 95% CI [0.37, 0.61]) and identified a strong effect for near transfer (g = 0.75, 95% CI [0.39, 1.11]) and a moderate effect for far transfer (g = 0.47, 95% CI [0.35, 0.59]). Positive transfer to situations that required creative thinking, mathematical skills, and metacognition, followed by spatial skills and reasoning existed. School achievement and literacy, however, benefited the least from learning to program. Moderator analyses revealed significantly larger transfer effects for studies with untreated control groups than those with treated (active) control groups. Moreover, published studies exhibited larger effects than gray literature. These findings shed light on the cognitive benefits associated with learning computer programming and contribute to the current debate surrounding the conceptualization of computer programming as a form of problem solving. (PsycINFO Database Record (c) 2018 APA, all rights reserved)

• Scherer, Ronny & Nilsen, Trude (2019). Closing the gaps? Differential effectiveness and accountability as a road to school improvement. School Effectiveness and School Improvement. ISSN 0924-3453. 30(3), p. 255–260. doi: 10.1080/09243453.2019.1623450. Full text in Research Archive
• Teig, Nani; Scherer, Ronny & Nilsen, Trude (2019). I Know I Can, but Do I Have the Time? The Role of Teachers’ Self-Efficacy and Perceived Time Constraints in Implementing Cognitive-Activation Strategies in Science. Frontiers in Psychology. 10. doi: 10.3389/fpsyg.2019.01697. Full text in Research Archive Show summary

  Considerable research has demonstrated that teachers’ self-efficacy plays a major role in implementing instructional practices. Only few studies, however, have examined the interplay between how teachers’ self-efficacy and the challenges that lie outside their influence are related to their implementation of cognitive-activation strategies (CASs), especially in science classrooms. Using the Trends in Mathematics and Science Study 2015 data from science teachers in Grades 4, 5, 8, and 9, we explored the extent to which teachers’ self-efficacy in science teaching and the perceived time constraints explained variations in the enactment of general and inquiry-based CAS. Findings from the overall sample showed that highly self-efficacious teachers reported more frequent implementation of both general and inquiry-based CAS, whereas those who perceived strong time constraints reported a less frequent use of inquiry-based CAS. These relationships also existed across grade levels, except on the relations between perceived time constraint and inquiry-based CAS, which was only significant for the science teachers in Grade 9. We discuss these findings in light of variations in the core competencies of science curriculum, teachers’ competences, and the resources for science activities between primary and secondary education. We also point to the theoretical implications of this study for enhancing the conceptual understanding of generic and specific aspects of CAS and the practical implications for teacher education, professional development, and educational policy.

• Scherer, Ronny; Siddiq, Fazilat & Tondeur, Jo (2019). The technology acceptance model (TAM): A meta-analytic structural equation modeling approach to explaining teachers’ adoption of digital technology in education. Computers & Education. ISSN 0360-1315. 128, p. 13–35. doi: 10.1016/j.compedu.2018.09.009. Full text in Research Archive Show summary

  The extent to which teachers adopt technology in their teaching practice has long been in the focus of research. Indeed, a plethora of models exist explaining influential factors and mechanisms of technology use in classrooms, one of which—the Technology Acceptance Model (TAM) and versions thereof—has dominated the field. Although consensus exists about which factors in the TAM might predict teachers’ technology adoption, the current field abounds in some controversies and inconsistent findings. This meta-analysis seeks to clarify some of these issues by combining meta-analysis with structural equation modeling approaches. Specifically, we synthesized 124 correlation matrices from 114 empirical TAM studies (N = 34,357 teachers) and tested the fit of the TAM and its versions. Overall, the TAM explains technology acceptance well; yet, the role of certain key constructs and the importance of external variables contrast some existing beliefs about the TAM. Implications for research and practice are discussed.

• Vanslambrouck, Silke; Zhu, Chang; Pynoo, Bram; Lombaerts, Koen; Tondeur, Jo & Scherer, Ronny (2019). A latent profile analysis of adult students' online self-regulation in blended learning environments. Computers in Human Behavior. ISSN 0747-5632. doi: 10.1016/j.chb.2019.05.021. Full text in Research Archive Show summary

  Self-regulated learning (SRL) is crucial for academic success; therefore support, to enhance and maintain SRL skills is important. In blended adult education, the heterogeneity of adults creates diversity in SRL abilities, which makes it necessary to provide tailored support. Conducting latent profile analyses for a sample of 213 blended adult students, we identified three profiles, namely high, low, and moderate SRL profiles which prove differences in SRL strategy use and imply tailored SRL support. Through multivariate analysis of variance (MANOVA) and multinomial logistic regression, we further explore the differences in SRL between the profiles and the extent to which the students’ personal background characteristics and achievement motivations predict their profile membership. The three profiles differ significantly in terms of the scores of all SRL subscales. Furthermore, only achievement motivation – more specifically, attainment and utility value – predicts profile membership. These results inform educational practice about opportunities for supporting and enhancing SRL skills. Anticipating attainment and utility value, time management, and collaboration with peers are all recommended. More specifically, teachers can, for example, use authentic tasks and examples during the learning process or be a role model regarding online interaction and information sharing.

• Scherer, Ronny & Teo, Timothy (2019). Unpacking teachers' intentions to integrate technology: A meta-analysis. Educational Research Review. ISSN 1747-938X. 27, p. 90–109. doi: 10.1016/j.edurev.2019.03.001. Full text in Research Archive Show summary

  The Technology Acceptance Model (TAM) is a key model describing teachers' intentions to use technology. This meta-analysis clarifies some of the contradictory findings surrounding the relations within the TAM for a sample of 45 studies comprising 300 correlations. We evaluate the overall fit of the TAM and its structural parameters, and quantify the between-sample variation through meta-analytic structural equation modeling. The TAM fitted the data well, and all structural parameters were statistically significant. On average, the TAM variables explained 39.2% of the variance in teachers' intentions to use technology. Several sample, measurement, and publication characteristics, including teachers’ experience and the representation of the TAM variables, moderated the relations within the TAM. Overall, the TAM represents a valid model explaining technology acceptance—however, the degree of explanation and the relative importance of predictors vary across samples. Implications for further research, in particular the generalizability of the TAM, are discussed.

• Scherer, Ronny & Siddiq, Fazilat (2019). The relation between students' socioeconomic status and ICT literacy: Findings from a meta-analysis. Computers & Education. ISSN 0360-1315. 138, p. 13–32. doi: 10.1016/j.compedu.2019.04.011. Full text in Research Archive Show summary

  This meta-analysis synthesized the relation between measures of socioeconomic status (SES) and students' information and communication technology (ICT) literacy—a skillset that has found its way in educational curricula. Using three-level random-effects modeling across 32 independent K-12 student samples that provided 75 correlation coefficients, we identified a positive, significant, and small correlation, = 0.214, 95% CI [0.184, 0.244]. This correlation varied between studies and was moderated by the type of SES measure, the type of ICT literacy assessment, the broad categories of ICT skills assessed, the assessment of test fairness, and the sampling procedure employed. The findings of this meta-analysis suggest that students’ ICT literacy differs between socioeconomic status groups, thus pointing to a gap in the domain of ICT. However, the relation between SES and ICT literacy was weaker than those reported in other educational domains, such as mathematics and reading. Carefully designed studies and measures for which a validity argument has been crafted are needed when studying achievement gaps in the domain of ICT in future studies.

• Scherer, Ronny; Tondeur, Jo; Siddiq, Fazilat & Baran, Evrim (2018). The importance of attitudes toward technology for pre-service teachers' technological, pedagogical, and content knowledge: Comparing structural equation modeling approaches. Computers in Human Behavior. ISSN 0747-5632. 80, p. 67–80. doi: 10.1016/j.chb.2017.11.003. Full text in Research Archive Show summary

  A large body of literature suggests that attitudes toward technology and its educational use are important determinants of technology acceptance and integration in classrooms. At the same time, teachers' Technological, Pedagogical, and Content Knowledge (TPACK) facilitates the meaningful use of technology for educational purposes. Overall, attitudes toward technology and TPACK play a critical role for technology integration and have been in the focus of many empirical studies. Albeit the attention that has been paid to these two concepts, their relation has not been fully understood. The present study contributes to the advancement of this understanding by examining the relations between three core technology attitudes (i.e., general attitudes towards ICT, attitudes towards ICT in education, and ease of use) and TPACK self-efficacy beliefs, based on a sample of N = 688 Flemish pre-service teachers in 18 teacher-training institutions. Using a variety of structural equation modeling approaches, we describe the TPACK-attitudes relations from multiple perspectives and present a substantive-methodological synergism. The analyses revealed that the attitudes toward technology and TPACK self-beliefs were positively related; yet, differences across the attitudes and TPACK dimensions existed, pointing to the delineation of general and educational perspectives on the use of ICT.

View all works in NVA

• Wang, Lei; Yang, Junyi; Scherer, Ronny; Melby-Lervåg, Monica & Anmarkrud, Øistein (2026). The Relationship Between Screen Use and Language and Literacy Skills in Childhood: The Roles of Family SES and Children's Prior Ability. doi: https:/www.triplesr.org/sites/default/files/2026-07/detailed_programme%20July%2015%202026.pdf. Full text in Research Archive
• Siddiq, Fazilat & Scherer, Ronny (2025). Learner's Screen Use: Time to Reload the Debate. Nordic Journal of Digital Literacy. p. 35–39. doi: 10.18261/njdl.20.1.6. Full text in Research Archive
• Siddiq, Fazilat; Berg, Tonje Karoliussen; Haraldsrud, Andreas; Scherer, Ronny; åsgård, ellen & Erstad, Ola [Show all 7 contributors for this article] (2024). Algoritmisk tenkning og samarbeidslæring - hvordan jobbe med programmering i skolen. In Berg, Tonje Karoliussen & Siddiq, Fazilat (Ed.), Algoritmisk tenkning og samarbeidslæring. Hvordan jobbe med programmering i skolen. Gyldendal Akademisk. ISSN 9788205598928. Full text in Research Archive
• Siddiq, Fazilat; Tondeur, Jo; Howard, Sarah K. & Scherer, Ronny (2023). New methods, new insights: Making sense of teachers' experiences with online education. Full text in Research Archive Show summary

  The Covid-19 pandemic provided a unique opportunity to understand school teachers' perceptions of and experiences with online practices on a large scale. However, there is a need for a more detailed understanding of the multiple associations among teachers’ perceptions and experiences in change processes. Therefore, the main goal of the current study is to untangle the complexity of teachers' experiences and perceptions during this great online transition. Based on the data of 222 teachers' responses to open-ended questions, we used a combined method of thematic and network analysis. To illustrate this novel method, we will focus on the strongest associations between the categories “learning design for online teaching” and “Student learning”. The results indicated for instance a strong association between ‘Self-paced’, which was a dimension of ‘learning design’, and ‘Flexibility’, which was discussed in relation to ‘student experience’. During the presentation the authors will provide more examples of the associations between teachers’ online experiences and perceptions. They will also demonstrate how the network model together with the thematic analysis can be used as a 'roadmap' towards a more integrated approach of online and blended learning in schools and to guide future professional development efforts and school policies.

• Melby-Lervåg, Monica; Amland, Tonje; Grande, German; Scherer, Ronny & Lervåg, Arne (2023). Does language underlie mathematical skill development? A systematic review and meta-analysis of concurrent and longitudinal predictors. Full text in Research Archive
• Gochyeev, Perman; Siddiq, Fazilat; Scherer, Ronny & WIlson, Mark (2022). Exploring collaborative problem-solving tasks for individual-level and team-level inferences using multilevel modeling. Full text in Research Archive Show summary

  Collaborative problem solving (ColPS) is necessary to succeed in the 21st century and has therefore been introduced into compulsory K-12 education in many Western countries. ColPS is the “capacity of an individual to: recognize the perspective of other persons in a group; participate as a member of the group by contributing their knowledge, experience, and expertise constructively; recognize the need for contributions and how to manage them; identify structure and procedure involved in resolving a problem; and, as a member of the collaborative group, build and develop knowledge and understanding” (Griffin et al., 2012, p.7). Several aspects need consideration when designing ColPS tasks, including group dependence, team size, and activities to promote collaboration and problem solving (Siddiq & Scherer, 2017). Since students are often randomly assigned to teams, observations within groups are not independent. Hence, psychometric models need to account for the hierarchical structure of the data. Wilson et al. (2017) presented an approach for analyzing ColPS tasks using multilevel models, which are widely used in educational research. The authors focused on modeling the group effects on the overall test performances. We propose focusing on group effects on individual task performance instead. We aim to classify ColPS tasks into ones with and without ignorable group effects, implying lower and higher collaboration. We also propose calibrating these tasks using a two-dimensional Rasch model to further understand the two constructs these items represent. We analyse data collected from the Norwegian Digital literacy (LDN-ICT) instrument (Siddiq et al., 2017). In total, 39 tasks were administered to 144 ninth-grade students who worked in groups of three or four. Prior to modelling, it will be helpful to use descriptive statistics in an exploratory way to establish a framing context for modelling. Preliminary findings suggest that tasks showing significant group effects can be identified based on the estimated intraclass correlation (ICC)—a commonly used measure of group dependence. ICCs closer to zero indicate ignorable group effects, while higher ICCs imply substantial group effects. The ICCs ranged between 0.00 and 0.88, with an average ICC of 0.31. We also found a significant agreement between estimated group dependence and intended group dependence. Moreover, tasks requiring reflection, evaluation, and accessing information exhibited higher ICCs than others. This study contributes to the design and analysis of ColPS assessments and illustrates tools to assess if tasks function as they were intended.

• Mittal, Oleksandra; Scherer, Ronny & Nilsen, Trude (2022). Assessing the Evidence for Comparability of Socioeconomic Status across Students with and without Immigrant Background in Norway and Sweden [Paper presentation]. Nordic Educational Research Association (NERA) Conference, Reykjavik, Iceland. Full text in Research Archive
• Blömeke, Sigrid; Howie, Sarah J.; Kaplan, David & Scherer, Ronny (2022). Introduction to the Comparative Large-Scale Studies in Education: Structure and Overview of the Handbook. In Nilsen, Trude; Stancel-Piątak, Agnes & Gustafsson, Jan-Eric (Ed.), International Handbook of Comparative Large-Scale Studies in Education. Springer. ISSN 9783030382988. doi: 10.1007/978-3-030-38298-8_1-1. Full text in Research Archive
• Iliescu, Dragos; Rusu, Andrei; Greiff, Samuel; Fokkema, Marjolein & Scherer, Ronny (2022). Why We Need Systematic Reviews and Meta-Analyses in the Testing and Assessment Literature. European Journal of Psychological Assessment. ISSN 1015-5759. 38(2). doi: 10.1027/1015-5759/a000705. Full text in Research Archive
• Mittal, Oleksandra; Scherer, Ronny & Nilsen, Trude (2021). Measuring Socioeconomic Status across Ethnic Majority and Minorities in PISA 2018: Testing Household Possessions Index for Differential Item Functioning [Paper presentation]. European Conference on Educational Research (ECER), Geneva (online). https://eera-ecer.de/ecer-programmes/conference/26/contribution/51551/. Full text in Research Archive
• Iliescu, Dragos; Greiff, Samuel; Proyer, René; Ziegler, Matthias; Allen, Mark & Claes, Laurence [Show all 15 contributors for this article] (2021). Supporting Academic Freedom and Living Societal Responsibility. European Journal of Psychological Assessment. ISSN 1015-5759. doi: 10.1027/1015-5759/a000652. Full text in Research Archive
• Siddiq, Fazilat; Scherer, Ronny & Tondeur, Jo (2020). Developing Students’ Collaborative Problem-Solving Competence (TEACH21st): Towards a linking science. Full text in Research Archive
• Hjetland, Hanne Næss; Brinchmann, Ellen Iren; Scherer, Ronny & Melby-Lervåg, Monica (2018). Preschool predictors of later reading comprehension ability: a Campbell systematic review. Full text in Research Archive
• Rozman, Mojca; Christiansen, Andrés; Strietholt, Rolf; Bundsgaard, Jeppe; Fraillon, Julian & Scherer, Ronny (2024). Improving Data in Electronic Surveys (IDES): Final Report. IEA-ETS Research Institute. Full text in Research Archive
• Fütterer, Tim; Lachner, Andreas; Scheiter, Katharina; Scherer, Ronny & Stürmer, Kathleen (2020). Will, Skill or Conscientiousness: What Predicts Teachers' ICT-Related Professional Development? (Study Protocol). Leibniz Psychology. Full text in Research Archive Show summary

  This study protocol outlines a study to explore the predictive power of cognitive, motivational and personality traits for teachers' tendency of participation in as well as teachers' willingness for ICT-related PD. For the study we will use data of teachers from the project tabletBW. TabletBW is a school trial for learning with tablets in classrooms which is accompanied by longitudinal scientific research from 2018 to 2021. To test our hypotheses, we will conduct stepwise multivariate multiple regression models (SEM) including teachers' utility value of educational technology, teachers' technological ability as well as teachers' conscientiousness as main predictors for teachers' tendency of participation in as well as teachers' willingness for ICT-related PD.

View all works in NVA