Our latest publications

• Gilbert, Joshua B.; Kim, Edward J.; Himmelsbach, Zachary; Ulitzsch, Esther & Zhang, Lijin (2026). Idiographic Item Response Theory: Modeling Person-Specific Differential Item Functioning in Intensive Longitudinal Data. Multivariate Behavioral Research. ISSN 0027-3171. doi: 10.1080/00273171.2026.2740068. Fulltekst i vitenarkiv
• Kastorff, Tamara; Scherer, Ronny; Müller, Maren; Booij, Goan; Kennedy, Alec & Strietholt, Rolf [Vis alle 7 forfattere av denne artikkelen] (2026). Do schools narrow the socioeconomic digital divide? Meta-analytic evidence from ICILS 2023. Large-scale Assessments in Education. 14(1). doi: 10.1186/s40536-026-00330-w. Fulltekst i vitenarkiv Vis sammendrag

  Abstract Second-level digital divides, often studied as the association between socioeconomic status (SES) and digital competence, remain a central concern in digital educational research. Whereas many countries have invested substantially in school digitalization, little is known about which school-level factors can mitigate SES-related disparities in digital competence. We addressed this gap using large-scale and representative data from the International Computer and Information Literacy Study (ICILS) 2023. Employing a two-step meta-analytic approach, country-specific multilevel models were first estimated and subsequently synthesized to quantify average effects and cross-national variation. Results indicated a positive, moderate association between SES and digital competence (β = 17.73, SE  = 0.93, 95% CI [15.84, 19.61]). Several digital classroom practices, including the use of general information and communication technology (ICT) applications, inquiry-based teaching, and teachers’ use of utility software were positively related to students’ digital competence. Crucially, the SES-digital competence association was not moderated by classroom practices or teacher- or school-level factors. Whereas digital media integration was linked to higher overall levels of digital competence, it did not systematically reduce socioeconomic disparities. These findings suggest that digital instructional practices function primarily as performance-enhancing rather than equity-enhancing mechanisms. For research in digital education, the results highlight the importance of distinguishing between average learning gains and distributional effects. If digital competence is to serve as a lever for educational equity, targeted and structural interventions may be required beyond general digitalization strategies.

• Vogelsmeier, Leonie V.D.E.; Siepe, Björn S.; Ulitzsch, Esther; Eisele, Gudrun; Geeraerts, Joran & Klocek, Adam [Vis alle 12 forfattere av denne artikkelen] (2026). Identifying careless responding in ecological momentary assessment: Inconsistent signals from different detection methods in the WARN-D Data. Psychological methods. ISSN 1082-989X. doi: 10.1037/met0000845. Fulltekst i vitenarkiv
• Eisele, Gudrun; Ulitzsch, Esther; Pihlajamäki, Milla; Myin-Germeys, Inez; Bakker, Jindra & Kirtley, Olivia J. [Vis alle 9 forfattere av denne artikkelen] (2026). Shortening Affect Scales for Ambulatory Assessment: A Comparison of Different Item Response Theory-Based Approaches. Assessment (ASM). ISSN 1073-1911. doi: 10.1177/10731911261484340. Fulltekst i vitenarkiv
• Tondeur, Jo; Howard, Sarah K.; Bucchiarone, Antonio; Scherer, Ronny & Braak, Johan van (2026). How we study what we study: A methodological review of 50 years of educational technology research. Computers & Education. ISSN 0360-1315. doi: 10.1016/j.compedu.2026.105760. Fulltekst i vitenarkiv Vis sammendrag

  Research in educational technology has been extensively mapped in terms of what has been studied, but far less attention has been paid to how it has been studied. This paper reports a methodological review of fifty years of research in Computers & Education, covering a corpus of 5,516 articles published between 1976 and 2026, with the principal findings based on the post-1996 full-text corpus. Using an AI-assisted analysis, applied to full texts and abstracts, we traced the evolution of research paradigms, approaches, and methods across the journal's full publication history. The principal findings are based on the post-1996 full-text corpus. Pre-1996, full-text machine-readable files were not available for analysis. Analysis of pre-1996 abstracts was possible and are included to provide context for trends observed post-1996. The findings show that quantitative approaches and questionnaire-based data collection have constituted the field's sustained methodological centre of gravity. Within this dominant pattern, two shifts stand out: 1) the rise of systematic reviews as a recognised approach, and 2) a computational turn. The range of methods used in research has grown steadily, suggesting a gradual broadening of the field's repertoire. These patterns are only visible across multiple decades. Findings reveal that quantitative dominance is not a recent development but a persistent structural feature, and that the computational surge is better understood as a second wave following a peak in the mid-2000s. These findings have implications for what kinds of knowledge educational technology research is equipped to produce. The study argues that understanding a field's methodological history is a prerequisite for extending it.

• Patsis, Panagiotis; Campos, Diego G. & Scherer, Ronny (2026). Educational inequality across countries: An integrative data analysis of TIMSS 2023 data. Studies in Educational Evaluation. ISSN 0191-491X. 91. doi: 10.1016/j.stueduc.2026.101683. Fulltekst i vitenarkiv Vis sammendrag

  Socioeconomic disparities in educational achievement remain a central challenge for education systems. This study examines cross-national variation in the relation between socioeconomic status (SES) and Grade 8 mathematics achievement, using data from TIMSS 2023. Employing an integrative data analysis approach, we estimated within-school, between-school, and contextual SES effects for 43 countries and applied random-effects meta-regression to examine system-level moderators. Results show that SES is a consistent predictor of achievement within schools, while between-school SES effects were substantially larger, indicating pronounced compositional inequalities. The pooled contextual SES effect was large (ES2 = 0.85, 95% CI [0.74, 0.95]) and highly heterogeneous across countries (τ² = 0.11; Q(42) = 642.40, p < .001), ranging from negligible in Nordic systems to very large in more stratified systems. Random-effects meta-regression analyses revealed that income inequality and between-school segregation strongly predicted contextual SES effects, whereas higher levels of democratic governance were associated with weaker effects.

• Ulitzsch, Esther; Nestler, Steffen; Johnson, Sverre Urnes; Snuggerud, Therese Ruud & Lüdtke, Oliver (2026). Capturing Heterogeneity in Levels, Variability, and Couplings across Persons and Time with a Hierarchical Time-Varying Coefficient Formulation of the Multivariate Normal. Psychometrika. ISSN 0033-3123. doi: 10.1017/psy.2026.10125. Fulltekst i vitenarkiv Vis sammendrag

  Time-varying coefficient modeling (TVCM), which represents regression coefficients as smooth functions of continuous time, provides a flexible framework for uncovering complex patterns of change in levels and associations in intensive longitudinal data. However, conventional TVCM remains limited to investigating directional effects across individuals. By introducing a TVCM formulation of the multivariate normal distribution, the present study extends TVCM to explore change in undirected associations (couplings) and variability, thereby broadening its utility for psychological research. We discuss three versions of this approach: an aggregate-level model and two hierarchical versions capturing interindividual differences in unfolding change, either via person-specific intercepts accounting for onset differences or through fully person-specific coefficient functions smoothed via partial pooling. To illustrate the proposed developments, we apply them to six weeks of intensive longitudinal data from 16 anxiety patients undergoing therapy and examine unfolding changes in the level and volatility of nervousness and threat monitoring, their coupling, as well as between-person heterogeneity in each of these. We further show how inspecting first-order derivatives of the coefficient functions supports identifying periods of stability and change. Finally, we discuss extensions incorporating person-level characteristics to explain heterogeneity in patterns of change and predict outcomes.

• Ulitzsch, Esther; Tetzlaff, Leonard; Goldhammer, Frank; Hahnel, Carolin; Kroehne, Ulf & Lüdtke, Oliver (2026). Does sequence mining uncover generalizable behavioral patterns? A methodological case study. Large-scale Assessments in Education. 14(1). doi: 10.1186/s40536-026-00289-8. Fulltekst i vitenarkiv
• Teo, Timothy; Scherer, Ronny; Fung, Artemis Sze Kok & Fung, Cynthia Sze Ling (2026). Factors Associated with Students’ Adoption of Artificial Intelligence Technology in Tertiary Education: A Meta-Analytic Review. Educational Research Review. ISSN 1747-938X. doi: 10.1016/j.edurev.2026.100804. Fulltekst i vitenarkiv Vis sammendrag

  This meta-analytic review synthesizes evidence on factors shaping post-secondary students’ adoption of artificial intelligence (AI) through a multilevel meta-analysis of 233 correlations from 32 studies (N = 16,977). A three-level model with correlated effects was identified as the optimal approach, yielding moderate positive weighted average correlations for individual (r = .57), contextual (r = .53), and technological factors (r = .50). No single category dominates adoption. Among specific predictors, usage intentions emerged as the strongest predictor (r = .64). Interestingly, perceived risks and trust showed weaker associations. Within the three broad categories of factors, the correlations exhibited large overall heterogeneity (I-squared >98%), which could be partly explained by the diversity of constructs and the students’ experience with AI. For practice, integration of AI should target specific constructs (e.g., learning objectives, intentions) and adapt to students’ experience levels. For research, the field over-relies on traditional technology acceptance models (e.g., TAM, UTAUT) and neglects AI-specific factors (e.g., anthropomorphism, ethics). Addressing this gap is important for advancing theory and evidence-based policy.

• Pietsch, Marcus; Eryilmaz, Nurullah; Strietholt, Rolf; Scherer, Ronny; Nascimento, Matheus Monteiro & Abels, Simone [Vis alle 7 forfattere av denne artikkelen] (2026). Endangering Just Sustainabilities: An IPD Meta-Analysis on the Social Stratification of Youth Environmental Attitudes in 65 Countries. Journal of Environmental Psychology. ISSN 0272-4944. 112. doi: 10.1016/j.jenvp.2026.103085. Fulltekst i vitenarkiv Vis sammendrag

  This study examines global socioeconomic inequality in environmental attitudes among young people. Using novel data from an international study of nearly 700,000 individuals across 65 countries, we combine a two-stage individual participant data (IPD) meta-analysis with a machine learning (Meta-CART) approach to examine cross-national variation in inequality in environmental attitudes and its associations with the macrostructural dimensions of capacity, distribution, and institutional openness. Higher socioeconomic status (SES) is consistently associated with stronger environmental preservation attitudes in both primary and secondary school. However, the strength of this relationship varies across countries. Revealing a structural paradox, press freedom and gross domestic product (GDP) interact to moderate SES disparities: the largest gaps appear in nations with both high institutional openness and affluence. Contrary to expectations, GDP and academic freedom do not exert consistent linear effects. The results suggest that national conditions shape the social distribution of environmental attitudes and that institutional enablers of knowledge may reinforce environmental engagement among advantaged youth, challenging the equity aspirations of sustainability education.

• Fütterer, Tim; König, Lars; Campos, Diego Gonzalez; Scherer, Ronny; Zitzmann, Steffen & Hecht, Martin (2026). Fast-Track Your Abstract Screening: Mastering ASReview for Accelerating Abstract Screening and Evaluating Decisions From Automatic-Screening Methods. Advances in Methods and Practices in Psychological Science (AMPPS). ISSN 2515-2459. 9(2), s. 1–36. doi: 10.1177/25152459261442150. Fulltekst i vitenarkiv Vis sammendrag

  Research syntheses, such as systematic reviews and meta-analyses, are crucial for synthesizing research to support evidence-based decision-making. However, the abstract-screening phase, during which researchers evaluate titles and abstracts for inclusion, is highly time-consuming and often results in cognitive biases and fatigue. To address these challenges, machine-learning-assisted tools, particularly those using active learning, have gained prominence. One such tool is Active Screening Review (ASReview), an open-source software for semiautomating title and abstract screening in systematic reviews. ASReview incorporates user feedback to prioritize relevant studies, reducing screening time and improving efficiency. Despite its potential, many researchers remain uncertain about integrating ASReview into their workflows and making evidence-based decisions regarding the tool’s configuration, training, and stopping criteria. In this tutorial, we provide a step-by-step guide to using ASReview, including practical examples from psychological research. We demonstrate the software’s application in two use cases: screening unlabeled abstracts using active learning and verifying results from automated-screening methods. In the tutorial, we also offer evidence-based recommendations for selecting stopping rules to balance sensitivity and efficiency. We also outline strategies for prescreening, data-set preparation, model setup, and progress monitoring to ensure that researchers can maximize the tool’s benefits while maintaining scientific rigor. By offering evidence-based guidance at each stage of the process for practitioners without coding skills, in this tutorial, we aim to help researchers harness artificial-intelligence-aided screening to enhance the quality and efficiency of research syntheses across disciplines.

• Ofstad, Sverre Berg; Tan, Tony & Andersson, Björn (2026). Comparing the difficulty of elective subjects in high school using a population-based Norwegian registry. Scandinavian Journal of Educational Research. ISSN 0031-3831. doi: 10.1080/00313831.2026.2648976. Fulltekst i vitenarkiv Vis sammendrag

  When grade point averages (GPAs) are used for admission into tertiary education, there is an implicit assumption that any two identical GPAs convey the same level of academic competency. However, this assumption does not hold when students are free to choose elective subjects and there are differences in their difficulty and dimensionality. In this study, we utilized item response theory to examine dimensionality and difficulty of 11 elective subjects in Norwegian high school, while accounting for self-selection bias, using data from 21,666 third year students. Our results show that the GPA is best represented as a multidimensional construct. Additionally, we demonstrate that the selection bias in observed subject grade averages for electives conceals considerable differences in difficulty levels. Hence, the GPA of Norwegian high school students does not only reflect academic competency but also the decisions made regarding which elective subjects to take.

• Zhang, Lijin; Rahal, Charles; Kanopka, Klint; Ulitzsch, Esther; Zhang, Zhiyong & Domingue, Benjamin W. (2026). Evaluating Model Predictive Performance in Confirmatory Factor Analysis with Binary Outcomes Using the InterModel Vigorish. Multivariate Behavioral Research. ISSN 0027-3171. doi: 10.1080/00273171.2026.2645212. Fulltekst i vitenarkiv
• Brinchmann, Ellen Irén & Braeken, Johan (2026). Rock-Paper-Screen: Fifth-graders' comprehension of simple and enhanced digital text. Learning and Individual Differences. ISSN 1041-6080. 128. doi: 10.1016/j.lindif.2026.102896. Fulltekst i vitenarkiv Vis sammendrag

  In this experiment, we compared 338 fifth-grade students' comprehension of text presented in three formats: paper, a simple linear digital format, and an enhanced digital format with an embedded dictionary. We also collected data on the students' prior reading comprehension skills, word reading efficiency, and daily digital device use, which allowed us to investigate whether medium effects varied depending on individual differences in reader characteristics. The results revealed no differences in students' text comprehension across the three reading formats. Additionally, there was no indication that performance across reading formats varied depending on students' prior reading skills or digital experience. Although we found no evidence that the embedded dictionary had an impact on students' text comprehension, the process data suggested that students used the dictionary strategically by clicking on words that were central to understanding the ideas presented in the text.

• Snuggerud, Therese Ruud; Ulitzsch, Esther; Nordahl, Henrik; Vrabel, Kari-Anne; Hoffart, Asle & Ebrahimi, Omid V. [Vis alle 9 forfattere av denne artikkelen] (2026). Dynamic changes in metacognitive mechanisms and symptoms during the attention training technique: Insights from ecological momentary assessment. Journal of Anxiety Disorders. ISSN 0887-6185. 119, s. 103136–103136. doi: 10.1016/j.janxdis.2026.103136. Fulltekst i vitenarkiv Vis sammendrag

  Background Anxiety disorders are prevalent and associated with substantial distress and impairment. Within the metacognitive model of emotional disorders, the cognitive attentional syndrome (CAS) and associated metacognitions maintain anxiety. The Attention Training Technique (ATT) targets the CAS and related metacognitions. However, less is known about how symptoms and metacognitive mechanisms evolve across treatment phases. Objective The study aimed to explore temporal dynamics in emotional symptoms and metacognitive mechanisms during group ATT by estimating mean levels, variability, and coupling across the pre-, during-, and post-treatment phases using ecological momentary assessment (EMA). Method Nineteen young adults with social anxiety disorder, panic disorder with agoraphobia, or generalized anxiety disorder participated. EMA was completed four times daily over six weeks: two weeks before treatment, two weeks during treatment, and two weeks after treatment. Each assessment included ten items measuring emotional symptoms (e.g., nervousness) and metacognitive mechanisms (e.g., worry, uncontrollability). Results Three patterns of change emerged: (1) reduced mean levels without change in variability (nervousness, self-consciousness, and worry); (2) reduced mean levels and reduced variability (threat monitoring); and (3) reduced mean levels and reduced variability during treatment, followed by an increase in variability after treatment (uncontrollability, difficulty collecting thoughts, and meta-worry). Conclusions The findings provide new insights into dynamic patterns of change in symptoms and metacognitive mechanisms before, during, and after ATT. Patterns of change in both intensity and stability may be relevant for understanding treatment response and vulnerability following treatment, suggesting the potential clinical relevance of monitoring these mechanisms.

• Braeken, Johan & Laar, Saskia Van (2026). Assessing the Measurement Equivalence of a set of Items: Item-specific Diagnostics on an Interpretable Metric. Practical Assessment, Research, and Evaluation (PARE). 31(1). doi: 10.7275/PARE.3048. Fulltekst i vitenarkiv
• Queiroz, Maria Gabriela Carneiro; Junior, Francisco Carlos Specian; Filho, Pedro Tadao Hamamoto; Santos, Thiago M.; Schauber, Stefan & Woltman, Andrea M. [Vis alle 7 forfattere av denne artikkelen] (2026). Comparison of cognitive workload between very short answer questions and multiple-choice questions: an eye-tracking experiment. Medical Education Online. 31(1). doi: 10.1080/10872981.2026.2621434. Fulltekst i vitenarkiv Vis sammendrag

  Very short answer questions (VSAQs) have gained attention for their superior psychometric properties compared to multiple-choice questions (MCQs). While VSAQs require knowledge recall, MCQs primarily involve knowledge recognition. This difference in cognitive processes may lead to varying cognitive workloads, defined as the amount of mental processing in working memory. Previous studies have not demonstrated consistent differences, likely due to reliance on self-reported measures. Eye tracking provides objective, process-level indicators of cognitive workload. This study investigated whether answering VSAQs requires a higher cognitive workload than answering MCQs. In a within-subject randomized crossover experiment, sixth-year medical students answered both VSAQs and MCQs. Cognitive workload was measured using screen-based eye tracking, focusing on the number of fixations and revisitations as objective indicators of mental effort. Data were analyzed using mixed-effects models. Thirty-four medical students participated, yielding 1,326 observations, which is the multiplication of the number of students by the number of questions (39 questions). Mixed-effects models showed a significant effect of question type on both workload indicators: VSAQs elicited more fixations and revisitations than MCQs (β_std = 0.30–0.39, p < .001). This effect remained after controlling for accuracy. Incorrect answers were associated with higher workload (β_std = −0.15–−0.16, p < .01). Heatmaps confirmed these findings, showing denser fixations on key diagnostic features for VSAQs and on answer options for MCQs. Answering VSAQs imposed a higher cognitive workload than MCQs. The presence of answer options in MCQs may reduce workload by providing unintentional cues, while VSAQs require active retrieval. Eye tracking proved valuable for distinguishing cognitive workload across assessment formats.

• Scherer, Ronny; Campos, Diego G.; Fütterer, Tim & Siddiq, Fazilat (2026). Unraveling gender disparities in teachers’ technological pedagogical and content knowledge: A large-scale meta-analytic review. Educational Research Review. ISSN 1747-938X. 50. doi: 10.1016/j.edurev.2026.100765. Fulltekst i vitenarkiv Vis sammendrag

  This large-scale meta-analytic review investigates gender disparities in pre- and in-service K-12 teachers' Technological Pedagogical and Content Knowledge (TPACK) self-efficacy and performance. Synthesizing 915 effect sizes from 102 primary studies and 420 teacher samples (N = 681,745 teachers) in 70 countries via multilevel meta-analyses, we found small but significant gender differences favoring male teachers in TPACK self-efficacy (d = −0.17), particularly in technological knowledge (d = −0.31) and technological pedagogical knowledge (d = −0.14). However, no significant gender disparities were observed in TPACK performance (d = −0.08). Our findings revealed considerable heterogeneity in these effects across samples, studies, countries, measures, and over time, which could be partly explained by factors such as educational level, country-level gender inequality, and economic development. These results do not support male teachers’ advantage in all aspects of technology-related teaching competencies. Our meta-analytic review further implies that gender disparities in TPACK vary across contexts or over time.

• Zhang, Lu & Andersson, Björn (2026). Likelihood ratio tests with marginal maximum likelihood using Laplace approximations and adaptive quadrature. Journal of Statistical Computation and Simulation. ISSN 0094-9655. s. 1–26. doi: 10.1080/00949655.2026.2616410. Fulltekst i vitenarkiv Vis sammendrag

  Estimation of generalized linear latent variable models is typically done with marginal maximum likelihood (MML) estimation which requires approximation methods to compute integrals without a closed form solution. In applied settings, likelihood ratio tests are often used to test hypotheses regarding parameter restrictions. This study examines the properties of likelihood ratio tests when using approximate MML estimation. We consider first- and second-order Laplace approximations, along with adaptive Gauss-Hermite quadrature with varying numbers of quadrature points. In two simulation studies we perform likelihood ratio tests, varying data types (binary, ordinal, and count data), sample sizes, and the number of observed variables. The first simulation examines slope parameter equivalence, while the second focuses on group restrictions in multiple group models. Results indicate adaptive quadrature and second-order Laplace approximations provide reliable inference across most settings. Meanwhile, the first-order Laplace approximation generally works well except in some binary data cases.

• Yoon, Ga; Chen, Chia-Wen & Midtbø, Tor (2025). Item performance in context: differential item functioning between Pilot and Formal administration of the Norwegian language test. Educational Assessment, Evaluation and Accountability. ISSN 1874-8597. 37(4), s. 527–562. doi: 10.1007/s11092-025-09467-8. Fulltekst i vitenarkiv Vis sammendrag

  The Norwegian language test (No: Norskprøven) administrated by Norwegian Directorate for Higher Education and Skills (Skills Norway) is a high-stakes assessment, the results of which are used by test takers in various ways. However, the item parameters used in multistage testing in Norskprøven are calibrated from a low-stakes situation, the Pilot test. Potential item parameter shift from the Pilot test to the Formal test might be a concern of practitioners since it reduces the test’s reliability and validity. In this study, differential item functioning (DIF) was examined between the Pilot and Formal reading comprehension tests in Norskprøven, using a log-likelihood ratio test method. Purification method was conducted to clean the invariant items and to improve the precision of DIF item detections. The results revealed 10 DIF items with a large effect size. DIF items also showed a tendency to vary more in item discrimination than in item difficulty. Lower discrimination parameters in the Pilot test indicated more random error and might have resulted from the low-stakes situation. Regarding the item features, item format and count of words, there was no clear evidence of being related to DIF. However, more items among the invariant items were piloted across two stages in multistage testing design, in contrast to the DIF items piloted in only one stage. Therefore, test administration and calibration design seem to be more related to the shift in individual item performance rather than to the item features.

• Andersson, Björn (2025). Multidimensional item response theory. I Nesi, Hilary & Milin, Petar (Red.), Reference Module in Social Sciences. Elsevier. ISSN 9780443157851. s. 1–7. doi: 10.1016/b978-0-443-26629-4.00225-2. Fulltekst i vitenarkiv Vis sammendrag

  Item response theory is a class of statistical models for the analysis of item scores from educational tests and psychological scales, where a latent construct is assumed to underlie the observed item scores. When multiple latent variables are measured by a set of item scores, a multidimensional item response theory model is required to accurately model the dependence between the observed variables. In this introduction to multidimensional item response theory, we present key models for binary, ordinal, and nominal data, and present how the models can be estimated and used to draw inferences at the individual level. We give examples of how to use the models to assess the psychometric properties of scales and tests, study the validity of test score uses and interpretations, and improve measurement precision for individual-level inferences.

• Domingue, Benjamin W.; Kanopka, Klint; Ulitzsch, Esther & Zhang, Lijin (2025). Implied probabilities of polytomous response functions for model-based prediction and comparison. Behaviormetrika. ISSN 0385-7417. 52(2), s. 705–705. doi: 10.1007/s41237-025-00262-9. Fulltekst i vitenarkiv
• Bulut, Okan; Yildirim-Erbasli, Seyma Nur; Ulitzsch, Esther & Gorgun, Guher (2025). Measuring Consumers’ Financial Well-Being: Uncovering the Causes and Consequences of Careless and Insufficient Effort in Self-Report Responses. Financial Counseling and Planning. ISSN 1052-3073. s. 0103–0103. doi: 10.1891/jfcp-2024-0103. Fulltekst i vitenarkiv Vis sammendrag

  This study investigates the prevalence and causes of careless and insufficient effort responding (C/IER) in a self-report instrument assessing financial well-being. Using an explanatory mixture item response modeling approach, we analyze data from 6,394 respondents who responded to the Consumer Financial Protection Bureau’s financial well-being and financial skills scales. We identify different patterns of C/IER and examine factors that contribute to its occurrence in the two scales. The results indicate that 14% of the responses were impacted by C/IER, with question characteristics (e.g., negatively worded or frequency-based questions) and respondent demographics (e.g., age, gender, and education level) exhibiting significant relationships with attentiveness. Respondents with higher financial knowledge were more attentive, while younger individuals and those experiencing financial shocks were more prone to careless responding. These findings underscore the importance of carefully designing self-report instruments to minimize response biases and ensure data quality. By addressing different question designs and respondent-level characteristics, future instruments can yield more reliable and valid insights into individuals’ financial well-being.

• Zhang, Lijin; Ulitzsch, Esther & Domingue, Benjamin W. (2025). Bayesian factor mixture modeling with response time for detecting careless respondents. Behavior Research Methods. ISSN 1554-351X. 57. doi: 10.3758/s13428-025-02797-x. Fulltekst i vitenarkiv Vis sammendrag

  Careless respondents inject noise into data which can distort research findings and compromise model fit. To address this, factor mixture modeling (FMM) has been widely used to identify careless respondents. Traditionally, researchers have relied on reverse-worded questions in FMM to facilitate the detection of careless responding. With the rise of online data collection platforms, response time has appeal as a means for understanding careless behavior. We introduce a Bayesian FMM that leverages this rich source of information to identify careless respondents. By jointly modeling responses and response time, this approach effectively identifies careless individuals rushing through the questionnaire without providing responses that reflect the to-be-measured traits. Our simulation studies demonstrate that this model accurately estimates parameters and classifies respondents as either attentive or careless, while maintaining error rates within acceptable limits. Furthermore, integrating response time enhances model convergence and the precision of classification and estimation. Using mediation models as an example, we illustrate how social science researchers can use this FMM approach to address careless responding in substantive research. An empirical study further tests the applicability of the proposed model in real-world scenarios, comparing its conclusions with traditional methods. To support its use, we provide an R function to streamline implementation.

• Zhang, Lijin; Domingue, Benjamin W.; Vogelsmeier, Leonie V. D. E. & Ulitzsch, Esther (2025). A Beta Mixture Model for Careless Respondent Detection in Visual Analogue Scale Data. Psychometrika. ISSN 0033-3123. 90(4), s. 1558–1581. doi: 10.1017/psy.2025.10041. Fulltekst i vitenarkiv Vis sammendrag

  Visual Analogue scales (VASs) are increasingly popular in psychological, social, and medical research. However, VASs can also be more demanding for respondents, potentially leading to quicker disengagement and a higher risk of careless responding. Existing mixture modeling approaches for careless response detection have so far only been available for Likert-type and unbounded continuous data but have not been tailored to VAS data. This study introduces and evaluates a model-based approach specifically designed to detect and account for careless respondents in VAS data. We integrate existing measurement models for VASs with mixture item response theory models for identifying and modeling careless responding. Simulation results show that the proposed model effectively detects careless responding and recovers key parameters. We illustrate the model’s potential for identifying and accounting for careless responding using real data from both VASs and Likert scales. First, we show how the model can be used to compare careless responding across different scale types, revealing a higher proportion of careless respondents in VAS compared to Likert scale data. Second, we demonstrate that item parameters from the proposed model exhibit improved psychometric properties compared to those from a model that ignores careless responding. These findings underscore the model’s potential to enhance data quality by identifying and addressing careless responding.

• Ulitzsch, Esther; Viechtbauer, Wolfgang; Lüdtke, Oliver; Myin-Germeys, Inez; Nagy, Gabriel & Nestler, Steffen [Vis alle 7 forfattere av denne artikkelen] (2025). Investigating the Effect of Experience Sampling Study Design on Careless and Insufficient Effort Responding Identified With a Screen-Time-Based Mixture Model. Psychological Assessment. ISSN 1040-3590. 37(8), s. 347–359. doi: 10.1037/pas0001379. Fulltekst i vitenarkiv Vis sammendrag

  When using the experience sampling method (ESM), researchers must navigate a delicate balance between obtaining fine-grained snapshots of phenomena of interest and avoiding undue respondent burden, which can lead to disengagement and compromise data quality. To guide that process, we investigated how questionnaire length and sampling frequency impact careless and insufficient effort responding (C/IER) as an important yet understudied aspect of ESM data quality. To this end, we made use of existing experimental ESM data (Eisele et al., 2022) from 163 students randomly assigned to one of two questionnaire lengths (30/60 items) and one of three sampling frequencies (3/6/9 assessments per day). We employed a novel mixture modeling approach (Ulitzsch, Nestler, et al., 2024) that leverages screen time data to disentangle attentive responding from C/IER and allows investigating how the occurrence of C/IER evolved within and across ESM study days. We found sampling frequency, but not questionnaire length, impacted C/IER, with higher frequencies resulting in higher overall C/IER proportions and sharper increases of C/IER across, but not within days. These effects proved robust across various model specifications. Further, we found no substantial relationships between model-implied C/IER and other engagement measures, such as self-reported attentiveness, attention checks, response-pattern-based attentiveness indicators, and compliance. Our findings contrast previous studies on noncompliance, suggesting that respondents may employ different strategies to lower the different types of burden imposed by questionnaire length and sampling frequency. Implications for designing ESM studies are discussed.

• Braeken, Johan; Lehre, Anne- Catherine W G & Marcq, Kseniia (2025). Pearls and perils when interpreting item-specific results from international large-scale educational assessments: The case of the ‘‘Nordic’’ fractions. Studies in Educational Evaluation. ISSN 0191-491X. 88. doi: 10.1016/j.stueduc.2025.101552. Fulltekst i vitenarkiv Vis sammendrag

  International large-scale assessments (ILSA) in education generate a lot of interest and debate among researchers and stakeholders alike. In this study, we take a deep-dive into a provocative claim that ‘‘Nordic students cannot do fractions’’ based on an item-specific result in the Trends in International Mathematics and Science Study (TIMSS) 2011. Using item response data from five cycles of the TIMSS mathematics achievement test, spanning from 2003 to 2019, across 12 countries representing diverse geographical regions (Africa, Europe, East Asia, and North America) and proficiency levels (from lower-performers to higher-performers), including about 54000 students per cycle and a total of about 800 mathematics items, we investigated the empirical basis and generalizability of the stated claim using an explanatory item response model (IRT) approach. Our study underscores the dangers of miscommunication, oversimplification, and overgeneralization of ILSA results, yet also highlights the hidden potential of deeper item-level analyses for educational researchers and stakeholders to generate critical inquiries into educational practice and our understanding of the subject domain.

• Fütterer, Tim; Campos, Diego Gonzalez; Gfrörer, Thomas; Lavelle-Hill, Rosa; Murayama, Kou & Scherer, Ronny (2025). AI tools for systematic literature reviews and meta-analyses in educational psychology: An overview and a practical guide. Learning and Individual Differences. ISSN 1041-6080. 126. doi: 10.1016/j.lindif.2025.102849. Fulltekst i vitenarkiv Vis sammendrag

  The rapid growth of research literature has made systematic reviews and meta-analyses increasingly time-consuming, limiting their utility in fast-evolving fields such as educational psychology. Artificial intelligence (AI) tools have enormous potential to streamline these processes, yet their adoption remains limited due to usability issues and a lack of systematic guidance. Out of 282 tools that we compiled from overviews that listed AI tools for research syntheses, we filtered a subset of 7 AI tools that met quality standards, such as transparency and accessibility. These tools were evaluated for their potential to support systematic reviews and meta-analyses in educational psychology. Our review highlights the tools' strengths, limitations, and ethical considerations for their responsible use by providing practical guidance and coding information.

• Skulberg, Katrine; Tan, Tony; Thøgersen, Dagfinn Mørkrid; Taraldsen, Knut & Bjørnebekk, Gunnar (2025). Multisystemic Therapy for Adolescents Through the COVID-19 Pandemic in Norway. Journal of Child and Family Studies. ISSN 1062-1024. 35, s. 124–140. doi: 10.1007/s10826-025-03222-9. Fulltekst i vitenarkiv Vis sammendrag

  Multiple studies have demonstrated that Multisystemic Therapy (MST) is an effective treatment for youths with serious conduct problems. This study aims to investigate the extent to which Norway’s MST was affected by the COVID-19 lockdowns and to examine the factors associated with treatment outcomes before and during the pandemic using a natural comparison group design. It included 2,021 adolescents (63.4% boys; Mage = 14.83, SDage = 1.60), and their families were referred to the MST for serious and persistent conduct problems. Treatment outcomes were compared among three temporal groups: A: fully prior to, B: partly prior to and C: during the pandemic. The Youth Level of Service/Case Management Inventory (YLS/CMI) and five behavioral treatment goals were assessed at intake and discharge. In addition, the five behavioral goals were assessed through parent interviews at 6-, 12-, and 18-month post-treatment. A decline in treatment fidelity was evident post-COVID-19 onset, and some subgroups experienced reduced benefits. Treatment fidelity was more strongly associated with treatment outcomes post-pandemic. Latent growth curve models revealed that boys entering MST post-pandemic had advantageous rates of behavioral improvements. Although most of the MST participants demonstrated progress in behavioral outcomes, several youths showed regress in treatment outcomes as the pandemic set in. Further studies are needed to identify factors particularly sensitive of subgroups of adolescents for deriving MST benefits in crisis situations.

• Rohatgi, Anubha; Scherer, Ronny & Hatlevik, Ove Edvard (2025). Digitale forskjeller i elevers digitale kompetanse og algoritmiske tenkning. I Rohatgi, Anubha & Hatlevik, Ove Edvard (Red.), Søkelys på digital læring: Sekundæranalyser fra ICILS 2023. Cappelen Damm Forskning/NOASP. ISSN 9788202905682. s. 141–174. doi: 10.23865/cdf.271.ch5. Fulltekst i vitenarkiv Vis sammendrag

  I mange land utgjør gapet mellom elever med og elever uten tilgang til teknologi en betydelig utfordring som forsterker sosiale ulikheter. Selv om dette ikke er like fremtredende i Norge, finnes det fortsatt digitale skiller, og disse har ikke blitt vesentlig redusert de siste ti årene. Begrepet digitale forskjeller omhandler forskjeller i tilgang til, bruk av og ferdigheter i digital teknologi. I dette kapittelet utforsker vi digitale forskjeller ved å undersøke sammenheng mellom elevers bakgrunn og elevenes prøveresultater i digital kompetanse (DK) og algoritmisk tenkning (AT). Det er en rekke variabler som har betydning for digitale skiller, for eksempel elevens bakgrunn, inkludert kjønn og språk som snakkes hjemme, familiens status, som foreldrenes utdanningsnivå og yrkesstatus, samt tilgang til digitale ressurser. Hver enkelt variabel har betydning for elevprestasjonen, men sammen kan de også forsterke forskjellene mellom elevenes resultater. Vi undersøker styrken i sammenhengene mellom de fem ulike variablene og utforsker hvorfor spesifikke elevgrupper har ulike resultater i DK og AT. Vi benytter flernivåanalyser som synliggjør interseksjonalitet som kjennetegner elever som risikerer å bli digitalt ekskluderte. Våre analyser av ICILS 2023-dataene viser signifikante relasjoner mellom de ulike variablene, som kjønn, språklig bakgrunn, familiens status og tilgang til digitale ressurser og elevenes prestasjoner i DK og AT. For de norske dataene ble det også avdekket tegn på interseksjonalitet. Elever som ikke snakker norsk hjemme, som mangler stabil internettforbindelse hjemme og har foreldre med lavt utdanningsnivå og yrker av lav status identifiseres som digitalt sårbare.

• Chen, Jianan; Brinchmann, Ellen Irén & Braeken, Johan (2025). Setting Discontinue Rules for Progressive Tests: A Practical and Transparent Toolkit. Assessment (ASM). ISSN 1073-1911. doi: 10.1177/10731911251385776. Fulltekst i vitenarkiv Vis sammendrag

  Progressive tests are a popular tailored test format where items are administered in increasing order of difficulty level and discontinued according to a rule system that should counter excessive response burden for test participants and guarantee efficient use of resources for test administrators. To facilitate evidence-based decision-making for setting appropriate discontinue rules, we propose a transparent approach that charts the impact of varied alternative discontinue rules on accuracy and efficiency. These A–E charts are based on retroactively applying discontinue rules to normative item response data. We show that a universal discontinue rule likely does not exist and that the optimal rule varies as a function of the desired efficiency-accuracy trade-off suitable for the intended test use and target population. The proposed approach provides a pragmatic solution for practitioners, researchers, test developers, and test publishers to rethink the existing discontinue rules, systematically evaluate the alternatives, and set appropriate rules.

• Paredes, Yuriko Kirilovna Sosa & Andersson, Björn (2025). Linking the first, second and third IEA science studies. Studies in Educational Evaluation. ISSN 0191-491X. 87. doi: 10.1016/j.stueduc.2025.101529. Fulltekst i vitenarkiv Vis sammendrag

  International comparative assessments are crucial for examining country-level educational changes over time, but requires a comparable achievement scale. This study develops a unified science achievement scale that links the First and Second International Science Studies with the Trends in International Mathematics and Science Study for upper primary school students. The science measures in each study are analyzed with respect to construct consistency, inferences, population differences, and test conditions. Analyses showed that, despite some evolution in frameworks and a transition from age-based to grade-based definitions, the core constructs remain consistent, enabling similar inferences. By identifying common items among the three studies, we rigorously assessed measurement invariance using item response modeling, with partial invariance models supporting reliable long-term comparisons. We applied latent regression models using contextual student data, creating imputed scale scores for each student. These results enable vast further research to investigate long-term inequalities in educational outcomes.

• Braeken, Johan & Laar, Saskia Van (2025). Reducing Calibration Bias for Person Fit Assessment by Mixture Model Expansion. Educational and Psychological Measurement. ISSN 0013-1644. doi: 10.1177/00131644251364252. Fulltekst i vitenarkiv Vis sammendrag

  Measurement appropriateness concerns the question of whether the test or survey scale under consideration can provide a valid measure for a specific individual. An aberrant item response pattern would provide internal counterevidence against using the test/scale for this person, whereas a more typical item response pattern would imply a fit of the measure to the person. Traditional approaches, including the popular Lz person fit statistic, are hampered by their two-stage estimation procedure and the fact that the fit for the person is determined based on the model calibrated on data that include the misfitting persons. This calibration bias creates suboptimal conditions for person fit assessment. Solutions have been sought through the derivation of approximating bias-correction formulas and/or iterative purification procedures. Yet, here we discuss an alternative one-stage solution that involves calibrating a model expansion of the measurement model that includes a mixture component for target aberrant response patterns. A simulation study evaluates the approach under the most unfavorable and least-studied conditions for person fit indices, short polytomous survey scales, similar to those found in large-scale educational assessments such as the Program for International Student Assessment or Trends in Mathematics and Science Study.

• Campos, Diego Gonzalez; Koskinen, Antti; Munkácsy, Balázs; Rolfe, Victoria; Patsis, Panagiotis & Tóth, Edit [Vis alle 12 forfattere av denne artikkelen] (2025). Educational inequalities in Europe: A scoping review of longitudinal studies in K-12 education. Studies in Educational Evaluation. ISSN 0191-491X. 87. doi: 10.1016/j.stueduc.2025.101523. Fulltekst i vitenarkiv Vis sammendrag

  Educational inequalities are a critical challenge. Understanding how these disparities emerge, evolve, and can be addressed requires longitudinal evidence. However, most existing research syntheses rely on cross-sectional data. In this scoping review, we map longitudinal research on educational inequalities among K–12 students in Europe. Drawing on 161 studies published between 1986 and 2024, we examined how disparities related to socioeconomic status, migration or ethnic background, gender, geographical location, or their intersections have been studied over time. Academic achievement and socioeconomic status were the most studied outcome and inequality dimensions, while inequalities in socio-emotional outcomes and geographical location received less attention. Four research clusters were also identified in the literature: individual psychological determinants, mechanisms of inequality, developmental patterns of inequality, and structural determinants of educational transitions. Overall, the findings underscore the heterogeneity of empirical approaches to longitudinal research on educational inequality in Europe and highlight gaps for future research.

• Aydın, Burak; Fokkema, Marjolein; Eryilmaz, Nurullah; Muijs, Daniel; Scherer, Ronny & Pietsch, Marcus (2025). How school leadership and innovation shape instructional pathways to student achievement across nations: Evidence from multilevel structural equation modeling and decision tree analysis. Studies in Educational Evaluation. ISSN 0191-491X. 87. doi: 10.1016/j.stueduc.2025.101521. Fulltekst i vitenarkiv Vis sammendrag

  Educational leadership, innovation and teaching play essential roles in shaping student achievement. However, extant literature primarily has relied on linear modelling approaches and has not focused on substantively testing a theory. The present study employs multilevel structural equation modelling (ML-SEM) and multilevel decision trees (MLM trees) to investigate associations between school leadership, team innovation, cognitive activation and student achievement using PISA-TALIS 2018 linked data across seven countries: Australia; Colombia; Czech Republic; Denmark; Georgia; Malta; and Türkiye. The ML-SEM findings indicated no significant indirect effects from leadership on achievement. The MLM trees highlighted country-specific patterns in associations between school leadership, innovation and student achievement, revealing potential nonlinear relationships. These findings suggest that the relationship between leadership, instructional practices and achievement is highly context-dependent. The study contributes to the literature by offering a comparative analysis of ML-SEM and MLM trees, highlighting traditional linear models’ limitations in educational research.

• Ye, Wangqiong; Aleman, Jose Manuel Arencibia; Nilsen, Trude & Widding-Havnerås, Tarjei (2025). Will what helps now help later? The role of school spending in shaping academic resilience and future success for Norway’s disadvantaged students. International Journal of Educational Research. ISSN 0883-0355. 134. doi: 10.1016/j.ijer.2025.102804. Fulltekst i vitenarkiv Vis sammendrag

  Academic resilience, which describes how disadvantaged students excel in schools, has gained attention for its potential to narrow achievement gaps and reduce educational inequalities. While malleable protective factors in the learning environment have been extensively studied, institutional factors like school spending have received less attention. This study utilizes Norwegian register data to examine how the amount and allocation of school spending may influence academic resilience through regression analyses. Instrumental variable (IV) analyses are then used to further examine support for the relationship between academic resilience and long-term outcomes, such as educational attainment and income by age 30. The findings reveal that class size and student-teacher ratios, while not predictors of academic resilience, are associated with long-term outcomes. The amount of school spending influences academic resilience; however, its effects on educational attainment and income vary. The allocation of school spending influences educational attainment and income, with specific investments in Norwegian language and special education support significantly predicting academic resilience. Academic resilience is predictive of both educational attainment and income, showing a more pronounced relationship with educational attainment. The IV analyses show consistent results. This study contributes to the literature by detailing the influence of school spending on academic resilience and its connection to future success, thus supporting policy development aimed at fostering social inclusion.

• Backfisch, Iris; Scherer, Ronny; Schneider, Jürgen & Lachner, Andreas (2025). How Valid, Really? A Meta-Analysis of the Validity Evidence of Technological Pedagogical and Content Knowledge (TPACK) Self-Report Assessments. Educational Research Review. ISSN 1747-938X. 49. doi: 10.1016/j.edurev.2025.100718. Fulltekst i vitenarkiv Vis sammendrag

  Technological Pedagogical and Content Knowledge (TPACK) plays a central role for teacher training and measuring teachers’ preparedness for technology-enhanced teaching. Hence, crafting a validity argument of the popular TPACK self-assessments is key to facilitating evidence-based practices in these areas. However, the existing body of validity evidence abounds in divergent findings, for instance on the factor structure of TPACK self-reports across pre- and in-service teacher samples. To synthesize this evidence and capture its heterogeneity, we conducted a systematic review with a meta-analysis. First, we examined the internal validity of TPACK by analyzing its factor structure using meta-analytic structural equation and network modeling. Second, we compared two theoretical models describing the relations among the TPACK dimension—the transformative and integrative models—against the data. Third, we assessed the external validity of self-reported TPCK by examining its correlation with actual technology use in teaching. Our findings revealed two distinguishable dimensions of TPACK for both pre- and in-service teacher samples, which represented the technological and non-technological dimensions. The integrative model provided an overall better fit to the data. Moreover, TPACK self-reports were moderately correlated with technology use (r̄ s = 0.23–0.49) and exhibited substantial heterogeneity. Our results underscore the importance of generating and considering both internal and external validity evidence when applying TPACK assessments to generate implications for educational research and practice.

• Xiao, Xingyao; Rabe-Hesketh, Sophia & Skrondal, Anders (2025). Bayesian Identification and Estimation of Growth Mixture Models. Psychometrika. ISSN 0033-3123. 90(2), s. 442–475. doi: 10.1017/psy.2025.11. Fulltekst i vitenarkiv Vis sammendrag

  This article addresses problematic behaviors of Markov chain Monte Carlo (MCMC) methods for finite mixture models due to what we call degenerate nonidentifiability. We discuss the reasons for these behaviors, propose diagnostics to detect them, and show through simulations that using more informative priors than the vague defaults can mitigate the problems in growth mixture models (GMMs). Our motivating example is an application of GMMs to data from the National Longitudinal Survey of Youth (NLSY) to examine heterogeneity in the development of reading skills in children aged 6–14. We also suggest ways of describing and visualizing within-class heterogeneity in GMMs, provide a literature review of likelihood identification and Bayesian identification, propose a viable definition of Bayesian identification for latent variable models based on the marginal likelihood (integrated over the latent variables), and give a brief didactic description of Hamiltonian Monte Carlo (HMC) as implemented in Stan.

• Schauber, Stefan & Homer, Matt (2025). Challenging the norm: Length of exams determined by classification accuracy or reliability. Medical Education. ISSN 0308-0110. doi: 10.1111/medu.15742. Fulltekst i vitenarkiv Vis sammendrag

  Purpose This paper challenges the notion that reliability indices are appropriate for informing test length in exams in medical education, where the focus is on ensuring defensible pass-fail decisions. Instead, we argue that using classification accuracy instead better suited to the purpose of exams in these cases. We show empirically, using resampled test data from a range of undergraduate knowledge exams, that this is indeed the case. More specifically, we address the hypothesis that the use of classification accuracy results in recommending shorter test lengths as compared to when using reliability. Method We analysed data from previous exams from both pre-clinical and clinical phases of undergraduate medical education. We used a re-sampling procedure in which both the cut-score and test length of repeatedly generated synthetic exams were varied systematically. N = 52 500 datasets were generated from the original exams. For each of these both reliability and classification accuracy indices were estimated. Result Results indicate that only classification accuracy, not reliability, varies in relation to the cut-score for pass-fail decisions. Furthermore, reliability and classification accuracy are differently related to test length. The optimal test length for using reliability was around 100 items, independent of pass-rates. For classification accuracy, recommendations are less generic. For exams with a small percentage of failed decisions (i.e., 5% or less), an item size of 50 did, on average, achieve an accuracy of 95% correct classifications. Conclusions We suggest a move towards the employment of classification accuracy using existing tools, whilst still using reliability as a complement. The benefits of re-thinking current test design practice include minimizing the burden of assessment on candidates and test developers. Item writers could focus on developing fewer, but higher quality, items. Finally, we stress the need to consider the effects of the balance false positive and false negative decisions in pass/fail classifications.

• Maehler, Débora B.; Martin, Silke; Gorges, Julia & Scherer, Ronny (2025). Determinants of cognitive skills in adulthood: Age cohort patterns. International Journal of Lifelong Education. ISSN 0260-1370. 44(2), s. 131–152. doi: 10.1080/02601370.2024.2390070. Fulltekst i vitenarkiv Vis sammendrag

  The current study examines change and stability of cognitive skills across life stages and delves into the factors contributing to changes in cognitive skills. Specifically, we examine the associations between cognitive skills and individual characteristics and contextual factors in a large sample of German adults aged 16–65 years (N = 2,430; PIAAC-L data). Across all age cohorts, cognitive skills were predicted mainly by a person’s educational background, but they were also associated with factors related to socialisation, lifestyle and personality. The findings indicate that specific factors influence cognitive skills at different life stages. For example, personality was related to cognitive skills solely in the two younger age cohorts, whereas the impacts of contextual factors were most pronounced in the middle cohort.

• Vogelsmeier, Leonie V. D. E.; Jongerling, Joran & Ulitzsch, Esther (2025). Accounting for Measurement Invariance Violations in Careless Responding Detection in Intensive Longitudinal Data: Exploratory vs. Partially Constrained Latent Markov Factor Analysis. Multivariate Behavioral Research. ISSN 0027-3171. 60(5), s. 878–897. doi: 10.1080/00273171.2025.2492016. Fulltekst i vitenarkiv Vis sammendrag

  Intensive longitudinal data (ILD) collection methods like experience sampling methodology can place significant burdens on participants, potentially resulting in careless responding, such as random responding. Such behavior can undermine the validity of any inferences drawn from the data if not properly identified and addressed. Recently, a confirmatory mixture model (here referred to as fully constrained latent Markov factor analysis, LMFA) has been introduced as a promising solution to detect careless responding in ILD. However, this method relies on the key assumption of measurement invariance of the attentive responses, which is easily violated due to shifts in how participants interpret items. If the assumption is violated, the ability of the fully constrained LMFA to accurately identify careless responding is compromised. In this study, we evaluated two more flexible variants of LMFA—fully exploratory LMFA and partially constrained LMFA—to distinguish between careless and attentive responding in the presence of non-invariant attentive responses. Simulation results indicated that the fully exploratory LMFA model is an effective tool for reliably detecting and interpreting different types of careless responding while accounting for violations of measurement invariance. Conversely, the partially constrained model struggled to accurately detect careless responses. We end by discussing potential reasons for this.

• Emslander, Valentin; Holzberger, Doris; Ofstad, Sverre Berg; Fischbach, Antoine & Scherer, Ronny (2025). Teacher-Student Relationships and Student Outcomes: A Systematic Second-Order Meta-Analytic Review. Psychological bulletin. ISSN 0033-2909. 151(3), s. 365–397. doi: 10.1037/bul0000461. Fulltekst i vitenarkiv Vis sammendrag

  Teacher–student relationships (TSRs) play a vital role in establishing a positive classroom climate and promoting positive student outcomes. Several meta-analyses have suggested significant correlations between positive TSRs and, for example, academic achievement, motivation, executive functions, and well-being, as well as between negative TSRs that result in behavior problems or bullying. These meta-analyses have differed substantially in TSR–outcome relationships, moderators, and methodological quality, thus complicating the interpretation of these findings. In this preregistered systematic review of meta-analyses plus original second-order meta-analyses (SOMAs), we aimed to (a) synthesize the meta-analytic evidence on relations between TSRs and student outcomes, (b) map influential moderators of these relations, and (c) assess the methodological quality of the meta-analyses. We synthesized over 70 years of educational research across 26 meta-analyses encompassing 119 meta-analytic effect sizes based on approximately 2.64 million prekindergarten and K–12 students. We conducted several three-level SOMAs and found that TSRs had similar large significant relations with eight clusters of student outcomes: academic achievement, academic emotions, appropriate student behavior, behavior problems, executive functions and self-control, motivation, school belonging and engagement, and well-being. The link with bullying was only marginally significant. Our moderator analyses suggested a larger TSR–outcome link for middle and high school students. Although more recent meta-analyses fulfilled more methodological quality criteria, these differences were not associated with TSR–outcome relations. These results map the field of TSR research; present their relations, moderators, and methodological quality in meta-analyses; and show how TSRs are equally important for a wide range of student outcomes and samples.

• Simonseier, Bianca A.; Kampmann, Kristin; Staub, Jacqueline & Scherer, Ronny (2025). The effects of programming interventions in early childhood: A meta-analysis. Learning and Individual Differences. ISSN 1041-6080. 120. doi: 10.1016/j.lindif.2025.102699. Fulltekst i vitenarkiv Vis sammendrag

  While recent meta-analyses confirm the positive effects of programming interventions in school and university students, it is still unclear whether teaching programming in early childhood settings is similarly effective. Our meta-analysis clarifies this issue by synthesizing the effects of programming interventions for 3- to 7-year-old children. Moreover, we propose and employ a taxonomy for instructional approaches commonly used in programming interventions to systematically analyze the relevance of instructional design. The results from 25 early childhood studies and 82 effect sizes showed a medium overall effect of d = +0.73, 95 % CI [0.51, 0.96]. Programming interventions effectively enhanced programming knowledge, computational thinking, mathematics knowledge, and cognitive skills. Our findings highlight the effectiveness of unplugged over screen-based activities and interventions, including robotics, over those using other program outputs. The results provide educators with valuable information about instructional methods. The taxonomy can guide researchers and educators in implementing instructional strategies when teaching programming.

• Ulitzsch, Esther; Belov, Dmitry; Lüdtke, Oliver & Robitzsch, Alexander (2025). Using Item Parameter Predictions for Reducing Calibration Sample Requirements—A Case Study Based on a High-Stakes Admission Test. Journal of Educational Measurement. ISSN 0022-0655. doi: 10.1111/jedm.12426. Fulltekst i vitenarkiv Vis sammendrag

  In item difficulty modeling (IDM), item parameters are predicted from the items' linguistic features, aiming to ultimately render item calibration redundant. Current IDM applications, however, commonly do not yield the required prediction accuracy. To immediately exploit even somewhat inaccurate IDM predictions, we blend IDM with established Bayesian estimation techniques. We propose a two-step approach where (1) IDM predictions are obtained and (2) employed to construct informative prior distributions. We evaluate the approach in a case study on small-sample calibration of the 3PL in a high-stakes test. First, concerning implementation, we find computationally efficient penalized maximum likelihood estimation to be comparable to the best-performing MCMC-based approach. Second, we investigate sample size reductions achievable with state-of-the-art IDM predictions, finding negligible gains compared to merely considering the historical distribution of parameters. Third, we evaluate the prediction accuracy required for a targeted sample size reduction by gradually increasing simulated IDM prediction accuracies. We find that required accuracies can counterbalance each other, allowing calibration sample size to be reduced when either high-quality item difficulty predictions or good predictions of item discriminations and pseudo-guessing parameters are available. We argue that these evaluations provide new, portable IDM benchmarks quantifying performance in terms of achievable sample size reductions.

• Gilbert, Joshua B.; Zhang, Lijin; Ulitzsch, Esther & Domingue, Benjamin W. (2025). Polytomous explanatory item response models for item discrimination: Assessing negative-framing effects in social-emotional learning surveys. Behavior Research Methods. ISSN 1554-351X. 57(4). doi: 10.3758/s13428-025-02625-2. Fulltekst i vitenarkiv Vis sammendrag

  Modeling item parameters as a function of item characteristics has a long history but has generally focused on models for item location. Explanatory item response models for item discrimination are available but rarely used. In this study, we extend existing approaches for modeling item discrimination from dichotomous to polytomous item responses. We illustrate our proposed approach with an application to four social-emotional learning surveys of preschool children to investigate how item discrimination depends on whether an item is positively or negatively framed. Negative framing predicts significantly lower item discrimination on two of the four surveys, and a plausibly causal estimate from a regression discontinuity analysis shows that negative framing reduces discrimination by about 30% on one survey. We conclude with a discussion of potential applications of explanatory models for item discrimination.

• Scherer, Ronny & Emslander, Valentin (2025). Utilizing primary study quality in meta-analyses in psychology: A step-by-step tutorial. Psychological methods. ISSN 1082-989X. doi: 10.1037/met0000751. Fulltekst i vitenarkiv Vis sammendrag

  Evaluating the quality of primary studies is a key step in meta-analyses in psychology. This step is aimed at reducing the risk of bias and establishing the validity of the inferences drawn from the meta-analytic findings. However, the extant body of research offers little guidance on how to represent and incorporate primary study quality (PSQ) in meta-analyses, and some common procedures, such as creating sum scores from a set of quality indicators, often lack the backing from measurement models. Addressing these issues, we present a tutorial that guides researchers in their analytic decisions and approaches to represent and incorporate PSQ in their meta-analyses. Specifically, we describe, review, and illustrate approaches to (a) represent PSQ by single or multiple quality indicators or aggregated scores; (b) examine the moderator effects of PSQ; and (c) test the sensitivity of moderator effects to PSQ. We illustrate these approaches and present a step-by-step tutorial with analytic code for researchers’ guidance. We also encourage meta-analysts to take a measurement perspective on representing PSQ if multiple quality indicators are aggregated into a quality score. Moreover, we argue for conducting moderator sensitivity analyses to obtain more evidence on the impact of PSQ in a meta-analysis.

• Hummelen, Benjamin; Langjord, Tuva; Paap, Muirne C. S.; Folmo, Espen Jan; Pedersen, Geir & Braeken, Johan (2025). Differential Item Functioning for Gender and Age of the DSM-IV Borderline Personality Disorder Criteria in a Large Clinical Sample. Journal of Psychopathology and Behavioral Assessment. ISSN 0882-2689. 47(1). doi: 10.1007/s10862-024-10183-8. Fulltekst i vitenarkiv
• Veletic, Jelena & Scherer, Ronny (2025). The Nordic perspective on the association between distributed leadership and teacher job satisfaction: A multilevel SEM approach. International Journal of Educational Research. ISSN 0883-0355. 130. doi: 10.1016/j.ijer.2025.102559. Fulltekst i vitenarkiv Vis sammendrag

  In this study, we address the lack of robust empirical evidence on the association between distributed leadership and teacher job satisfaction with the work environment in the Nordic countries by examining this relationship at the teacher and school levels, for both teachers and principals. We also explore how teachers’ shared perceptions of distributed leadership are associated with individual job satisfaction (i.e., contextual effects), surpassing the limitations of the single-level analyses. Using data from 11,788 teachers in 654 schools who participated in the Teaching and Learning International Survey (TALIS) 2018 in Denmark, Finland, Norway, and Sweden, we found a strong positive association between teacher job satisfaction with the work environment and teacher-reported distributed leadership at the school level, and a moderately strong association at the teacher level. In contrast, we found a negative contextual effect which suggested that stronger distributed leadership at the school level was associated with lower satisfaction of individual teacher in Norway and Finland. Additionally, principal-reported distributed leadership was weakly, positively associated with teacher job satisfaction with the work environment only in Sweden. This study shows mixed results: at the teacher level, stronger distributed leadership has potential to enhance job satisfaction, while at the school level, it can be linked to lower individual teacher satisfaction as shown in Norway and Finland. Our study further highlights the distinction between teacher- and principal-reported distributed leadership, as well as individual- and shared-teacher perceptions, and their potential differential effects on job satisfaction.

• Hautz, Wolf E; Marcin, Thimo; Hautz, Stefanie C.; Schauber, Stefan; Krummrey, Gert & Müller, Martin [Vis alle 22 forfattere av denne artikkelen] (2025). Diagnoses supported by a computerised diagnostic decision support system versus conventional diagnoses in emergency patients (DDX-BRO): a multicentre, multiple-period, double-blind, cluster-randomised, crossover superiority trial. The Lancet Digital Health. 7(2), s. e136–e144. doi: 10.1016/S2589-7500(24)00250-4. Fulltekst i vitenarkiv Vis sammendrag

  Background Diagnostic error is a frequent and clinically relevant health-care problem. Whether computerised diagnostic decision support systems (CDDSSs) improve diagnoses is controversial, and prospective randomised trials investigating their effectiveness in routine clinical practice are scarce. We hypothesised that diagnoses made with a CDDSS in the emergency department setting would be superior to unsupported diagnoses. Methods This multicentre, multiple-period, double-blind, cluster-randomised, crossover superiority trial was done in four emergency departments in Switzerland. Eligible patients were adults (aged ≥18 years) presenting with abdominal pain, fever of unknown origin, syncope, or non-specific symptoms. Emergency departments were randomly assigned (1:1) to one of two predefined sequences of six alternating periods of intervention or control. Patients presenting during an intervention period were diagnosed with the aid of a CDDSS, whereas patients presenting during a control period were diagnosed without a CDDSS (usual care). Patients and personnel assessing outcomes were masked to group allocation; treating physicians were not. The primary binary outcome (false or true) was a composite score indicating a risk of reduced diagnostic quality, which was deemed to be present if any of the following occurred within 14 days: unscheduled medical care, a change in diagnosis, an unexpected intensive care unit admission within 24 h if initially admitted to hospital, or death. We assessed superiority of supported versus unsupported diagnoses in all consenting patients using a generalised linear mixed effects model. All participants who received any study treatment (including control) and completed the study were included in the safety analysis. This trial is registered with ClinicalTrials.gov (NCT05346523) and is closed to accrual. Findings Between June 9, 2022, and June 23, 2023, 15 845 patients were screened and 1204 (591 [49·1%] female and 613 [50·9%] male) were included in the primary efficacy analysis. The median age of participants was 53 years (IQR 34–69). Diagnostic quality risk was observed in 100 (18%) of 559 patients with CDDSS-supported diagnoses and 119 (18%) of 645 with unsupported diagnoses (adjusted odds ratio 0·96 [95% CI 0·71–1·3]). 94 (7·8%) patients suffered a serious adverse event, none related to the study. Interpretation Use of a CDDSS did not reduce the occurrence of diagnostic quality risk compared with the usual diagnostic process in adults presenting to emergency departments. Future research should aim to identify specific contexts in which CDDSSs are effective and how existing CDDSSs can be adapted to improve patient outcomes.

• Scherer, Ronny & Campos, Diego Gonzalez (2025). Meta-analyzing correlation matrices in the presence of hierarchical effect size multiplicity. Research Synthesis Methods. ISSN 1759-2879. 16(6), s. 828–858. doi: 10.1017/rsm.2025.10027. Fulltekst i vitenarkiv Vis sammendrag

  To synthesize evidence on the relations among multiple constructs, measures, or concepts, meta-analyzing correlation matrices across primary studies has become a crucial analytic approach. Common meta-analytic approaches employ univariate or multivariate models to estimate a pooled correlation matrix, which is subjected to further analyses, such as structural equation modeling. In practice, meta-analysts often extract multiple correlation matrices per study from various samples, study sites, labs, or countries, thus introducing hierarchical effect size multiplicity into the meta-analytic data. However, this feature has largely been ignored when pooling correlation matrices for meta-analysis. To contribute to the methodological development in this area, we describe a multilevel, multivariate, and random-effects modeling approach, which pools correlation matrices meta-analytically and, at the same time, addresses hierarchical effect size multiplicity. Specifically, it allows meta-analysts to test various assumptions on the dependencies among random effects, aiding the selection of a meta-analytic baseline model. We describe this approach, present four working models within it, and illustrate them with an example and the corresponding R code.

• Belov, Dmitry I.; Lüdtke, Oliver & Ulitzsch, Esther (2025). A supervised learning approach to estimating IRT models in small samples. British Journal of Mathematical & Statistical Psychology. ISSN 0007-1102. doi: 10.1111/bmsp.12396. Fulltekst i vitenarkiv
• Marcq, Kseniia & Braeken, Johan (2025). From framework to functionality: A cross-country analysis of PISA 2018 reading assessment framework’s item features as determinants of item difficulty. Large-scale Assessments in Education. doi: 10.1186/s40536-025-00261-y. Fulltekst i vitenarkiv Vis sammendrag

  Background Theoretical frameworks excel in conceptualising reading literacy, yet their value hinges on their applicability for real-world purposes, such as assessment. By combining diverse theoretical frameworks, the Programme for International Student Assessment (PISA) 2018 designed an assessment framework for assessing the reading literacy of over 600,000 students globally. The PISA 2018 framework was operationalised into assessment items through construct-relevant item features aimed at determining item difficulty by mirroring the expectations set by the assessment framework. Methods Using cross-classified mixed effects models, we model the PISA 2018 reading item difficulty as a function of the PISA-defined construct-relevant and -irrelevant item features to gauge the extent to which item features determined item difficulty on average and across 71 countries. Results All five, delineated by the PISA framework, construct-relevant item features combined only reduce the variance in item difficulty by an average of 20%, with ‘Text Type’ being the only promising predictor. However, the construct-irrelevant ‘Item Format’ alone has half the predictive value of all construct-relevant features combined (10%), implying that external factors may unduly influence student outcomes. Conclusions These findings suggest that the functionality of item features within the framework leaves much to be desired, pointing to considerable shortcomings that necessitate a re-evaluation of the framework and its more effective implementation. We discuss implications for future PISA design and other stakeholders and call for bridging the gap between the PISA reading assessment framework and its practical application to ensure its validity, effectiveness, and credibility among educational stakeholders worldwide.

• Hukkelberg, Silje Sommer; Steinmann, Isa & Nærde, Ane (2025). The relative age effect on teacher-rated academic competence: a study among early primary school students. Scandinavian Journal of Educational Research. ISSN 0031-3831. 70(1), s. 18–31. doi: 10.1080/00313831.2025.2459402. Fulltekst i vitenarkiv Vis sammendrag

  Older students tend to outperform their younger classmates on academic achievement tests, an asymmetry known as the relative age effect (RAE). In Norway, students are not formally tested in the early school years, and less is known about how the RAE impacts teacher evaluations. In this study, we investigated the RAE on teacher-rated academic competence in 2nd grade and social skills, externalizing, and internalizing problems in 1st grade. The data stem from the Behavior Outlook Norwegian Developmental Study (BONDS) and included N = 1,157 children (48.2% girls), their families and teachers. Path models were specified to investigate direct and indirect effects between the variables. The results showed a RAE on academic competence and social skills but not on externalizing and internalizing problems. Mediation analysis indicated that part of the RAE on academic competence could be attributed to students’ social skills. Implications of the findings are discussed.

• Hagen, Tor Løstegaard; Tan, Tony; Thøgersen, Dagfinn Mørkrid & Bjørnebekk, Gunnar (2025). Functional family therapy across the COVID-19 pandemic. Frontiers in Psychology. 16. doi: 10.3389/fpsyg.2025.1531738. Fulltekst i vitenarkiv Vis sammendrag

  The present study examines the effectiveness of the Functional Family Therapy (FFT) program in Norway through the COVID-19 pandemic using archival data. Since Norway's national lockdowns imposed significant disruptions on program delivery, concerns arose regarding FFT's effectiveness during the public health emergency; however, this hypothesis remained untested. This exploratory study uses a multigroup quasi-experimental design by comparing clients before and during the COVID-19 pandemic. The data includes 518 adolescents and their families referred to FFT for serious and persistent antisocial behavior. The mean age of the youth was 14.2 years and the representation between boys and girls was close to equal (49.4% girls). Statistical analyses showed that clients' attributes at admission remained stable, and FFT remained effective in bringing about behavioral improvements and risk reductions throughout the pandemic. Additionally, regression results suggested that older clients tended to have greater reductions in behavioral problems and risk levels before lockdowns; however, this age effect disappeared after the onset of COVID. Clients returning home from institutional care reported stronger behavioral gains, whereas those living in foster homes showed less favorable risk outcomes. Collectively, these results suggest that both the FFT target population and treatment outcomes remained stable despite disruptions in program delivery during the pandemic.

Sjå alle arbeida i NVA

• Melby-Lervåg, Monica; Hulme, Charles; snowling, maggie; west, gillian; Crane, claudine & Campos, Diego Gonzalez (2026). The efficacy of early language intervention on reading: an individual participant data (IPD) analysis. Fulltekst i vitenarkiv
• Chen, Jianan & Braeken, Johan (2026). Traditional Progressive Tests with Stopping Rules from Computerized Adaptive Testing: Do Potential Advantages Outweigh Added Complexity? Fulltekst i vitenarkiv
• Wang, Lei; Yang, Junyi; Scherer, Ronny; Melby-Lervåg, Monica & Anmarkrud, Øistein (2026). The Relationship Between Screen Use and Language and Literacy Skills in Childhood: The Roles of Family SES and Children's Prior Ability. doi: https:/www.triplesr.org/sites/default/files/2026-07/detailed_programme%20July%2015%202026.pdf. Fulltekst i vitenarkiv
• Vartun, Marika (2026). To elever med samme karaktergjennomsnitt er ikke nødvendigvis like faglig sterke. Forskning.no. ISSN 1891-635X. doi: https:/www.forskning.no/elever-partner-pedagogiske-fag/to-elever-med-samme-karaktergjennomsnitt-er-ikke-nodvendigvis-like-faglig-sterke/2675805. Fulltekst i vitenarkiv Vis sammendrag

  Ny forskning viser at noen valgfag er mye vanskeligere å få gode karakterer i enn andre.

• Vartun, Marika & Heie, Magnus (2026). Slik kan skolen løfte elevene som starter med færrest muligheter. Forskning.no. ISSN 1891-635X. doi: https:/www.forskning.no/barn-og-ungdom-partner-skole-og-utdanning/slik-kan-skolen-lofte-elevene-som-starter-med-faerrest-muligheter/2633496. Fulltekst i vitenarkiv Vis sammendrag

  Det handler om hvordan skolen bruker pengene.

• Vartun, Marika (2026). Nei, mannlige lærere er ikke bedre på digitale ferdigheter enn kvinnelige lærere. Forskning.no. ISSN 1891-635X. doi: https:/www.forskning.no/digitalisering-laerere-partner/nei-mannlige-laerere-er-ikke-bedre-pa-digitale-ferdigheter-enn-kvinnelige-laerere/2629097. Fulltekst i vitenarkiv Vis sammendrag

  Forskere finner ingen forskjeller i faktisk kompetanse.

• Vartun, Marika (2026). Ungdom med alvorlige adferdsproblemer: Hva skjedde med behandlingstilbudet under pandemien? Forskning.no. ISSN 1891-635X. doi: https:/www.forskning.no/barn-og-ungdom-helsetjenester-pandemi/ungdom-med-alvorlige-adferdsproblemer-hva-skjedde-med-behandlingstilbudet-under-pandemien/2615645. Fulltekst i vitenarkiv Vis sammendrag

  Forskere har sett på hva som skjedde da Norge stengte ned.

• Chen, Jianan; Brinchmann, Ellen Irén & Braeken, Johan (2025). Setting Stopping Rules for Progressive Tests: A Practical and Transparent Toolkit. Fulltekst i vitenarkiv
• Ystrom, Eivind (2025). Fagprat på fredag med Eivind Ystrøm: Sammenhengen mellom genetikk og skoleprestasjoner - Senter for spesialpedagogisk forskning og inkludering (SpedAims). Fulltekst i vitenarkiv Vis sammendrag

  De fleste kjenner til at utfordringer som dysleksi, matematikkvansker og ADHD ofte har en arvelig komponent. Men det er mindre intuitivt at også hvordan vi presterer i fag som norsk og matematikk generelt kan være påvirket av genetiske faktorer gjennom arvbare trekk – ikke bare av sosial bakgrunn. I denne fagpraten presenterer Eivind Ystrøm, professor i personlighetspsykologi og leder for genetikkgruppen ved PROMENTA Research Centre (GeoGen), ny kunnskap om hvordan arv og miljø virker sammen i barns utvikling. Ystrøm forsker på hvordan genetiske og miljømessige faktorer former tidlig sosioøkonomisk utvikling og psykisk helse, blant annet gjennom slektskapsstudier og molekylærgenetiske analyser. Forskningen hans omfatter også «genetic nurture» – hvordan foreldres arvelige egenskaper påvirker barns utvikling, selv når barna ikke arver de samme genene – og hvordan genetiske effekter varierer på tvers av skoler og nærmiljøer.

• Ystrom, Eivind (2025). Writing and Publishing: Reflections from JCPP Advances Editors (Eivind Ystrøm). [Internet]. The Association for Child and Adolescent Mental Health. doi: 10.13056/acamh.13851. Fulltekst i vitenarkiv Vis sammendrag

  In this video from the Writing and Publishing: Reflections from JCPP Advances Editors series, Professor Eivind Ystrøm discusses how unstructured thinking time can lead to creative scientific insights. He reflects on using dialectic reasoning and narrative techniques drawn from film to engage readers and shape research arguments. Drawing on personal habits and storytelling, he shares how self-imposed constraints can support productivity and encourages early-career researchers to view the review process as a constructive dialogue. Learning Objectives 1. Recognise how unstructured time and everyday routines can foster research creativity. 2. Understand how dialectic reasoning and narrative techniques can be used to engage readers. 3. Reframe the peer review process as a dialogue with editors and reviewers.

• Braeken, Johan; Lehre, Anne- Catherine W G & Marcq, Kseniia (2025). The risks and perils of cherry picking results from international large-scale educational assessments: The case of the «Nordic» fractions. Fulltekst i vitenarkiv
• Ofstad, Sverre Berg; Tan, Tony & Andersson, Björn (2025). Comparing Elective Subject Difficulty Using a Joint Item Response Theory Model. Fulltekst i vitenarkiv
• Campos, Diego Gonzalez (2025). Research Synthesis after the Reproducibility and Open Science Reforms: How do Systematic Reviews and Meta-Analysis Practices in Education Need to Adapt? Fulltekst i vitenarkiv Vis sammendrag

  Reproducibility and open science reforms have become powerful drivers of transparency and rigour in contemporary research, yet systematic reviews and meta-analyses face unique challenges when applied in the social sciences. In this presentation, I will explore the impact of these reforms on research synthesis, identify current barriers to conducting systematic reviews and meta-analyses in an open and reproducible way, and highlight the need for adaptive practices in response to an evolving research landscape. Key findings and takeaways will be highlighted to emphasise the importance of continuing to improve systematic review and meta-analysis methods in the era of reproducibility and open science reforms.

• Campos, Diego Gonzalez (2025). Doing Meta-Analysis: A Meta-Analysis Workshop for the Social Sciences. Fulltekst i vitenarkiv Vis sammendrag

  Meta-analyses are a set of statistical techniques used to combine the results of multiple studies on a specific topic. This workshop will illustrate the steps for synthesising research findings using meta-analytic methods. Specifically, the instructor will focus on identifying the structure of meta-analytic data sets, establishing meta-analytic baseline models, and conducting moderator analyses. Participants will learn about random- and mixed-effects models, heterogeneity, and multilevel meta-analysis and get to know ways of including international large-scale assessment data in the modeling phase of a meta-analysis. The workshop materials will utilise the R package “metafor” and some supplementary R packages. Researchers, statisticians, and students can all benefit from this workshop, which will provide the knowledge and tools to model meta-analytic data.

• Campos, Diego Gonzalez (2025). A Hands-On Introduction to Analyses of Educational Equity Using International Large-Scale Assessment (ILSA) Data. Fulltekst i vitenarkiv Vis sammendrag

  This workshop introduces participants to the use of International Large-Scale Assessments (ILSA) such as PIRLS, TIMSS, and PISA for investigating questions of educational equity. Participants will gain insights into the structure and scope of ILSA data, as well as the opportunities and methodological challenges they present. Aims: • Develop an understanding of the structure, scope, and potential of ILSA datasets for educational equity research • Become familiar with the key design features and measurement frameworks underlying ILSA studies • Gain practical, hands-on experience in accessing, preparing, and analyzing ILSA data with statistical software • Apply appropriate methodological approaches to investigate equity-related questions

• Andersson, Björn (2025). Trend estimation in the Norwegian National Tests. Fulltekst i vitenarkiv Vis sammendrag

  In Norway, mandatory national tests are used to monitor performance over time and to identify student weaknesses and strengths. The testing program came under scrutiny in 2023, when it was uncovered that the results had been incorrectly reported for many years. We give an overview of the Norwegian National Tests in numeracy, reading, and English and discuss some particular features of the tests and linking design. We outline the source of the incorrect results and explain how the problem came to be. In a robust re-analysis of the original data, we present revised estimates of the performance trends across 2014-2021. The results show that performance in all domains varied during 2014-2021 and that the results differ markedly compared to the originally reported results, especially in the English domain. We assess the potential impact of model choice and measurement invariance on the trend estimation, demonstrating that these factors can not explain the differences to the original results. We present recommendations for best practices in trend estimation in large-scale assessments and discuss how to avoid similar problems occurring in the future.

• Andersson, Björn (2025). Higher-order extended variational approximation to estimate latent variable models. Fulltekst i vitenarkiv Vis sammendrag

  Generalized linear latent variable models is a flexible class of models which supports joint modeling of continuous, count, ordinal, and binary variables with arbitrary structures among latent variables and observed variables. Estimation of GLLVMs is challenging when specifying complex models that include many types of observed variables and many latent variables. Recently, various methods based on the variational approximation have been proposed in the literature to estimate GLLVMs. To improve the estimation accuracy of variational approximation methods, we derive an extended variational approximation estimator for GLLVMs based on a higher-order expansion and evaluate its properties. In the implementation, we analytically compute the higher-order derivatives required and exploit the sparsity of the model structure to realize computational efficiencies. The approach is compared against alternative methods, such as the computationally efficient but less accurate Laplace approximation and the gold standard method of adaptive Gauss-Hermite quadrature. We discuss the trade off between accuracy and computational efficiency with respect to different estimation methods and different model structures.

• Maurer, Michelle Nina; Campos, Diego Gonzalez; Gowenlock, Anna E.; Norbury, Courtenay Frazier & Melby-Lervåg, Monica (2025). The relationship between socioeconomic factors and oral language in children: A systematic review and meta-analysis. Fulltekst i vitenarkiv
• Siddiq, Fazilat & Scherer, Ronny (2025). Learner's Screen Use: Time to Reload the Debate. Nordic Journal of Digital Literacy. s. 35–39. doi: 10.18261/njdl.20.1.6. Fulltekst i vitenarkiv
• Ofstad, Sverre Berg & Andersson, Björn (2026). Karakterulempen ved å velge realfag: I hvilken grad veier realfagspoengene opp for den? Universitetet i Oslo. doi: https:/www.uv.uio.no/cemo/english/about/news-events-and-publications/news/2026/bonus-points-natural-science.html. Fulltekst i vitenarkiv
• Ulitzsch, Esther & Rucinska, Aleksandra Wanda (2026). When Measurement Quality Limits Prediction: Evaluating Extreme Gradient Boosting Under Unaccounted-for Measurement Error. Universitetet i Oslo. Fulltekst i vitenarkiv Vis sammendrag

  Machine-learning models such as XGBoost are increasingly used for prediction in the social, behavioural, and educational sciences, but their performance may depend on measurement quality and benchmark specification. This Monte Carlo simulation examined how predictor reliability, outcome reliability, latent explained variance, and structural composition shaped XGBoost’s out-of-sample performance relative to three ordinary least squares (OLS) benchmarks. Lower reliability reduced predictive performance and compressed model differences. XGBoost showed the largest positive differences relative to main-effects OLS when the structure was interaction-dominant, and measurement quality was high. These differences were reduced or reversed when the relevant interaction was specified in regression. The findings suggest that apparent gains from flexible machine-learning models should be interpreted in relation to measurement quality, recoverable structure, and benchmark specification.

• Andersson, Björn & Atta, George Boateng (2026). Properties of Marginal Maximum Likelihood Estimators in a Multiple-Group Item Response Model. Effects of Many Groups, Missing Data, Test Length and Small Sample Sizes. Universitetet i Oslo. Fulltekst i vitenarkiv Vis sammendrag

  --- Abstract and files will be published: 2028-05-06

• Scherer, Ronny; Emslander, Valentin & Edvardsen, Ludvik (2026). Doing Meta-Meta Right. Developing and Evaluating a Checklist for Assessing the Quality of Second-Order Meta-Analyses. Universitetet i Oslo. Fulltekst i vitenarkiv Vis sammendrag

  Second-order meta-analyses (SOMAs) synthesise evidence across multiple metaanalyses, offering a powerful tool for identifying overarching trends in research. Nevertheless, no standardised tool presently exists for evaluating their methodological quality. This study was aimed at developing a quality assessment tool for SOMAs and generating new knowledge about its psychometric properties. A 25-item binary checklist was developed by adapting existing quality frameworks, including AMSTAR, PRISMA, MARS, and ROBIS, to specifically meet the methodological demands of second-order syntheses. The checklist was applied to a corpus of 79 published SOMAs spanning multiple disciplines, and descriptive analyses revealed that most SOMAs performed reasonably well on basic reporting criteria, but also showed consistent weaknesses in overlap handling, verification practices, and open science. EFA did not reveal a stable latent factor structure, suggesting that the checklist items cannot be treated as indicators of a single underlying construct. Whether SOMA quality is genuinely multidimensional or whether the current item pool lacks sufficient structural coherence to support any stable factor solution remains an open question. Latent class analysis identified two distinct quality profiles, which distinguished between a higher-rigour group comprising 74% of the corpus and a lower-rigour group comprising 26%. Open science practices were shown to be rarely applied across both groups. The SOMA Quality Checklist provides a structured and theoretically grounded tool for evaluating methodological quality in second-order synthesis, filling a gap that existing frameworks do not adequately address. Keywords: Second-order meta-analysis, study quality, latent class analysis, factor analysis, research synthesis, quality assessment

• Scherer, Ronny & PLANCE, RAPHAEL AN-NUNANG (2026). Examining the Association Between School Climate and Early Grade Reading Proficiency: A Multilevel Analysis of South African PIRLS Data Before and After COVID-19. Universitetet i Oslo. Fulltekst i vitenarkiv Vis sammendrag

  Reading literacy is a foundational skill with profound implications for academic success, economic productivity, and societal development. South Africa has consistently recorded among the lowest reading scores in international assessments, a challenge further compounded by the COVID-19 pandemic. This study examines whether school climate variables—academic climate, safety (bullying), parental involvement, and institutional resource shortage—are associated with Grade 4 reading achievement in South Africa, and whether these associations changed between the PIRLS 2016 and 2021 cycles. Drawing on the Wang and Degol (2016) multidimensional school climate framework and employing multilevel structural equation modelling (MSEM) with South African PIRLS data (2016: n = 3,462; 2021: n = 9,564), the study finds a dramatic decline in mean reading achievement of approximately 133 points across cycles. At the between-school level, academic climate showed a significant positive association with reading achievement in 2016 that disappeared by 2021, while resource shortage emerged as a newly significant negative predictor in 2021. Significant contextual effects were found for safety and academic climate in both years, and for parental involvement in 2021. Formal interaction tests confirmed significant changes in the associations for safety and academic climate across cycles. These findings suggest that the pandemic may have disrupted the mechanisms through which school-level academic culture supports reading outcomes, and that resource inequality became a more consequential barrier to learning in 2021. Implications for educational policy and school recovery interventions in South Africa are discussed.

• Ulitzsch, Esther & Matejko, Ylva Zoe (2026). Treating Item Omissions with a Modified Laplace Smoothing Approach. Consequences for Country-Level Conclusions in International Large-Scale Assessments. Universitetet i Oslo. Fulltekst i vitenarkiv Vis sammendrag

  International large-scale assessments (ILSA) data typically contain unnegligible portions of unplanned missing data, such as item omissions. The mechanisms behind item omissions are not known in practice, yet the accuracy of conclusions drawn from ILSAs hinges on their treatment. As a result, the choice of item omission treatment rests on unverifiable assumptions, with potential consequences for the accuracy of ILSA results. The spectrum of available techniques for handling item omissions in ILSAs is broad, and only recently has Modified Laplace Smoothing been proposed and implemented for handling item omissions in TIMSS 2023. However, no research has yet addressed its impact on ILSA conclusions. The present study therefore investigated to what extent this new approach affects the conclusions drawn from ILSAs, compared to classical approaches. To this end, a TIMSS 2023 subset encompassing achievement data of 43 countries was reanalysed under three treatments: (1) Treating omissions as not administered; (2) Scoring them as incorrect; and (3) Modified Laplace Smoothing. On average, countries shifted by less than one rank position in the achievement ranking and approximately one rank in the variance ranking, with maximum differences of three and four rank positions respectively. The effects were not uniform, suggesting that certain countries are particularly vulnerable to changes in item omission treatment. Given that such rank differences can trigger undue reactions from media and policymakers, item omission treatments, and changes thereto, should be carefully considered from a validity and fairness perspective, and communicated clearly.

• Olsen, Rolf Vegar; Løkken, Jacopo Francesco Taraldsrud & Kalchenko, Olena (2026). Decomposing the Decline in Norway’s Mathematics Performance: PISA 2012 vs. 2022. Universitetet i Oslo. Fulltekst i vitenarkiv Vis sammendrag

  This study investigates the decline in mathematics performance in Norway between 2012 and 2022 using data from the Programme for International Student Assessment (PISA). While recent discussions often attribute performance declines to the COVID-19 pandemic, this study instead examines underlying structural and compositional factors related to student characteristics and learning environments. The primary objective is to determine the extent to which the observed decline can be explained by changes in students’ socio-demographic background and mathematics-related dispositions, as opposed to changes in the relationships between these factors and achievement. Finland is included as a comparative case to assess whether similar patterns emerge within a comparable Nordic education system. The analysis applies a three-fold Oaxaca–Blinder decomposition to partition the performance gap into endowment, coefficient, and interaction components. The model includes socio-economic status, language background, mathematics self-efficacy, mathematics anxiety, and perceived teacher support. The analysis accounts for the complex PISA design through the use of plausible values, balanced repeated replication, and multiple imputation. The findings indicate that the decline in Norway is primarily associated with compositional changes in student characteristics, particularly shifts in affective factors such as increased mathematics anxiety and reduced self-efficacy. In contrast, Finland exhibits a relatively stronger contribution from structural changes in the relationships between predictors and achievement. These findings contribute to understanding recent performance trends by highlighting the role of socioemotional factors in the observed decline. Keywords: PISA; mathematics achievement; Oaxaca–Blinder Decomposition; multiple imputation; socio-economic status; mathematics anxiety; self-efficacy; Norway; Finland

• Braeken, Johan; Marcq, Kseniia & Straten, Luisa thor (2026). Gender Differences in Item Nonresponse in the TIMSS 2023 Student Questionnaire. Universitetet i Oslo. Fulltekst i vitenarkiv Vis sammendrag

  --- Abstract and files will be published: 2027-05-06

• Sher, Noorulhuda (2025). Evaluating Measurement Invariance in the Norwegian Pupil Survey: A Cross-Grade Comparison between Grade 7 and Grade 10. Universitetet i Oslo. Fulltekst i vitenarkiv Vis sammendrag

  This study investigates the measurement invariance and psychometric validity of selected constructs from the Norwegian Pupil Survey across two key educational stages: Grade 7 and Grade 10. Using Confirmatory Factor Analysis (CFA) and Multi-Group CFA, the research evaluates whether students across these grade levels interpret core survey constructs in a comparable manner. Results indicate that while configural and metric invariance are generally achieved, scalar invariance is not consistently supported, limiting the comparability of mean scores across grades. The findings also reveal weaknesses in theoretical grounding and inconsistent psychometric validation within UDIR’s existing framework. The study contributes to a better understanding of how to improve the structural validity and interpretability of large-scale educational assessments in Norway. It offers recommendations for strengthening the measurement foundations of national school evaluations and underscores the critical role of validity, reliability, and measurement equivalence in policy-relevant educational research.

• Goian, Ancuta-Maria (2025). Solution Patterns is a Problem-Solving Task: Consistency Across Educational Systems. Universitetet i Oslo. Fulltekst i vitenarkiv Vis sammendrag

  Leveraging log files from a PISA 2012 Creative Problem-Solving task, this study applies a network-based analytical approach to examine the extent to which problem-solving patterns are consistent across distinct educational systems. Students’ interactions were modeled as weighted directed networks to preserve the sequential and temporal structure of task execution. From these networks, seven theoretically grounded features were extracted to reflect key cognitive processes related to problem-solving. Gaussian Mixture Models were then applied to these features to identify latent behavioral clusters separately for students in Colombia, Norway, and Singapore, using the United States as a reference case. Several clusters aligned with previously identified solution patterns, namely: trial-and-error, deliberate, adaptable, less-able, and low-effort strategies, suggesting that certain behaviors are cross-nationally robust. At the same time, substantial variation in model structures across countries and performance groups also emerged, pointing to context-specific patterns potentially shaped by cultural and educational influences. These findings underscore the potential of process data and network analysis to uncover latent behavioral strategies and support more tailored approaches to assessment and instructional design.

• Løkken, Jacopo Francesco Taraldsrud (2025). Handling missing data in Random Forests. Universitetet i Oslo. Fulltekst i vitenarkiv Vis sammendrag

  Missing data are a common issue in psychological and educational research. Concurrently, machine learning methods like Random Forests have gained popularity for their predictive performance and ease of implementation with high dimensional datasets. This study evaluated how listwise deletion, single imputation and proximity-based imputation affect Random Forest predictive performance. Using a Monte Carlo simulation with 1,000 replications, missingness was induced completely at random (MCAR) at 20%, 50% and 80% rates in the strongest predictor (sex) of the TITANIC3 public dataset (N = 1,045). The performance of each missing data handling technique was assessed within two extreme cases: a baseline model with 0% missingness, and a lower-bound model, with 100% missingness in the sex variable. Evaluation metrics were based on the out-of-bag (OOB) error rates, aggregated across replications for each combination of missingness rate and missing data handling approach. Listwise deletion yielded error rates (50 77%) substantially and systematically exceeding the lower-bound model (29%). Single and proximity-based imputation showed nearly identical error rates (24-28%) consistently below the lower-bound model, and above the baseline model (19%). OOB error rates were also examined for specific subgroups, namely for males and females, and for complete and missing cases. Variable importance was measured for the baseline and lower-bound models, and its stability assessed through the variance of rankings across replications. Using listwise deletion results in all predictors losing predictive power as the missing data proportion increases, although the importance hierarchy was nearly constant relatively to the baseline model. For both imputation methods, as missingness increased, sex lost importance in favour of other predictors, and the importance hierarchy was less stable across replications.

• Meng, Xianqiao (2025). Longitudinal Associations Between Teachers’ Integration of Technology in Classrooms, Self efficacy, and Perceived Usefulness: A Cross-Lagged Panel Approach. Universitetet i Oslo. Fulltekst i vitenarkiv
• Georgieva, Milena Andonova (2025). Probing Robustness of Machine Learning Models for Academic Resilience in a Bulgarian Context. Universitetet i Oslo. Fulltekst i vitenarkiv Vis sammendrag

  Resilient students achieve academic success despite adversity in their lives, however conceptualizing academic resilience presents theoretical and methodological challenges. The current study explores how different definitions of academic resilience impact on the predictability of this construct in mathematics among Bulgarian 4th grade students (N = 4,766), using data from the TIMSS 2019 assessment. Applying XGBoost, a machine learning extreme gradient boosting algorithm, model performance and feature importance rankings were compared across eight distinct definitions created by operationalizing adversity differently (e.g., number of books in the home, parental education) and varying the method of constructing the resilience variables (dichotomous vs continuous). Results revealed that while predictive performance was generally modest (low F1 scores and R2 values), sensitivity to the specific definition of academic resilience was evident - influencing resilience rates (1% - 11%), evaluation metrics and rankings of feature importance. These findings highlight the critical impact of methodological choices on research outcomes and emphasize the need for caution when applying and interpreting machine learning approaches in academic resilience research.

• Mannsaaker, Thea Aurora Hillingseter (2025). Unravelling Norway`s Mathematics Decline: A Cross-Country Decomposition of Mathematical Literacy in PISA 2022. Universitetet i Oslo. Fulltekst i vitenarkiv
• Thunem, Ane Fjermestad (2025). Addressing Local Item Dependence in Reading Assessments: A Bifactor Model Evaluation of the Norwegian National Reading Test. Universitetet i Oslo. Fulltekst i vitenarkiv Vis sammendrag

  This study investigated how test scores from the Norwegian National Reading Test might be affected when several questions are linked to the same text passage. We analysed data from over 230,000 students in Grades 5 and 8 who took the test in 2022 and 2023. These tests are usually scored using models that assume that reading proficiency is the only factor influencing the item scores, but that may not hold when multiple questions are linked to the same passage. We compared the current scoring method with a more advanced model that accounts for how questions related to the same text passage might be impacted by aspects unrelated to reading proficiency. The more advanced model fit the data better and provided more accurate information about each question, including how difficult each question was and how well it distinguished between stronger and weaker readers. It also showed that the test may measure more than one skill at a time, and that the current model may slightly underestimate how much uncertainty is involved in students’ scores. Importantly, the new model predicted the actual student responses just as well as the current one. While most students were classified the same way under both models, around 4-5% of students were placed into different mastery levels depending on the model used. These results show that ignoring how questions relate to one another can influence how test results are interpreted and how students are classified. Using more flexible modelling approaches could make large-scale reading tests more accurate, fair, and meaningful.

• Bilberg, Simen (2025). Exploring Gender Specific Patterns in Health Measurement: A Machine Learning Approach to the Predictive Validity of Self-Reported Health. Universitetet i Oslo. Fulltekst i vitenarkiv Vis sammendrag

  Self-reported health measures are widely used, and are generally considered robust, valuable measurements of health. However, to accurately interpret self-report as a valid health measurement, we must know that the questions function similarly for different individuals and groups, in that individuals understand and answer questions in the same way. Previous research indicates that men and women understand their own health differently, and that they therefore report their health differently, regardless of their actual health status. This master thesis explores whether the relationship between self-reported health and assessed health are different for men and women using data from the Norwegian Armed Forces selection process. We use machine learning classification models to predict whether a participant is likely to be assessed as healthy or unhealthy at on-site medical assessment based on their responses to a self-reported pre-selection survey that includes a wide range of questions related to health. By comparing predictions with actual assessed health and investigating how items themselves contribute to prediction, we find gender differences in overall worse prediction accuracy for women, and that some health proxies predict differently for men and women, indicating that there might be gender differences in the predictive validity of self-reported health measures and proxies towards assessed health.

• Sakyar, Rayad (2025). Teachers and Technology: A Longitudinal Investigation of Attitudes, Perceived Usefulness, and Ease of Use. Universitetet i Oslo. Fulltekst i vitenarkiv Vis sammendrag

  Do teachers change their views about technology as they gain more experience with digital tools in the classroom? To find out, we followed 432 teachers in the United Kingdom over nine months as they used technology in their teaching. Three times during the year, we asked how much they liked the technology, how useful they found it for teaching, and how easy it was to use. Teachers’ general attitudes and perceptions of usefulness stayed largely consistent over time, suggesting these views tend to persist. In contrast, perceptions of ease of use shifted more, especially across age groups. Older teachers initially reported more difficulty using technology but showed greater improvement over time, suggesting experience may help close early gaps. Male teachers, on average, began with more positive attitudes and found technology easier to use.

• Dåvang, Emmelin Sahra (2025). Resilient Minds: A Machine Learning Comparative Study of Academic Resilience in Science Among 8th Grade Students in Norway and Malaysia. Universitetet i Oslo. Fulltekst i vitenarkiv
• Marcq, Kseniia (2025). Item in Focus: A Large-Scale Examination of Cross-Country Differences in the PISA 2018 Through an Item-Centric Perspective. Universitetet i Oslo. Fulltekst i vitenarkiv Vis sammendrag

  Research on International Large-Scale Assessments (ILSAs), such as the Programme for International Student Assessment (PISA), has traditionally focused on student performance and the associated school or system-level determinants. Within this student-centric approach, the assessment items —the core building blocks of ILSAs—often receive less attention despite their potential to provide insights when examined more closely. This thesis pivots the focus towards these core elements in ILSAs. Using the PISA 2018 response data from over 70 countries, this thesis aims to explore the “item side” of PISA, demonstrating how examining assessment items in ILSAs can offer countless possibilities for item-level validation, test design improvement, and enhancing our understanding of student performance across diverse educational settings. To situate this item-centric exploration within the broader field of ILSA research, the thesis broadens its scope along three dimensions. First, it encompasses a broad geographic coverage, using response data from every participating country and jurisdiction in the PISA 2018, allowing for cross-country generalisability and identifying patterns across regions. Second, the thesis spans a wide range of PISA items, both from the achievement tests and the student questionnaire, thus permitting a detailed investigation of various item features. Lastly, the thesis hones in on two outcomes: item difficulty and item nonresponse, illustrating how item features can influence various outcomes. The thesis is organised into two applications. Application I explores how item features influence item difficulty within the PISA 2018 achievement tests (Articles I, II, and III). Here, the findings show (i) considerable differences in item difficulties, a vast source of largely untapped potential for ILSA research, and (ii) how these differences in item difficulties relate to construct-relevant and -irrelevant item features to varying extent across different countries. Application II (Articles IV and V) examines how item features such as item position within the student questionnaire and the scale’s format and content relate to item nonresponse patterns in the PISA 2018 student background questionnaire and how these item features interact with student characteristics, such as gender, resulting in systematic patterns of missing data across countries. Both applications have implications for assessment design, validity, and comparability of results across countries. In summary, this thesis advocates for research on the overlooked ‘item side’ of ILSAs, highlighting the importance of items and their features in shaping various ILSA outcomes and demonstrating the potential of item-centric research to inform future test design, enhance fairness, comparability, and validity of ILSA results, and promote a more nuanced understanding of global educational trends.

• Johan, Braeken,; Rolf, Strietholt, & Mario, Olivera Toro Lopez, Manuel (2025). When Measurement Decisions Shape Findings: A Multiverse Analysis of Socioeconomic Status and Science Achievement in TIMSS 2019 Data from Norway and Chile. Universitetet i Oslo. Fulltekst i vitenarkiv Vis sammendrag

  --- Abstract and files will be published: 2026-11-14

• Bredal, Morten (2024). Who Becomes a Teacher or a Doctor? A Mixed Effect Model Approach to Analyzing the Socioeconomic Status, Academic Achievement and Gender of Students who Enter Medical and Teaching Educations in Norway (2013-2019). Universitetet i Oslo. Fulltekst i vitenarkiv Vis sammendrag

  Resolving current shortages in education and healthcare is a high priority for many countries. Depending on how these shortcomings are rectified, they may introduce further disparities within both sectors. This thesis investigates how academic achievement, gender, and socioeconomic status as a product of an individual’s parents' education levels and household income affect who enters a medical program compared to a teaching program in Norway. Using register data provided to the project by the Norwegian government, we can furnish precise background characteristics on a large proportion of students. To this end, we employ mixed effect modeling to determine the probability of a student choosing a medical program compared to a teaching program under various conditions. The sample consisted of 9684 individuals who entered medical or teaching programs at a Norwegian higher education institution in 2013, 2016, and 2019. Our findings suggest that academic achievement, measured as a student’s high school grade point average, is the most influential factor differentiating students by which programs they are accepted into. Socioeconomic status plays an important secondary role, which, at some levels, can distinguish between medical and teaching students. Females are overrepresented in medical programs even when we consider the gender distribution among university students. For students with the formal requirements to enter a medical program but otherwise enter a teaching program, we see that these gender differences almost disappear and slightly favor males.

• Hossain, Sujon (2024). Switching from a linear to a multi-stage to a fully adaptive test administration: A hybrid simulation study for the Norskprøven reading comprehension test. Universitetet i Oslo. Fulltekst i vitenarkiv Vis sammendrag

  There is a growing trend in large-scale assessments to shift from traditional test formats to adaptive testing formats to improve measurement efficiency and precision. This study explores the gain of shifting the Norskprøve reading comprehension test from a non-adaptive linear format to a module-level adaptive multi-stage test (MST) and the feasibility of implementing an item-level adaptive computerized adaptive test (CAT). Using hybrid data generated from real response data (n=6768), we have retroactively simulated different test formats and compared the measurement precision and efficiency of the current MST with other formats. Results show that MST reduces test length by 62.41% on average compared to administering the full item bank. The linear test reduces the MST test length by 31.93% without sacrificing precision. The linear test and full item bank show a higher mean of average item information than MST, indicating that even a linear test can be highly competitive with the current MST. The equal precision CAT reduces MST test length by 62.78% on average, with similar precision. The equal-length CAT shows improved measurement precision for 99.88% of the test takers with the same test length. The fixed precision CAT reduces the MST test length by 84.55% on average with a desired overall high measurement precision. All three formats of CATs provide higher mean average item information than MST, indicating more efficiency. For further research on the feasibility of CAT implementation, an adaptive classification CAT has been suggested with a well-designed item bank, balanced content, and item bundles with testlet designs.

• Krog, Maria Mehammer (2024). Evaluation of Measurement Invariance - A Validation Study of the Norwegian Student Survey. Universitetet i Oslo. Fulltekst i vitenarkiv
• Pasha, Monirul Hoque (2024). Cross-National Variations in Self-Efficacy and Attitudes Towards ICT Use in Teaching: An Integrative Data Analysis Across ICILS Countries. Universitetet i Oslo. Fulltekst i vitenarkiv Vis sammendrag

  Information and communication technology (ICT) use in teaching is a significant concern in educational systems worldwide. Teachers’ psychological factors, such as their self-efficacy and attitudes toward ICT use, have been considered influential in implementing ICT in teaching. However, previous studies have focused on small sample sizes or a limited number of countries and have largely ignored the mediation effect of attitudes on the relationship between self-efficacy and ICT use. This study addresses these limitations by using data from the International Computer and Information Literacy Study (ICILS) to examine the relationship between these factors. Specifically, the study synthesizes evidence on the mediation effect of attitudes on the relation of self-efficacy and ICT use in teaching, controlling for ICT resources at school and teachers' backgrounds. In addition, the study examines the variation in this relationship across varied educational settings. The findings suggest that attitude significantly and partially mediates the relationship between self-efficacy and ICT use in teaching, with variations across different countries. The country’s ICT infrastructure partly explains this cross-country variability in the mediation relationship. This large-scale data research reveals the mediation effect of attitudes mediate their self-efficacy and ICT use in teaching. This relationship varies across ICILS countries and is influenced to some extent by the country’s ICT infrastructure, which informs global educational policies to improve ICT implementation.

• Nery, Emily Rose Cruz (2024). Examiner Cut Score Stringency in the OSCE: A Simulation Study Exploring the Impact of Differential Examiner Stringency in the Objective Structured Clinical Examination under a Borderline Regression Method of Standard Setting. Universitetet i Oslo. Fulltekst i vitenarkiv Vis sammendrag

  The Objective Structured Clinical Examinations (OSCE) are performance-based assessments administered to medical students and physicians globally. Pass/fail decisions after the OSCEs are set by applying a standard setting method. The most widely used is the Borderline Regression Method (BRM). Psychometric post-hoc analysis of OSCE data have shown that examiner variance remains a significant factor in performance evaluations. The “hawk-dove” effect is one example of rater-related error that may affect the OSCE. However, although present at the station level, the overall effect is believed to balance out at the exam level. There is a hypothetical concern regarding the potential impact of attenuation bias on cut-score stringency within a regression-based standard setting approach in the presence of examiner severity. However, this area of study remains under-research. A simulation study is conducted to explore the technical issue of attenuation and its impact on cut-score stringency predicted by BRM. A simulation protocol based on Rasch models is also introduced as an initial framework to mimic OSCE settings. Findings show that although lenient examiners lower the cut scores at the station level, the effect is ameliorated at the exam level. Overall, stringent and nonsystematic examiner tendencies result in a systematically weaker slope of the regression line resulting to cut score stringency in both station and exam level. This amounted to an alteration of the passing rate of between 2-3% (N=750,000). The impact to BRM, a review of OSCE station metrics as well the potential role of simulation in OSCE research are discussed.

• Lööf, Fanny (2024). Science Achievement Gaps Through an Intersectional Lens: An Intersectional Approach to Differential Item Functioning Evaluation for TIMSS 2019 science assessment in Sweden. Universitetet i Oslo. Fulltekst i vitenarkiv
• Zhang, Lu (2024). Benefits of Multi-stage Testing in the Norwegian National Numeracy Tests. Universitetet i Oslo. Fulltekst i vitenarkiv

Sjå alle arbeida i NVA