-
Scherer, Ronny & Nilsen, Trude
(2023).
Accountability for the future? International testing and future directions.
In Tierney, Robert J.; Rizvi, Fazal & Erkican, Kadriye (Ed.),
International Encyclopedia of Education (4th Edition).
Elsevier Science.
ISSN 978-0-12-818629-9.
p. 324–340.
doi:
10.1016/B978-0-12-818630-5.09060-6.
-
Gochyeev, Perman; Siddiq, Fazilat; Scherer, Ronny & WIlson, Mark
(2022).
Exploring collaborative problem-solving tasks for individual-level and team-level inferences using multilevel modeling.
Show summary
Collaborative problem solving (ColPS) is necessary to succeed in the 21st century and has therefore been introduced into compulsory K-12 education in many Western countries. ColPS is the “capacity of an individual to: recognize the perspective of other persons in a group; participate as a member of the group by contributing their knowledge, experience, and expertise constructively; recognize the need for contributions and how to manage them; identify structure and procedure involved in resolving a problem; and, as a member of the collaborative group, build and develop knowledge and understanding” (Griffin et al., 2012, p.7).
Several aspects need consideration when designing ColPS tasks, including group dependence, team size, and activities to promote collaboration and problem solving (Siddiq & Scherer, 2017). Since students are often randomly assigned to teams, observations within groups are not independent. Hence, psychometric models need to account for the hierarchical structure of the data.
Wilson et al. (2017) presented an approach for analyzing ColPS tasks using multilevel models, which are widely used in educational research. The authors focused on modeling the group effects on the overall test performances. We propose focusing on group effects on individual task performance instead. We aim to classify ColPS tasks into ones with and without ignorable group effects, implying lower and higher collaboration. We also propose calibrating these tasks using a two-dimensional Rasch model to further understand the two constructs these items represent. We analyse data collected from the Norwegian Digital literacy (LDN-ICT) instrument (Siddiq et al., 2017). In total, 39 tasks were administered to 144 ninth-grade students who worked in groups of three or four.
Prior to modelling, it will be helpful to use descriptive statistics in an exploratory way to establish a framing context for modelling. Preliminary findings suggest that tasks showing significant group effects can be identified based on the estimated intraclass correlation (ICC)—a commonly used measure of group dependence. ICCs closer to zero indicate ignorable group effects, while higher ICCs imply substantial group effects. The ICCs ranged between 0.00 and 0.88, with an average ICC of 0.31. We also found a significant agreement between estimated group dependence and intended group dependence. Moreover, tasks requiring reflection, evaluation, and accessing information exhibited higher ICCs than others.
This study contributes to the design and analysis of ColPS assessments and illustrates tools to assess if tasks function as they were intended.
-
Mittal, Oleksandra; Scherer, Ronny & Nilsen, Trude
(2022).
Assessing the Evidence for Comparability of Socioeconomic Status across Students with and without Immigrant Background in Norway and Sweden [Paper presentation]. Nordic Educational Research Association (NERA) Conference, Reykjavik, Iceland.
-
Blömeke, Sigrid; Howie, Sarah J.; Kaplan, David & Scherer, Ronny
(2022).
Introduction to the Comparative Large-Scale Studies in Education: Structure and Overview of the Handbook.
In Nilsen, Trude; Stancel-Piątak, Agnes & Gustafsson, Jan-Eric (Ed.),
International Handbook of Comparative Large-Scale Studies in Education.
Springer.
ISSN 978-3-030-38298-8.
doi:
10.1007/978-3-030-38298-8_1-1.
-
-
Mittal, Oleksandra; Scherer, Ronny & Nilsen, Trude
(2021).
Measuring Socioeconomic Status across Ethnic Majority and Minorities in PISA 2018: Testing Household Possessions Index for Differential Item Functioning [Paper presentation]. European Conference on Educational Research (ECER), Geneva (online). https://eera-ecer.de/ecer-programmes/conference/26/contribution/51551/.
-
-
Siddiq, Fazilat; Scherer, Ronny & Tondeur, Jo
(2020).
Developing Students’ Collaborative Problem-Solving Competence (TEACH21st): Towards a linking science.
-
Hjetland, Hanne Næss; Brinchmann, Ellen Iren; Scherer, Ronny & Melby-Lervåg, Monica
(2018).
Preschool predictors of later reading comprehension ability: a Campbell systematic review.
-
Scherer, Ronny; Siddiq, Fazilat & Sánchez Viveros, Bárbara
(2018).
Technology and the Mind: Does Learning to Code Improve
Cognitive Skills?
Proceedings / ACM Multimedia Conference.
doi:
10.1145/3183654.3183658.
Show summary
Modern technology permeates almost every area in society and thus necessitates the development of new skillsets--including computer coding. Many leaders around the world consequently encourage students to learn to code, as this may help them to develop other skills, including problem solving, reasoning, and numeracy. The current study, based on a meta-analysis, puts to test this largely accepted claim that learning how to code improves other cognitive skills. We identified 105 relevant studies that examined the transfer of coding skills to other cognitive skills, such as reasoning, problem solving, creativity, literacy, and numeracy. These studies provided more than 150 effect sizes and followed experimental and quasi-experimental designs across all educational levels. A three-level random-effects model was specified to the resultant dataset to summarize the transfer effects. Hedges' g formed the basis for this modelling approach. The aggregated size of the transfer effects--independent of the context of transfer--was positive and mediocre (g = 0.49, 95% CI = [0.37, 0.61]), and showed considerable variation within and between studies. At the same time, this effect differed across transfer contexts--for skills closer to computer coding, effects were larger (near transfer) than for skills that overlap only marginally with coding (far transfer). Moreover, the purpose of coding moderated the overall effects. Our response to the question whether learning to code improves other cognitive skills is positive: Indeed, positive transfer effects can be detected. Nevertheless, learning to code has differential rather than uniform effects on other cognitive skills.
-
Scherer, Ronny & Nilsen, Trude
(2017).
Relations between School Climate, Instructional Quality, and Educational Outcomes.
-
Siddiq, Fazilat & Scherer, Ronny
(2017).
Is there a gap? A meta-analytic review of the gender and SES effects on students’ ICT literacy.
-
-
Nerstad, Christina G. L.; Glasø, Lars; Dysvik, Anders & Scherer, Ronny
(2016).
The mechanism to protest at work: A multilevel study.
Academy of Management Proceedings.
ISSN 0065-0668.
doi:
10.5465/AMBPP.2016.10864abstract.
-
Nerstad, Christina G. L.; Glasø, Lars; Dysvik, Anders & Scherer, Ronny
(2016).
The mechanism to protest at work: A multilevel study.
-
Siddiq, Fazilat & Scherer, Ronny
(2016).
Investigating student-student interactions in an assessment of collaborative problem solving: An in-depth analysis of think-aloud protocols.
-
Siddiq, Fazilat & Scherer, Ronny
(2016).
Do Age and Gender Matter for the Relation Between
Teachers’ Computer Self-efficacy and their Emphasis on Developing Students’ Digital Skills?
-
Fütterer, Tim; Lachner, Andreas; Scheiter, Katharina; Scherer, Ronny & Stürmer, Kathleen
(2020).
Will, Skill or Conscientiousness: What Predicts Teachers' ICT-Related Professional Development? (Study Protocol).
Leibniz Psychology.
Show summary
This study protocol outlines a study to explore the predictive power of cognitive, motivational and personality traits for teachers' tendency of participation in as well as teachers' willingness for ICT-related PD. For the study we will use data of teachers from the project tabletBW. TabletBW is a school trial for learning with tablets in classrooms which is accompanied by longitudinal scientific research from 2018 to 2021. To test our hypotheses, we will conduct stepwise multivariate multiple regression models (SEM) including teachers' utility value of educational technology, teachers' technological ability as well as teachers' conscientiousness as main predictors for teachers' tendency of participation in as well as teachers' willingness for ICT-related PD.
-
Kluge, Anders; Dolonen, Jan Arild; Scherer, Ronny & Ludvigsen, Sten Runar
(2018).
Testing and Teaching Critical Thinking and Problem Solving in a Changing World. Summative report from the PROBE-project: Critical thinking and problem-solving skills in a changing world. An approach to the design of classroom-based assessment and teaching.
University of Oslo and the Brookings Institution.
-
Scherer, Ronny & Nilsen, Trude
(2017).
Relations between School Climate, Instructional Quality, and Educational Outcomes: Findings from TIMSS 2015 in Norway.
IEA-IRC Conference Proceedings.
-
Kluge, Anders; Boothby, Russel Clark; Dolonen, Jan Arild; Scherer, Ronny & Ludvigsen, Sten Runar
(2017).
Critical Thinking in a Changing World: ASSESSMENT FRAMEWORK.
University of Oslo and the Brookings Institution.
-
Kluge, Anders; Boothby, Russel Clark; Dolonen, Jan Arild; Scherer, Ronny & Ludvigsen, Sten Runar
(2017).
Critical Thinking in a Changing World: A Formative Report of the Conceptualization, Assessment, and Implementation of Critical Thinking Skills in 21st Century Education.
University of Oslo and the Brookings Institution.
