Introducing QUINT postdoctoral research fellow Mark White

There are many assumptions about classroom instruction. Mark White wants to develop more complex and realistic models for understanding how instruction affects student learning. 

Mark White sitting outside in Niels Henril Abels hus.

Mark's project will contribute to understanding whether there is a Nordic view of instruction that is shared. Photo: Larissa Lily, QUINT/UiO

Mark received his PhD from the University of Michigan in 2017. He first got into education research through working in the mental health field. Working with kids and adults with behavioral issues led him to think about ways of preventing some of these challenges. This is turn led him to look at the field of social-emotional learning in the US.

Social-emotional learning refers to ways that schools are trying to help students explicitly develop their ability to self-regulate with the idea that this will be preventative of behavioral issues.The main goal of these programmes is to change instruction and to promote different teaching styles that are more conducive to students learning to self-regulate and practice social skills.

What were some of the questions that got you interested in school research?

It was looking into social-emotional learning programmes to see if they are effective or not. This got me into questioning: what is instruction, how do we measure it, and how do we think about it? Especially in the context of interventions that are meant to change instructional practice.

I got into research in order to understand what it is we are actually measuring, and are we measuring what we think we are measuring.

I wanted to see how QUINT uses these observation instruments to understand interventions meant to help teachers develop their capacity to give better instruction.

How can we measure instruction?

There is no great measure of instruction currently. The observation instruments that are widely used are good for research purposes but they are problematic in many ways when you try to use them in practice. There has not been much research on what are they measuring, or in fact, how well they are measuring it.

What are some of the problems with these observation instruments?

The claim is that you are objectively measuring instructional quality, but the evidence suggests that a large proportion of what you are measuring is raters disagreeing with each other. Scores may just reflect rater’s idiosyncratic views of instruction rather than the formal definition of what is instructional quality.

You are also measuring all sorts of different things. For example, reading and writing get very different scores. That may be caused by instruments being biased towards specific types of instruction. This leads to different scores. Until we really understand what is leading to low and high scores, and how instruments may be biased by the requirements of teaching different types of content, it is difficult to know how well they are going to function as useful instruments to improve teaching.

A lot of observation instruments, PLATO included, are dividing instruction to very broad categories, like intellectual challenge or classroom discussion. Classroom discussion is a very broad thing; you are taking potentially dozens of specific interactions between teachers and students, or between the students, and then internally aggregating them all as a rater. There is a lot that is implicit in that practice, where we don’t know why or how raters are getting the scores.

Tell us more about your project at QUINT:

The Protocol for Language Arts Teaching Observations (PLATO) is a classroom observation protocol designed to capture features of English/Language Arts (ELA) instruction. It was developed to study the relationship between teachers' classroom practices and their impact on student learning. 

For my project, I proposed that we try to code individual discreet behavior and interactions. We take each of those interactions and try to recreate PLATO scores, based on the most low-inference observable actions we can detect. The idea is to get a better sense of what raters are doing: are they noticing the same behaviors; are they interpreting the same behaviors in the same way; are they placing behaviors in the same categories? That gives you a better sense of what these scores are actually composed of.

Another basic assumption of these instruments is that you can go into any lesson at any time and measure instructional quality, and the measure will function equally well. That contradicts a lot of theories of instruction that view teachers as professionals that are making specific choices to do specific things, in order to reach specific goals. With this very discreet, low order coding we can test that theory and see whether in a lesson segment where the goal is to introduce information, different behaviours are happening than in a lesson segment where the goal is to review information.

The aim is to see whether the differences in behaviours based on the teacher’s goals are actually leading to different implications for what the score actually means. Trying to incorporate this idea that teachers are intentionally making choices into the measuring process rather than assuming that everything is equally important at all times.

A group of researchers sitting around a table during PLATO training
Mark leading a workshop in scoring videos before the PLATO training. Photo: Larissa Lily, QUINT/UiO

What are you working on now:

We are training a group of people on PLATO. We’ve had them score videos before the training, and we are going to score the same videos after the training. We are trying to understand how raters think differently about the scores, and how they justify their scores in different ways after the training. The aim is to see whether PLATO actually changes the way people think about instructional quality.

We are also going to have raters indicate some degree of uncertainty in the scoring. We want to see if that additional uncertainty gives us more information about what is a good or bad classroom. The questions is: can we make PLATO more informative by incorporating a measure of uncertainty the raters feel in the scoring.

There are many assumptions about the nature of instruction and how instruction leads to student learning.  One of them is that all aspects of instruction are equally important, and that all aspects are equally important at all times. This is a very simplistic way of thinking about how instruction affects student learning. We are building more complex models based on the PLATO scores to try to predict student learning, and linking each of those models to different theories about how instruction might affect learning. We want to see if we can find empirical evidence that will support more complex and realistic models of how instruction affects learning.

Is there a particular relevance to your project that QUINT is a Nordic collaboration?

PLATO was developed in an American context, and because it is an American tool, it may have implicit bias that favours American style instruction.

One of the things we are trying to understand is whether there is a Nordic view of instruction that is shared. Once we do the same training with the Finland group, we can compare Norwegian and Finnish and see if there is consistency across the two. The pre-training scoring hopefully might reveal a Nordic view of what is good instruction.

And finally, what do you do to relax:

Swing dancing. I like being active and moving. It is a little more expressive so it utilises different parts of the brain. And I really like the music.

I also like gardening because it creates a more physical, tangible result. You get food out it.

By Larissa Lily, QUINT/UiO
Published Oct. 31, 2019 3:22 PM - Last modified Nov. 7, 2019 4:52 PM