QUINT Observation System Seminar: February

Image may contain: Photograph, Shirt, Videographer, Green, Tripod.

Illustration, camera and teaching situation

Welcome to the QUINT Observation Systems Seminars (OBS seminars).  This series will discuss classroom observation systems as a tool for understanding and improving teaching quality.

This and upcoming OBS seminars are open for all interested parties.  We want it to become a meeting arena for scholars genuinely interested in observation systems and related issues. Therefore we recommend that you join our network by subscribing to the network mailing list. You must confirm your email address in the confirmation email you receive to complete signing up to the mailing list. 

If you are interested in presenting your research or have questions, please contact the organizer, QUINT Postdoctoral Fellow Mark White.

Program

In this seminar, taking place on February 13th at 9am ET/ 15.00 CET, Mark White will present his preprint titled: What’s in a Score? Augmented Decompositions of Scores from Observation Systems

Abstract: Efforts to directly study instruction are important for understanding the impact of educational interventions and policies. Observation systems represent a formalized tool to directly measure instructional quality and support such understanding. Concerns, however, have been raised about the quality of scores in observation systems. This paper explores the quality of scores from observation systems. Combining generalizability theory with approaches from hierarchical linear modelling and harmonizing analyses across operational and calibration data, I show the both systematic and random rater errors are by and far the largest contributor to observation scores. I also point to numerous other challenges that, together, suggest past efforts to evaluate the quality of observation scores were overly optimistic. Implications for using observation systems are discussed.. 

 

Published Sep. 6, 2022 11:06 AM - Last modified Feb. 3, 2023 1:34 PM