QUINT Observation System Seminar: April 13th, 2021
This is the second webinar in the QUINT Observation Systems Seminars (OBS seminars). This series will discuss classroom observation systems as a tool for understanding and improving teaching quality.
This and upcoming OBS seminars are open for all interested parties. We want it to become a meeting arena for scholars genuinely interested in observation systems and related issues. Therefore we recommend that you join our network by subcribing to the network mailing list using your institutional email address. For security reasons, we cannot accept subcriptions submitted from free email accounts. Please contact the organisers if you have questions. You must confirm your email address in the confirmation email you receive to complete signing up to the mailing list.
If you are interested in presenting your research or have questions, please contact the organizer, QUINT Postdoctoral Fellow Mark White.
In this seminar, taking place April 13th at 16.00 CET/10am ET, Sean Kelly will be presenting work examining global teacher observation protocols, entitled:
From global teacher observation protocols to fine-grained, automated methods: Strengths and emerging limitations
Abstract: An essential feature of many modern teacher observation protocols is their “global” approach to measuring instruction, where trained observers provide a summary evaluation of multiple domains of instruction scored over an interval of time or even entire class session. In this talk, I highlight several limitations of global protocols that have emerged, referencing findings from the Measures of Effective Teaching data reported by Kelly et al. (2020; Educational Policy Analysis Archives). These limitations include: limited discriminatory power to categorize instruction, reliability that is highly contingent on rater training, highly correlated subdomains that make it difficult to distinguish specific practices, the confounding of classroom context with teachers’ own contribution to instruction, and evaluation solely on a continuum of effectiveness rather than measures focused on tradeoffs in time-use or emphasis. As a frame of reference for understanding global protocols, I also consider the properties of automated systems of observation currently being developed, which address some of these limitations but also introduce new challenges.
Bio: Dr. Sean Kelly is a Professor in the Department of Educational Foundations, Organizations, & Policy at the School of Education at the University of Pittsburgh. He is currently serving as Co-Editor for the American Educational Research Journal.