Computer-Assisted Learning in the Real World: How Khan Academy Influences Student Math Learning
Taryn Eamesa, Emma Brunskillb, Bogdan Yamkovenkoc, Kodi Weatherholtzc, and Philip Oreopoulos
"Computer-Assisted Learning (CAL) offers an affordable way to implement a mastery learning approach in the classroom. However, while experimental research suggests CAL can enhance student outcomes, such findings often rely on experimental conditions not easily replicated in ordinary classroom settings (e.g., opt-in participation, extensive training and support, and high CAL usage targets). To assess the real-world impact of CAL, we draw on a large three-year panel of administrative data covering over 200,000 students in school districts that licensed Khan Academy’s MAP accelerator, a program designed to support math learning. To identify causal effects, we exploit within-teacher and within-school changes in average classroom CAL practice time—a strategy that yields precise, policy-relevant estimates even at modest usage levels. We find that a classroom with 6.6 hours of annual Khan Academy practice (about 11 minutes per week) experiences a +0.031 SD gain in math test score performance compared to no practice. For classrooms with higher usage levels, we find approximately linear gains, with projected effects rising to +0.085 SD at the recommended 30 minutes per week. Higher-achieving students benefit most, in part because they spend more time on CAL and progress through more skills than lower-performing peers. Teachers might reduce achievement gaps and boost overall gains by encouraging more productive use of the platform (focused on skill mastery)—especially among struggling students."