Ed tech can be good, too
Student engagement is the holy grail
Note: Today’s piece originally appeared at We’reAllSolvers.org.
A recent mini-study looked at what happened after a subset of schools in Newark, New Jersey adopted Khan Academy as a supplemental math tool. Not only did they see strong usage numbers, but students who used it more frequently saw larger gains, and the gains were particularly large for previously low-performing students.
Across three school years, 13 schools in one part of the district offered Khan Academy as a math supplement to roughly 6,700 students. Of those students, 87% of the students qualified for free- or reduced-price lunch, a proxy measure for income. In the last year of the three-year period, 70% of students met the recommended usage of at least 30 hours minutes per week.
That alone is kind of a big deal. There’s plenty of research suggesting that high-quality apps like Khan Academy are helpful, but only for the kids who actually use them regularly. Laurence Holt dubbed this the “5 percent problem” because so few kids actually engage with the tools enough to see results. So the fact that Newark was able to get 70 percent engagement is quite good.
On average, the Khan Academy supplement helped produce small but noticeably positive gains. The returns were also linear, in the sense that kids who spent more time on the app and completed more lessons learned more. This may sound obvious, but it’s a good reminder that engagement really is the holy grail of learning.
But now look at the right side the chart above. When the authors sorted the effects of Khan Academy implementation across performance levels the largest effects, by far, were for the lowest-performing students (NJSLA level 1).
This is an important finding given that achievement scores across the country have fallen so dramatically for the lowest-performing students. In other words, schools and districts have tools available to address one of their biggest challenges. But students have to actually use those tools to see any benefits.
Reading List
Oreopolous et al: In-school supervised EdTech support produces massive learning gains (in India)
K12 Dive: 12 years later, San Francisco USD to offer Algebra I in 8th grade again
CALDER: The Divergent Paths of High- and Low-Achieving Students in Algebra I After the Pandemic
Ah! Okay — I like that study design. It’s a real test of instructional quality and dosage. Thanks!
I think a subtlety worth noting here is the difference between technology as a supplement to instruction, and technology used during classroom instruction. This study (if I'm reading it correctly) works to get students doing additional math practice on Khan Academy beyond the typical math class. That seems trivially good: practice is important, we're using digital tools to get students more math practice in a way that is sustainable without a teacher present.
That's different from a lot of the pushback I'm seeing right now, which focuses more on using Chromebooks during classroom instruction. When screens come out during class it introduces all sorts of distractions, can influence teachers to be more passive, and can reduce effort. Lots of popular math curricula right now don't have much practice, so technology as a practice supplement seems like a good use case.