I think a subtlety worth noting here is the difference between technology as a supplement to instruction, and technology used during classroom instruction. This study (if I'm reading it correctly) works to get students doing additional math practice on Khan Academy beyond the typical math class. That seems trivially good: practice is important, we're using digital tools to get students more math practice in a way that is sustainable without a teacher present.
That's different from a lot of the pushback I'm seeing right now, which focuses more on using Chromebooks during classroom instruction. When screens come out during class it introduces all sorts of distractions, can influence teachers to be more passive, and can reduce effort. Lots of popular math curricula right now don't have much practice, so technology as a practice supplement seems like a good use case.
That's exactly right. There's a huge distinction between YouTube and structured math practice. The former is likely an unhelpful distraction, while structured practice is quite helpful.
Chad, you can’t mean that the “recommended usage is at least 30 hours per week” — 6 hours a day on Khan during the school week? I’d also look at Carl Hendrick’s blog — he’s fabulous at summarizing the research. While practice can show pre-test/post-test gains on what kids have just practiced, that knowledge typically doesn’t “stick” past a day or two. Did you look at that? How long after the practice was the test administered; was the post-test a Khan-developed test or some standardized instrument; and did the test tie to what the kid had just learned (‘yes’ is a good answer here, either way!)?
Yes, that was a pretty bad typo on my part! It should say 30 *minutes* per week. They measured whether kids hit that dosage suggestion across the course of the year, and then their outcome variable of interest was the NJ state test.
"In the last year of the three-year period, 70% of students met the recommended usage of at least 30 hours per week." The dosage can't be 30 hours a week. What was the recommended dosage?
Ah! Okay — I like that study design. It’s a real test of instructional quality and dosage. Thanks!
I think a subtlety worth noting here is the difference between technology as a supplement to instruction, and technology used during classroom instruction. This study (if I'm reading it correctly) works to get students doing additional math practice on Khan Academy beyond the typical math class. That seems trivially good: practice is important, we're using digital tools to get students more math practice in a way that is sustainable without a teacher present.
That's different from a lot of the pushback I'm seeing right now, which focuses more on using Chromebooks during classroom instruction. When screens come out during class it introduces all sorts of distractions, can influence teachers to be more passive, and can reduce effort. Lots of popular math curricula right now don't have much practice, so technology as a practice supplement seems like a good use case.
That's exactly right. There's a huge distinction between YouTube and structured math practice. The former is likely an unhelpful distraction, while structured practice is quite helpful.
On that point, see this other study I linked to from India: https://cepr.org/voxeu/columns/school-supervised-edtech-support-produces-massive-learning-gains-khan-academy-field or the studies on the ASSISTMents program: https://nospin.evidencebasedpolicy.org/articles/assistments-low-cost-online-tool-support-7th-grade-math-learning
Sorry — just saw the comment above re the dosage. The rest of my comment is still germane.
Chad, you can’t mean that the “recommended usage is at least 30 hours per week” — 6 hours a day on Khan during the school week? I’d also look at Carl Hendrick’s blog — he’s fabulous at summarizing the research. While practice can show pre-test/post-test gains on what kids have just practiced, that knowledge typically doesn’t “stick” past a day or two. Did you look at that? How long after the practice was the test administered; was the post-test a Khan-developed test or some standardized instrument; and did the test tie to what the kid had just learned (‘yes’ is a good answer here, either way!)?
Yes, that was a pretty bad typo on my part! It should say 30 *minutes* per week. They measured whether kids hit that dosage suggestion across the course of the year, and then their outcome variable of interest was the NJ state test.
"In the last year of the three-year period, 70% of students met the recommended usage of at least 30 hours per week." The dosage can't be 30 hours a week. What was the recommended dosage?
Grr, yes, that should be 30 *minutes* a week.
That's what I thought, but wanted to confirm. Thank you!