Remember the counter-factual
ESSER money helped; it just wasn't enough
Here’s the chart of the day, courtesy of the team at the Education Recovery Scorecard.
You can see the peak in scores about a decade ago, especially in math. You can see the huge declines in the wake of COVID. You can see a small recovery in math and continued declines in reading.
That’s the main, depressing, finding.
But some people have jumped to the conclusion that charts like this show that the federal relief money (ESSER) failed.
That’s not a valid conclusion. In fact, the same report looked at the causal impact of the money and concluded that the ESSER money had a statistically significant, positive impact on student learning. The chart below was their attempt to explain this visually. It looks at math achievement losses by district poverty rate. As you can see, higher-poverty districts suffered larger losses during COVID.
The light gray line shows the losses as of 2022, and the dark gray lines shows the losses as of 2024. The 2024 line is higher, meaning there was some recovery going on. The researchers added the green bar to show the effect of the ESSER money. (I added the red arrows to draw your attention to the ESSER effects.)
Now look at the left axis. All of the numbers are negative, reinforcing the fact that students still have a long way to go to fully recover back to their 2019 levels. That’s important, but so is the fact that achievement would likely be even worse if districts didn’t have the money.
(The reading story is even more complicated, but the authors suggest that the continued declines would have been even worse without the money.)
Ok, but aren’t these tiny effects? The researchers found that the ESSER money increased math scores by the equivalent of .0047 standard deviations for every thousand dollars spent per student. That sure sounds tiny.
Well, yes and no. They are tiny in the sense that students still have a long way to go. But it’s also close to what researchers have found for the effects of school spending on student achievement in other contexts. ESSER was a little less effective, per dollar spent, than a typical funding boost.
But… there are a few different interpretations here. One is just to note that the context was very different. The COVID virus itself, inflation, competition for labor, etc. all made for challenging circumstances.
Two, the response to COVID was so bad, and student achievement fell so far, that even $190 billion in federal funds was not enough to help students meaningfully recover. ESSER just wasn’t big enough to make up for the losses.
Three, you would also be on safe ground to say that ESSER should have made more of a dent. It’s frustrating that we spent all this money and didn’t see more of an effect. And in fact, the authors dug into California data and found that districts that invested more of their ESSER money on academic interventions, such as tutoring or summer school, saw bigger recoveries. The feds could have done a better job of prioritizing what the money was for.
But please don’t go around saying ESSER “didn’t work” because it didn’t help students fully recover their COVID learning losses. The best evidence we have suggests that the money helped; it just wasn’t enough to tackle the full depth of the problem.