EdFix Episode 31: A Solution to the COVID Learning Loss Problem

Transcript

RICK HANUSHEK:
The losses in achievement that have been measured on the NAEP test are permanent unless we do something to improve our schools.

MICHAEL J. FEUER:
Welcome to EdFix, your source for insights about the promise and practice of education. I'm your host, Michael Feuer. I'm the Dean of the Graduate School of Education and Human Development at GW. Today I am delighted and honored to have as my guest, Dr. Eric Hanushek. Rick, as he is known to many, is the Paul and Jean Hanna Senior Fellow at the Hoover Institution of Stanford University. Prior to coming to Stanford, Rick held positions at the University of Rochester and has also served in the federal government.

Rick, it is a wonderful pleasure to have you with us. I should just add that in among Rick's many accomplishments, most recently, we were all very happy to know that Rick had been honored by receiving the Yidan Prize for Education Research. Rick is the author of too many widely cited studies on education policy, the economics of education, everything from class size reduction, to accountability, to teaching, and many other related topics, and has become an extremely well-known and influential voice in the ongoing debate about international comparisons. That is how the US performs compared to other countries and the role of education in assuring our economic future. A distinguished graduate of the US Air Force Academy, Rick has his PhD in economics from the Massachusetts Institute of Technology. Rick Hanushek, welcome to EdFix. What a pleasure to be together.

RICK HANUSHEK:
It's wonderful to be with you, Michael. Looking forward to our conversation today.

MICHAEL J. FEUER:
Well, thank you so much. Let me get right into it. Let's start with, Rick, your sense of the data about what we tend to refer to as the learning loss, or at least the fact that students stopped improving at the rate that, even though it was perhaps disappointing and not good enough, but at least it was positive before the pandemic. What's your sense of where we are today?

RICK HANUSHEK:
Well, I think that we're facing a real challenge because our students who have been in school during the pandemic have really been harmed, and they've been harmed because they've missed out on the good teachers that they would've had otherwise when there was the lockout from schools, and the remote learning and hybrid learning just has not been up to the standards that we expected before for our schools in terms of the quality of education.

If you look at the data, the data are just pretty vivid, although they're hard to understand for most people. What comes across is a public announcement that the National Assessment of Educational Progress, our national testing of students, shows an eight point loss in mathematics between 2019 and 2022. You get varied reactions to that. One reaction is seen in a couple media stories in the New York Times and the Washington Post that says, "Ah, it wasn't as bad as we thought. It was only eight points." So nobody quite understands what this means.

Well, what it means, Michael, is that we pushed the slow improvement in mathematics that we saw over the last 25 years back two decades, so that we lost two decades of improvement in the schools. More than that, students who were in schools over this period can expect to lose something on the order of 6% of their lifetime earnings by virtue of this slowdown in schools. I think most people would understand that 6% is a real number.

MICHAEL J. FEUER:
Well, here you've already touched on a number of really very important nuances about both the effects of the pandemic and what the data mean and what we should be thinking about, but I want to go back to one point which has to do with the effect of the students not having access to the good teachers that they had in their schools because the schools were closed. This suggests that many of our students had reasonably good teachers, or at least had teachers who were helping them achieve, if not to somewhat utopian levels, but certainly making progress. Say a little bit more about what you mean by students not having access to the good teachers in their schools.

RICK HANUSHEK:
You want me to get into controversial areas right from the start, I take it. Well, before the pandemic, as you hinted, there was a lot of interest in actually improving the quality of our schools. Our schools, by the international test, come out something like 31st in mathematics in the world, and that's not the image that most people want, and it's also not what is good for either the students or the economy to fall behind in that way. When I talked about the teachers, what we know is that in-person instruction is really valuable, and the average in-person instruction that we had before the pandemic was lost and degraded by first shutting the schools for varying number of months, depending upon what state and what city you lived in, and the alternative instruction modes just weren't up to the same as in-person instruction.

Now, you're pushing me for the controversial part, and the controversial part is that we actually know a lot about the distribution of quality of teachers. All of us can point to one or more really important teachers in their lifetime. People in the past that had a great influence on us, both in terms of our learning and our motivation and our perspective on the world, but each of us can also probably point to one or two or a few really bad teachers that didn't do much for us.

If we look at the data on the distribution of quality of teachers, quality measured in terms of what kids are learning in their class, it turns out that if we could cut down on the really harmful, poor teachers just a little bit, we would be internationally competitive. So if you talked about taking the bottom five to 10% of the distribution of effectiveness of teachers and replacing them with an average teacher, the US performance, by what we've seen historically, would grow to the top of the international league tables, so that when I say quality teachers, I'm actually talking about the average teacher and the whole range. I'm not talking about just having only superstars in the classroom, where we know there are superstars, but I'm talking about getting us just a bit better by opening up kids who have had teachers at the bottom to an average teacher.

MICHAEL J. FEUER:
Part of the debate that one sees is between people who argue that until we actually eradicate poverty, there's not much hope for educational improvement because the schools can only do so much given the effects of poverty. On the other hand, your statement, which by the way, I tend to agree with, is that schools can be doing a lot better, and even until we fix poverty. Help me unpack that a little bit.

RICK HANUSHEK:
Sure, I'd be happy to. We've seen that poverty is important, but we also know that teachers in very poor, disadvantaged schools can do a terrific job and can, in fact, move their kids far along. One of the earliest studies that I did looking at teacher quality looked at schools in Gary, Indiana. They were all poor kids. They were all Black kids in the income maintenance experiment in Gary, Indiana. If you walked down the hallway in the schools of Gary, Indiana, you saw that the kids in some classrooms were learning at a year and a half pace. They were learning half a year more of material than expected of them. You walk down to another room in that same school and you see that the kids are learning at half a year's pace, so that depending upon what classroom you were put in, you could end up a full academic year behind in one calendar year.

That demonstrated two things, or three things actually. There are big differences in teachers. Secondly, there are really good teachers in what we call disadvantaged schools, and it's just hard to spot them because the kids over a period of time haven't stayed at grade level, but they've fallen behind. But some of those teachers are bringing them up and faster than a grade level at a time, at a year. Then thirdly, if we altered the distribution of teachers, we could perhaps deal with some of the really large achievement gaps that we see by family background, because good teachers can, in fact, by everything we observe, ameliorate and in fact eliminate some of the background differences among kids.

MICHAEL J. FEUER:
Which brings us to then, since we're talking about schools and teachers, let's go back to the business of now the measured or the estimated learning loss. First of all, I want to ask you to talk about the effects this has not only on the kids who have lost out to some extent, but also, and in terms of their earnings capacity, which you mentioned, but this sounds to me as if it has effects on the economy more broadly, and I know you've been writing and thinking about that also.

RICK HANUSHEK:
Absolutely. But before I do that, let me just circle back for one little footnote on what I just said. It's clear that the pandemic made teaching a much more difficult task, and the teachers are under a lot of pressure today and being pushed in all kinds of directions and trying to learn new tricks of how you deal with hybrid instruction and so forth. So we shouldn't forget that, and we should, in fact, pay attention to the fact that teachers are under this pressure.

Let me talk for a second about the aggregate impacts of the pandemic on the US economy. What we know from past research is that economic growth of nations is very closely tied to the skills of the population in each country, and the skills of the population in each country can at least pretty well be measured a proxy by achievement, achievement on these international tests that we talked about. There's the PISA test and the TIMSS test, international test. Performance on those tests indicate a level of skills that's really important for economies.

Now, what the pandemic did was say that there's one cohort of students that was in school during this pandemic period that's going to go out into the world most likely with lower skills, and the whole country over time will have a less skilled workforce than it would have, had there been no pandemic. Now, we actually have ways to estimate that from the historical record of how economic growth relates to qualities of the schools in the kids, in the quality level of education and achievement. The pandemic by my estimates cost us three to 4% lower GDP, gross domestic product, for the remainder of this century. Three to 4% of GDP is much, much, much greater than all of the fights in your home city of Washington, DC about budget issues, because what we're seeing is that the economy is going to suffer, and we could actually take care of many of those fights, at least in a fiscal sense, by getting our population back on track.

Now, I keep hedging a little bit here. All of the historical evidence that we have, from other countries in particular, but also from what we see over summers in the US and so forth, is that the losses in achievement that have been measured on the NAEP tests are permanent unless we do something to improve our schools. So if we just this academic year got back to our January 2020 levels of performance in the schools, that's a permanent loss to the kids that are involved when they're adults and to the economy. The only way that we can deal with this is to actually make the schools better than they were in March of 2020 when we closed the doors for the first time.

MICHAEL J. FEUER:
I think we're getting now a structure of this argument. We understand that there's been a significant loss in learning because of the pandemic and the schools being closed, et cetera. We know that this effect has not been distributed evenly. We know that the effect has impacts, both for the kids in terms of their earnings, what we would call private returns to education, but also in terms of social returns because of the condition of the whole economy. Therefore, the question becomes what do we do about it? I'm anticipating that part of your answer will be about the quality of teachers. So what could we be doing next?

RICK HANUSHEK:
Good guess, Michael. You've hit-

MICHAEL J. FEUER:
I read well, good.

RICK HANUSHEK:
If you look at the schools across the country, they've been struggling a bit to figure out how to just get back to even footing, where they were in March of 2020. I think many of the schools of the country haven't done it for a whole host of reasons. It might be that they've lost some teachers. It might be that parents are reluctant to send their kids into schools all the time. It could be kids have become disillusioned with the idea that you go to school every day as opposed to do other things. All kinds of reasons.

The reaction of schools in large part has been to let's provide a little bit more of what we've been doing. Let's provide either some lengthened school years, lengthened school days, or what has become the buzzword, high dosage tutoring. If you look at the record on providing more time, more tutoring and so forth, and you think, what would happen if we could implement this perfectly across the country? Would we take care of this problem? The historical evidence suggests that these approaches would only take care of a small portion of the total learning loss that we've seen during the pandemic.

The thing that I would come back to, as you forecast, was getting high quality teachers mobilized to, in fact, deal with these learning losses, and we can actually do it today. We could do it in time to get most of the kids that have been hurt. How? Well, you provide incentives to the best teachers in each school to take on more kids, larger classes, more attention. You provide them support, extra support in doing this. You provide them extras, salaries for the harder work they're doing. And then, at the other end, the part where... We'll get back to the controversy that you like to encourage in your podcast, we could actually think of buying out the contracts of some of our poorest teachers, our less effective teachers.

Now you say, "Well, that all sounds like a lot of money." Well, it turns out that almost every school district in the country today is just flush with money because the federal government in the pandemic relief acts put a huge amount of money into schools and into individual schools. The schools haven't actually spent this yet. They have to spend it, at least according to current rules, in another year and a half, but they could use these funds to, in fact, provide incentives for our best teachers to help more kids and to change the incentives for our poorest teachers.

MICHAEL J. FEUER:
This also suggests that we have enough confidence in the way we evaluate the quality of teachers to make the distinction between those in whom we're going to try this incentive model as being the better teachers. How confident are you that we actually know how to measure teaching quality, holding constant all the other things that affect academic performance?

RICK HANUSHEK:
Well, I'm probably more confident than others that we can measure teacher quality. I would never do it, by the way, just on test score basis. I would use test score information if I had it, but it turns out that observational information is pretty good too if you have good principals or raters of teachers. I don't think that there's much doubt about who's in the top half of the distribution and who's in the bottom half. I think that anybody could walk into a school and in very short order tell you which teachers were in the top half of the distribution and which were in the bottom half. The principal clearly knows a lot about the distribution. The other teachers know about the distribution of teachers. The parents and the kids know about the distribution of teachers. The janitor knows about the distribution of teachers. So using information of a fairly coarse nature, like putting teachers in the top half of the distribution, would probably be sufficient to, in fact, improve things, if we shifted the balance from the bottom half of the distribution to the top half of the distribution.

MICHAEL J. FEUER:
At some point, even just to get some kind of program like this started, do you think it would require a major federal investment? Or are you suggesting that school districts operating in their states, because they currently have this extra treasury due to the pandemic relief, should be willing and able to bear the cost of, first of all, doing the proper measurement and differentiation of teaching quality, and then providing these incentives, which probably will have some at least short term economic costs? Who's going to do all this?

RICK HANUSHEK:
Well, in the short run, the schools have the money provided by the federal government, but with very few strings attached to those funds. In longer term, we're not going to change or think of changing the fact that the states are in charge of schools and they run schools and the federal government doesn't. The federal government provides specialized funding for basically special education and compensatory education, helping poor kids. They don't do a particularly good job at monitoring how those funds are used. We have little evidence of that, but I think we have to rely on the states to, in fact, take the lead in this. So there are examples of where this is done first, where effective teachers are rewarded and ineffective teachers aren't.

We have Washington, DC, where you are having your office today, that has been evaluating and rating teachers for a dozen years now. They give very large bonuses to the best teachers and they fire the worst teachers. That's been going on for a dozen years. Dallas, Texas has started to put its system on a similar basis of an elaborate, and I think pretty good, evaluation system, that's used to determine pay and position in the school system, used entirely. The experience and degree levels, the classic determinants of teacher salaries, don't come into play. What comes into play is how effective the teachers are.

When Dallas put in this system a few years ago under the leadership of Mike Miles, they first implemented across the school district, so that's in place now. Secondly, they had a bonus system to get the most effective teachers to work in the most disadvantaged schools. It turns out that teachers respond to incentives. So they had a monetary incentive to do this. They moved to the disadvantaged schools, and the disadvantaged schools got better almost immediately by getting a core of good teachers in them. The State of Texas, seeing this, passed legislation that provided general incentives across the states to other school districts that had some sensible evaluation system, not completely specified, they didn't have to duplicate Dallas. They had to have a sensible evaluation system, and they had to have a policy of putting good teachers in the most needy schools. So they've put this into effect. It hasn't been evaluated because it's too early, but they have, in fact, devised ways to do this in Texas. Other states could conceivably do the same.

MICHAEL J. FEUER:
It's a two part question. If it had been a matter of creating incentives for better teachers to take on more of the tough classrooms and do a better job, we didn't really need a pandemic. We could have been doing this for decades. That's question number one. Number two, to what extent does the lack of capacity across the states to actually know how to evaluate their teachers play into this? Even if they had the goodwill and the intention to try the kind of model you're suggesting, would they know how to do it?

RICK HANUSHEK:
How do we do this? Why hasn't it happened? There are two things about Washington, DC and Dallas that are important. One is that they had really strong leadership in both places that were visionary in some real sense. Secondly, they had restrictions on the impact of the teachers' unions on thwarting these policies so that US law that governs the DC schools doesn't allow contract negotiations over anything but salaries and benefits, as opposed to working conditions and evaluation systems and so forth. Texas has a lack of collective bargaining. So those things obviously had an influence. The teachers' unions have not been very supportive of policies that would seek to improve the quality of schools through dealing differential with their workforce. So that's been a long term policy. Secondly, I think that even a dean of an education school in Washington, DC would be able to evaluate the teachers in each school. I don't think that this is as mysterious as you say. What is mysterious is that partly the impact of the teachers' unions on this problem.

20 years ago when I was talking to the president of one of the teachers' unions of this country, she said, "Well, we don't have anything really against performance pay and evaluations, as long as it's objective and fair." Well, we developed objective and fair methods that had to do with statistical evaluation of performance of students and what happened there, and of course, as you know, that was anathema to the unions to actually use data on student performance to judge the effectiveness of teachers.

So there's been a longstanding push and resistance to doing things that basically everybody in the country knows how to do. Nobody's going to object to the idea that we pay attention to teacher quality. You can't find a person in this country who's going to object to that. What people object to is the fact that you might have a system, a personnel system based on that. But in fact, 80% of the country, including deans of education schools at major universities, are judged on their performance. We don't have objective unassailable evaluations of any of, or most of these 80%, but we do have the ability to make decisions based upon agreed upon management rules. And if the management screws up, then you try to replace the management, not the system.

MICHAEL J. FEUER:
In education, it's because it means so much to so many of us that we get passionate about it, and argumentation about education is something that Rick is familiar with, because people have been arguing with him and around him and using his name in arguments for a long time. I just want to thank you, Rick, for all that you have done for the profession, even if I don't necessarily agree with everything that you're espousing, and there's a quality of discourse and there's a quality of reliance on data, and those are things that I think matter the most for the future of our democracy and for the use of knowledge to advance the social good. So thank you for being with me on this podcast.

RICK HANUSHEK:
Thanks for the kind words and thanks for the conversation. I always enjoy the conversation. Sensible people don't agree with everything I say, but I'm trying to persuade you that there's more and more that I say that you should agree with.

MICHAEL J. FEUER:
On that happy note, let me say again, thank you to Rick Hanushek for joining me today. To our listeners, if you enjoyed this episode, you can subscribe to EdFix on your Apple Podcasts or Spotify or iHeartRadio, Player FM or wherever else you might listen to podcasts. We also have a website called edfixpodcast.com. Special thanks to our producer, director, engineer, and all around maven of everything related to podcasting here, Touran Waters. Thank you again, Rick Hanushek. All best wishes.

(Rick Hanushek's full policy brief: “A simple and complete solution to the learning loss problem.”)

EdFix: A Podcast About the Promise and Practice of Education

Hosted by Michael Feuer, Dean of GW's Graduate School of Education and Human Development (GSEHD), EdFix highlights the effective strategies and provocative ideas of researchers, practitioners and policymakers on how to improve our education system. Listen in as Dean Feuer connects their worlds to take on some of education's most complex issues.

From preschool to postsecondary, get your fix with EdFix!

Subscribe on Apple Podcasts, Spotify, iHeartRADIO, Google Podcasts, YouTube, or wherever you listen to podcasts.