How did students fare relative to the COVID-19 learning loss projections?

By Angela Johnson, Megan Kuhfeld, Beth Tarasawa

The SARS-CoV-2 (COVID-19) pandemic has disrupted traditional forms of education and continues to create challenges for K-12 school systems in the United States and the students they serve. Districts attempted to transition academic and non-academic activities to remote settings in spring of 2020, and many continue to do so through the fall of 2020, with varying degrees of success. Policy makers and educational leaders face difficult decisions around school re-opening and how to best target limited resources to students well into the 2020-21 school year and beyond.

To help educators and policymakers better prepare for the potential magnitude of the impact of COVID-19 on learning, NWEA researchers and collaborators published a set of projections that considered multiple scenarios: (a) typical learning (assuming 2019-20 were a normal school year), (b) partial absenteeism (assuming students received half of their normal instruction in spring 2020), and (c) COVID-19 Slide (assuming spring 2020 school closures operated like a typical summer break). These projections indicated that students would likely return in fall 2020 with 63-68% of a typical year’s learning in reading and 37-50% in math.

With actual fall data in hand, we can now move beyond forecasting to begin to describe what did happen. We recently released a research brief summarizing our findings of how school shutdowns impacted student achievement at the start of the 2020-21 school year. This research was conducted with fall 2020 data on the NWEA MAP Growth test taken by millions of students in grades 3-8 in over 8,000 schools across the U.S. The scope of these data is unparalleled, providing a national perspective on student progress since COVID-19 school closures in March 2020.

Using these data, we compared actual student performance in fall 2020 to our projections. We calculated the average test score for students who tested within specific timeframes in fall 2019, winter 2020, and fall 2020 and overlaid those observed means (circles) on our projected academic growth trajectories (solid/dashed lines) Students in our sample were higher achieving relative to the broader sample taking the MAP Growth tests in fall 2019. Because the scenarios were only projected to the start of the 2020-2021 school year (e.g., September 1, 2020), we extrapolated academic growth assuming typical grade-level growth (based on NWEA MAP Growth norms) to extend the projections through fall 2020 for all three scenarios during the 2020-21 school year.

In reading, average fall 2020 test scores were consistent with the typical learning scenario. In fact, the sixth- and eighth-grade averages at all three time points are at or above the typical reading growth trajectory. This means the students in the sample were performing above the 50th percentile prior to the start of the pandemic and were aligned with the typical-learning scenario through fall 2020.

In fall 2019, average math scores in our sample were above or on the typical growth line. In math, average fall 2020 test scores were for some grades on par with typical growth scenario (e.g., eighth graders) and for others below typical and close to the partial absenteeism growth scenario (e.g., fourth and sixth graders). This suggests that the rate of math growth for 2020 was reduced.

Then, we approximated actual fall 2019 to fall 2020 growth as a percentage of learning gains in a typical year. In reading, while we expected students to return with 63-68% of a typical year’s learning, observed data show that students started fall 2020 having made an average of 86-107% of the prior year’s learning. In math, we observed that students started with the year having made 34%-80% of prior-year learning gains, relative to the expected 37-50% under the COVID Slide assumptions.

These initial findings are encouraging: students appear to be doing better than our projected learning loss scenarios. However, there are systematic patterns of missing data that could be impacting our results. Approximately one in four students who tested within our sample of schools in fall 2019 did not test in fall 2020. The students observed in fall 2020 had higher 2019-20 average test scores than the missing students, thus we are likely not capturing the students hit hardest by the ongoing disruptions to learning. It will be essential to continue examining students’ academic progress throughout the 2020-21 school year to understand how recovery and growth unfold amid the continuing pandemic.

Read the article

Article details

Projecting the Potential Impact of COVID-19 School Closures on Academic Achievement

Megan Kuhfeld, James Soland, Beth Tarasawa, Angela Johnson, Erik Ruzek, Jing Liu

DOI: 10.3102/0013189X20965918

From Educational Researcher

About the authors

Featured

Dr. Megan Kuhfeld is a senior research scientist at NWEA whose work focuses on understanding students’ trajectories of academic and social-emotional learning and the school and neighborhood influences that promote optimal growth. Dr. Kuhfeld’s work covers a range of topics, including longitudinal growth modeling, achievement gaps, and summer learning loss. It has been featured in Educational Researcher, the Los Angeles Times, the Journal of Research on Educational Effectiveness, and Psychological Methods. Prior to joining NWEA, Dr. Kuhfeld completed a PhD in quantitative methods in education and a master’s degree in statistics from the University of California, Los Angeles (UCLA). Her dissertation was funded by a grant from the American Educational Research Association and was the recipient of the 2017 Brenda Loyd Outstanding Dissertation Award from the National Council on Measurement in Education. She also worked at the non-profit policy institute Child Trends, the Population Research Center at the University of Texas at Austin, and UCLA CRESST.

Featured

Dr. Angela Johnson specializes in experimental, quasi-experimental, and mixed methods research designs. Her research focuses on identifying and reducing inequality. Much of her work is done in partnership with schools and districts, with an aim to produce findings that immediately inform practice and policy. Her ongoing projects examine academic opportunities and achievement growth for students with diverse needs, such as English Learners and students eligible for Special Education services. Before embarking on a full-time research career, she was a teacher for many years. She taught English as a second language in the US and abroad for a decade. More recently, she taught a variety of graduate-level courses on economics of education and quantitative research methods. Dr. Johnson holds a PhD in economics of education from Stanford University and master’s degrees in economics and teaching English to speakers of other languages (TESOL) from the University of Southern California.

Featured

Dr. Beth Tarasawa, Executive Vice President of Research at NWEA, is relentlessly committed to using data-driven insights to transform education and eliminate structural barriers to achieve equitable outcomes for all students. She leads NWEA’s Research teams, the Center for School & Student Progress and Collaborative for Student Growth, that are devoted to advancements in assessment and growth measurement, answering fundamental questions to inform policy and practice, and drive educational innovation. Dr. Tarasawa’s research focuses on issues related to educational equity, particularly those concerning social class, race, and linguistic diversity. She also teaches in the College of Education at Portland State University. Prior to joining NWEA, she was a faculty member at Washington State University and St. Norbert College. She earned her PhD and MA in the sociology of education with a concentration in education policy at Emory University.