The National Assessment of Education Progress (NAEP) Report Card on learning loss was a sobering but not unexpected reckoning for how deeply and broadly the pandemic impacted student learning and achievement.
NAEP state-level findings of drops in math and reading scores were followed by the release of the Education Recovery Scorecard, which leveraged NAEP data to offer the first comparable view of district level learning loss during the pandemic. This one-two punch confirms that COVID-19 learning loss was extensive and, in some cases, worse than expected. Recommendations on how to move forward are not in short supply, and for many, data lies at the heart of transitioning from learning loss to learning recovery.
Funding, policy, and learning decisions without data is a recipe for disaster – particularly given estimates that it will take hundreds of billions of dollars to offset the impact of learning loss. But we also need the right data and the right approach to interpreting this data, to initiate a successful learning recovery process.
Holistic learning loss data only goes so far
During my tenure as South Dakota Secretary of Education, I witnessed firsthand the importance of data to support and enhance all aspects of student, teacher, and institutional performance. With roughly 150 school districts, statewide data held value, but the diversity of education experiences across urban and rural areas underscored the need for individual student data as well. In 2019, 40 percent of South Dakota students attended rural public schools, which meant different student-teacher ratios and access to digital learning. Despite assumptions, however, standardized test scores in rural areas often kept pace or outpaced those in more populated areas.
State and district-level learning loss data is critical. But learning recovery requires analysis of individual student-level data. That’s why attention is being paid to a dozen states that have been specifically tracking COVID learning loss all the way down to individual students. All told, the data from these states represented approximately 15 million students who participated in state assessment programs. The individual state analyses used students’ entire available testing histories in all tested grades and subjects. In this approach, students are compared to themselves.
This statistical approach is used to predict how students would have scored on assessments absent the pandemic. By comparing those results to the expected scores and assessing how students performed versus how they were expected to perform, one can arrive at a student-specific measure of learning loss. The intended value is to reveal the strengths and struggles by school, grade, subject, student group, and individual students. That’s the level of information education leaders, and teachers need to make instructional decisions and allocate resources for learning recovery and acceleration.
Student-level data will help guide learning recovery
Regardless of how states responded to the pandemic, this look at student-level data revealed commonalities that mirror the national findings, as well as anomalies that should be heeded when making learning recovery investments.
Core data trends mirrored state and district-level findings.
In some cases, individual student-level findings tracked closely with expected results. For example, students who received mostly or all virtual instruction tended to experience greater learning loss than those who primarily received in-person instruction. Chris Neale, Missouri Assistant Commissioner of Education, shared earlier this year that Missouri education leaders found “striking results” confirming the value of direct student contact with teachers relative to virtual instruction. The negative impact of virtual learning was more pronounced in Black and Hispanic student populations, which were more likely to use virtual instruction.
Data outliers challenged conventional wisdom.
The importance of individual student-level data was underscored by findings that did not follow perceived learning loss variations. Dr. Jeni Corn of the North Carolina Office of Learning Recovery and Acceleration said, “We used the data to unpack and identify promising practices – ‘positive outliers of the pandemic’ is the language we use. Despite the chaos of the pandemic, learning was happening. Teachers were connecting with their students.”
Dr. Corn explained there were some interesting anomalies. Notably, students with disabilities and English language learners outperformed compared to the general population as it relates to learning loss. These populations have built-in supports, which Dr. Corn suspects made a difference, “We think it was [a result of] the direct, targeted services for those learners that we could activate as soon as the pandemic hit.”
These compelling findings are made possible by using all available assessment data, not a sample, and comparing students against themselves instead of a different set of test takers from years earlier. The results provide districts and states with valuable data for teachers as they return to the classroom to make decisions on targeted interventions for individual students.
Looking beyond recovery
The federal government has poured billions of dollars into learning recovery, and states are already being asked to account for how that money was spent. Did students get back on track? Did they make up what was lost? Did they even accelerate beyond pre-pandemic expectations?
This is an opportunity to rethink, innovate and transform education so that academic progress accelerates as the pandemic’s educational effects wane. Equipped with a trove of valuable learning loss data, states can understand their own unique and evolving challenges. They can more effectively prioritize interventions, continuously analyze and improve learning strategies and show how they used once-in-a-lifetime funding to impact all students positively.
Related:
Understanding historical trends and patterns in student data
- The importance of the ITS and Facilities relationship - March 22, 2024
- How edtech is transforming bilingual education in the U.S. - March 21, 2024
- 5 ways to keep schools safer with innovative visitor management - March 21, 2024
