When COVID hit the nation in early 2020, the nation’s schools shut down. The disruption continued into the 2020-2021 school year, as most schools offered remote or hybrid learning rather than in-person instruction. Some reopened but suspended in-person instruction because of COVID in classrooms. These decisions became contentious, with critics pointing to their likely impact on student performance. How much did the measures taken to limit the spread of COVID in schools affect student learning in New York State?
The Federal Government recently released the National Assessment of Educational Progress (NAEP) data for 2022. According to the U. S. Department of Education, the report showed, “The national average score declines in mathematics for fourth- and eighth-graders were the largest ever recorded in that subject.” The report also found that the Pandemic had a more significant adverse effect on mathematics learning than reading. New York’s performance was mixed.
In New York State, fourth-grade students saw slightly greater declines in reading and math scores than the nation, but eighth-graders had smaller drops between 2019 and 2022. Fourth-grade student scores in reading dropped from 220 to 214. National scores dropped from 219 to 216. Eighth-grade students in New York saw average mathematics scores drop from 280 to 274, compared with a national decline from 281 to 273. However, the NAEP data is limited because it is only provided for states and large cities.
New York’s annual assessments provide data for each school and school district, allowing a comparison of student performance on the State’s tests between 2019 and 2022. For this analysis, I reviewed the performance of students in school districts in fourth-grade English Language Arts and eighth-grade Mathematics.
New York has more than 700 school districts. Many are quite small, with relatively few students in each grade. Because of the small number of students in many school districts, student performance could be affected by random variation. To reduce the effect of random variation, this analysis includes only those districts with one hundred or more tested students for each exam. Even so, readers should be aware that although there is a high degree of certainty that analyses comparing the performance of all schools as a group are accurate, individual school results might show some random variation.
Fourth Grade English Language Arts
In 2019, for most New York City school districts, student performance at each grade level was better than in districts outside the City at the same levels of economic disadvantage. But, both there and outside the City, student performance in school districts was strongly related to students’ economic status. Sixty-five percent of the variation in performance between school districts was associated with the percentage of disadvantaged students.
Two other factors were significantly associated with school performance – the percentage of English language learners and the rate of disabled students. In 2019, seventy percent of the difference in performance between school districts outside New York City was related to these three factors.
In large city school districts, scores were relatively low. In 2019, statewide, 45% of students were proficient. Only 14% of 4th-grade students in Rochester received proficient scores on the State’s ELA exam. But because of the percentage of disadvantaged students in the district, who were English language learners and those with disabilities, only 15% would be expected to meet the state standard. The performance of students in most large school city school districts was close to what was predicted from the combination of these three factors.
English Language Arts – 2022 vs. 2019
The performance of fourth-grade students who took the State’s English Language Arts exam in 2022 was weaker than in 2019. 40.1% of students outside New York City received proficient scores in 2022 compared with 45.2% in 2019 – a decline of 5.1%. In New York City school districts, the percentage of students who passed dropped by 5.9% to 43.7%.
Districts outside New York City with higher percentages of disadvantaged students had more significant losses than those with lower rates. For the 20 districts with the highest percentages of students with economic disadvantage, the average decrease in percentage passing was 6% compared with 3% for the 20 districts with the lowest rate of disadvantaged students.
Economically disadvantaged New York City School Districts also saw a more significant decrease in student performance on the fourth-grade English Language Arts exam from 2019 to 2022 than more affluent districts. The ten New York City districts with the highest percentages of economically disadvantaged students had an average decrease of 9% in the rate of students who passed the exam—compared with a decline of 6% for the ten districts with the lowest percentages.
Eighth Grade Mathematics
Student performance on the eighth-grade mathematics exam in 2019 showed a similar relationship between economic disadvantage and the percentage of students who were disabled as the English results. The twenty districts with the smallest percentage of economically disadvantaged students averaged a 65% proficiency rate, compared with 10% for the twenty districts with the highest percentage of disadvantaged students. Sixty-seven percent of the difference in performance between school districts outside New York City was associated with the two variables. By itself, economic disadvantage accounted for 65% of the difference.
Overall, student performance on the eighth-grade math exam was weaker than that on the 4th grade English exam, with 33% rated proficient, compared with 45%. New York City school districts overall did better than those outside the City on the exam, with 36% being rated proficient, compared with 28% outside the City.
Large upstate city school districts again had relatively low scores. Still, in places like Buffalo, Rochester, and Syracuse, the percentage of students rated proficient on the eighth-grade math exam was higher than predicted from the percentage of economically disadvantaged and disabled students.
Eighth Grade Mathematics 2019 vs. 2022
Outside New York City, 21% of students taking the Math 8 tests in school districts received proficient scores in 2022, compared with 27% in 2019, a drop of 6%. Scores in the school districts with the highest percentage of economically disadvantaged students dropped by an average of 3%, compared with 8% for those with the lowest rates.
In New York City, the percentage of students receiving proficient scores on the math exam decreased more than those outside the City – from 31% to 22%. Although schools with higher rates of disadvantaged students had lower percentages of proficient students than those with more affluent populations, the ten school districts with the highest rate of economically disadvantaged students saw smaller drops (5%) in the percentage of students who were proficient on the eighth-grade math exam than the ten districts with the smallest rate (10%).
Conclusions
Like the National Assessment of Educational Progress, New York’s annual student assessment performance declined significantly between 2019 and 2022. The percentage of students receiving proficient ratings on the fourth-grade English Language Arts exam declined by 5.1% outside New York City and 5.9% in the City. On the eighth-grade math exam, the percentage of students receiving proficient scores decreased by 9% in New York City and by six percent outside it.
Although students’ performance on the State’s annual performance assessment continues to be strongly related to the percentage of economically disadvantaged students at the school district level, in most cases, districts with smaller percentages of disadvantaged students saw slightly more significant declines in student performance, on average, than did those with higher rates.
The decisions to temporarily close schools in the spring of 2020 and to use remote and hybrid learning in the 2020-2021 school year reflected the potential impact of a disease with no effective prevention or treatment until vaccines were widely adopted in the Spring of 2021. In-person interaction posed real risks to teachers, school staff, and students.
Some have criticized the decisions of school leaders to move to remote instruction because of its negative impact on student learning. But, they did not offer concrete alternatives that would not have increased risk in a challenging environment. In 2020-2021, there was great uncertainty about COVID’s spread and potential impact in classrooms, but its mortality was well known. Without vaccines or effective treatment, protecting the health of teachers and students became paramount.