The COVID-19 Pandemic disrupted teaching and learning at schools nationally and in New York State. School closures and remote learning removed the opportunity for students to interact directly with teachers. National testing shows that this led to significant learning losses. New York State students lost six points on the fourth-grade English (220 to 214) and eighth-grade mathematics (280 to 274) National Assessment of Educational Progress tests.
Each year, New York State evaluates the academic performance of third through eighth graders. The State’s annual assessments measure student proficiency in English Language Arts (reading and writing) and Mathematics. Because of the disruptive effects of COVID-19 in 2020 and 2021, 2022 is the first year after the Pandemic struck when school activities returned to near normal. The State Education Department reports, “Due to the ongoing impacts of COVID-19, it may not be appropriate to compare 2021-22 NYS standardized assessment results with results from prior years.” The data permits the analysis of several critical factors to assess the continuing impact of the pandemic-related measures, including the relative performance of schools with high percentages of economically disadvantaged students with those with lower rates and district-operated versus charter schools.
Economic Disadvantage
Historically, economic disadvantage has been the strongest socioeconomic predictor of school-level student performance. In this analysis, in both 2019 and 2022, at district-operated schools, that rate was associated with more than sixty percent of the difference in the percentage of students who achieved proficient scores on New York’s English Language and Mathematics exams in grades three through eight.
Charter schools are independently operated public schools created to break the relationship between economic disadvantage and poor student performance. The relationship between student disadvantage and student performance at charter schools is much weaker than at district-operated schools. However, the weaker relationship would have somewhat disappointed school reformers. Students at charter schools with higher percentages of disadvantaged students performed more poorly than those with fewer disadvantaged students. Nor was charter school performance uniformly excellent. The chart above, which compares student performance in 2022 at charter and district-operated schools in New York City on the grades 3-8 mathematics exams, shows that at some charter schools, student performance excelled, with almost all students attaining proficient scores. But, in others with the same percentages of disadvantaged students, less than 10% did.
Higher percentages of charter school students in New York City attained proficient scores than at district-operated schools. For those where 90% were disadvantaged, on average, 38% of charter school students passed, compared with 25% at district-operated schools. In 2022, thirty-six percent of charter schools scored at least 20% higher than the average of district-operated schools. Even so, twenty-nine percent of charter schools performed below the district-operated average.
Outside New York City, charter school performance was very varied. At 31% of charter schools outside of the City, student performance on the math exam was at least 20% better than at district-operated schools, controlling for disadvantage. But, students at twenty-nine percent of charter schools performed less well than the average of district schools.
Student performance on the math exam at charter and district-operated schools outside New York City was weaker than at schools in the City. Seven percent of students at district-operated schools, where 90% of students were disadvantaged, passed the math exam on average outside New York City, compared with 23% in it. At charter schools, 38% passed in New York City compared to 22% outside.
Performance in 2019 and 2022: English Language Arts
The average performance of third through eighth-grade students on the State’s English Language Arts exam in 2022 and 2019, controlling for the percentage of students who were disadvantaged at district schools, was almost identical. But, students at schools operated by New York City did significantly better than those outside it in both years. On average, in New York City District Schools, where 10% of students were disadvantaged, 92% passed the state English Language exam. For schools where 90% of students were disadvantaged, 36% passed. Outside New York City, the comparable percentages were 63% and 18%. Other studies have noted the more robust performance of students in New York City, where the rate of disadvantaged students is similar.
Charter school performance in New York City varied slightly in 2019 and 2022. Although the average performance of charter schools controlling for economic disadvantage was better than that of district-operated schools, in 2019, 24% of New York City charter schools had lower proficiency rates than the average for district-operated schools, controlling for economic disadvantage, while in 2022, twenty-two percent did. Outside New York City, the percentage was 18% in 2019 and 16% in 2022.
On average, Charter school English Language Arts performance was better than public school performance in 2019 and 2022. However, charter school outperformance compared with district-operated schools declined. In New York City, controlling for the percentage of economically disadvantaged students, 36% of charter schools outperformed the average of district-operated schools on the English Language Arts exam by 20% or more in 2019. In 2022, 26% did. Outside New York City, 33% of charter schools outperformed the average of district-operated schools by at least 20% in 2019, while in 2022, 29% did.
Performance in 2019 and 2022: Mathematics
Student performance on the third through eighth-grade math exams was more affected by the Pandemic than the English exams, suggesting that the loss of in-person classroom time had a more significant impact on math scores. In New York City, at district-operated schools with the highest percentage of disadvantaged students, about 18% of students passed on average, compared with 28% in 2019. For district-operated schools outside New York City in 2022, on average, almost all students in schools with high percentages of disadvantaged students failed to reach proficient levels of performance compared with a 15% passing rate in 2019.
District-operated schools with low percentages of disadvantaged schools saw smaller performance declines than those with high ratios. In New York City, passing rates at schools with low percentages decreased by six percent, compared with 10% for those with high percentages. Outside New York City, the difference was 10% at schools with low ratios, compared with 13% at schools with high rates.
Charter schools also saw declines in the percentage of students who scored proficient in the State assessments. Outside New York City, the rate of students who passed the exams decreased by 17% on average, with the decrease being slightly more significant in schools with high percentages of disadvantaged students. On average, charter schools with high rates of disadvantaged students had about 15% higher passing percentages than district-operated schools.
Charter schools in New York City saw a significant drop in performance on the math assessment between 2019 and 2022, mainly where high percentages of economically disadvantaged students were present, with an average decrease of 26%. In 2022, charter schools’ advantage over the average performance of district-operated schools with nearly 100% economically disadvantaged students was only 12%, compared with 28% in 2019.
As with the ELA scores, student performance at charter schools varied significantly. In 2022, in New York City, at 36% of charter schools, the percentage of students passing the Math assessment for grades three through eight controlling for disadvantage was at least 20% higher than at typical district schools. But, at 29% of charter schools, student performance on the assessment was weaker than the district school average.
Outside of New York City, at 31% of charter schools, student passing rates controlling for economic disadvantage were at least 20% better than the average of district schools. Passing rates were weaker at 29% of charter schools than at typical district schools.
Conclusions
Although national data shows the COVID-19 Pandemic resulted in student learning losses in mathematics and English language arts, New York state assessments show more significant losses in mathematics. The performance losses in mathematics were more important for schools with high percentages of disadvantaged students.
Data from the grades three through eight student proficiency assessments shows little change in the percentage of students who received proficient scores on the English Language Arts examination. Student performance was strongly related to the rate of economically disadvantaged students at district-operated schools.
On average, student performance at charter schools was better than at district-operated schools; but, controlling for economic burden, differences in charter school performance were substantial, with many having much higher passing rates than district-operated schools and others having significantly weaker performance. At charter schools, the relationship between performance and disadvantage was very weak. However, the average performance was better at charter schools with low percentages of disadvantaged students than at schools where rates were high
On the State’s mathematics proficiency examinations, student performance declined significantly between 2019 and 2022. Less than 10% of students at district-operated schools outside New York City, with high percentages of disadvantaged students, passed after the COVID pandemic struck. Although students at many charter schools did somewhat better, at some, almost all students failed the math exam.
The data shows the impact of lost time in class and in-person learning time on student performance in New York schools, particularly in mathematics. Over time, the lingering effects of the COVID disruption may be expected to lessen, though how much long-term damage will remain is unknown. The unknown nature of the COVID threat to students and staff in the Pandemic’s initial stages warranted a cautious approach that focused on minimizing exposure. But as important as that was, rapid adoption of policies to reinstate regular classes was also needed. To achieve this, close monitoring and analysis of exposure impacts are critical to informed decision-making.