Note: A more recent, closer look at this subject may be found here: https://policybynumbers.com/a-closer-look-at-student-performance-in-upstate-city-schools
In 1966, James S. Coleman and associates wrote the report, “Equality of Educational Opportunity,” for the United States Department of Health Education and Welfare, as required by the Civil Rights Act of 1964. The report was commissioned to examine the causes of differences in educational outcomes experienced by minority group students, particularly African-American children compared to majority group whites. Initially, the report’s sponsors had expected that differences in resources available to black, Hispanic and white students would explain the differences.
Coleman, however, found that another factor was far more important. “The first finding is that the schools are remarkably similar in the way they relate to the achievement of their pupils when the socioeconomic background of the students is taken into account….When these factors are statistically controlled, however, it appears that differences between schools account for only a small fraction of differences in pupil achievement.” (pp. 21-22). Coleman’s report has led to the notion that factors like parents’ education and incomes determine the performance students in schools.
But Coleman went on to do further research. In 1982 he published, “Cognitive Outcomes in Public and Private Schools.” Coleman found “strong evidence that there is, in vocabulary and mathematics, higher achievement for students in Catholic and private schools than in public; the results are less consistent in reading.” In large part, Coleman attributed this to his finding that “Private schools provide. a safer, more disciplined, and more ordered environment than do public schools.” Coleman’s study is controversial. Critics have argued that his findings could be attributed to other factors, particularly the likelihood that self-selection may be an unmeasured but important factor in these schools’ outperformance.
In New York State, as in other places, students in central cities are much less likely to pass the state’s standardized tests than those in other, more affluent, locations. One of the consequences of the long-term problem of high failure rates in these schools has been the call for reforms, such as the establishment of charter schools to provide alternatives – presumably better ones – to schools operated by public school districts. Others have called for state takeovers of underperforming school systems.
What does the data show? How strongly does socioeconomic status influence performance in school districts in New York State? Do some districts outperform or underperform expectations?
Upstate Central City Student Performance
Few students in upstate central city school districts do well on New York’s standardized examinations. Fewer than one in five students in these cities passed the state’s English and mathematics examinations given between third and eighth grades in 2016 and 2018. Utica was the only large upstate city school district in which more than 20% of grade 3-8 students passed the state’s English Language Arts and Mathematics exams in 2016 and 2018. In Rochester, only 9% passed, while in Syracuse, 13% passed. In some selected nearby suburban districts, the picture was quite different – in most cases, more than 60% of students passed the state exams.
Economic Disadvantage and Parents’ Education as Predictors of Student Performance
New York State defines economically disadvantaged students and family as those who take part in assistance programs “such as the free or reduced-price lunch programs, Social Security Insurance (SSI), Food Stamps, Foster Care, Refugee Assistance (cash or medical assistance), Earned Income Tax Credit (EITC), Home Energy Assistance Program (HEAP), Safety Net Assistance (SNA), Bureau of Indian Affairs (BIA), or Family Assistance: Temporary Assistance for Needy Families (TANF).” When two years of data were combined to reduce random variation associated with small sample sizes, 72% of the variation in student performance was associated with the percentage of students defined as economically distressed. The chart below shows the strong relationship between the percentage of school district students passing New York’s Grades 3-8 ELA and Math exams and the percentage of disadvantaged students in school districts.
We cannot directly measure the educational levels of parents in the school districts studied, but Census data permits us to examine the relationship between the percentage of adult district residents who have college degrees and student performance. Separately, the percentage of college graduates is associated with 48% of the variation in student performance in school districts. The chart below shows the relationship.
Although economic disadvantage proved to be more closely related to student test outcomes at the school district level, the percentage of adults in a district with at least a four-year college degree increased the percentage of variation explained by a small amount – about 2%. Consequently, a linear model that includes both variables was created to understand the combined relationship between economic disadvantage, adult education levels and student performance. Including both variables in the model increased the percentage of variation explained by the model to 74% – nearly three-quarters.
Although the relationship between student performance and economic status and district educational levels is very strong, about one-quarter of the differing performance of school districts is not explained. One factor that could be associated with this unexplained variance is the instructional efficacy of schools. Note, however, that this possible explanation is inferential. Other unexamined factors could be in play as well.
Characteristics of School District Populations
In upstate central cities, high concentrations of disadvantaged students combine with adult populations that have relatively low educational levels. More than 85% of students in Rochester, Syracuse, Schenectady and Utica were economically disadvantaged in recent years. Only about one-quarter of adults were college graduates.
The characteristics of student bodies in large communities surrounding upstate cities are quite varied. In some communities less than 20% of students were economically disadvantaged. In almost all these communities, 50% or more of the students passed the state examination. In a few cases, more than 70% passed. In other communities outside upstate central cities, more than 50% of students were economically disadvantaged. In many cases, less than 40% of students passed the state’s ELA and Mathematics exams. But in only three cases (Lackawanna, Solvay and Watervliet) was the percentage of economically disadvantaged students as high as it was in the central cities. In these communities, as in upstate central cities, more than 70% of students failed the state exams.
As in the case of economic disadvantage, in most cases, cities had lower percentages of people with at least a four-year college degree than did most residents of communities outside them. But, the difference between the educational levels of city residents and residents of other communities was less clear-cut. In fact, about 10% of communities outside central cities had lower percentages of college graduates than did central city residents. And, in general students in school districts in those communities performed better on state tests than central city residents. Overall, the relationship was not as strong as the relationship with economic disadvantage.
Which School Districts Outperformed and Underperformed?
Calls for reform of central city school districts, such as state school district takeovers and the charter school movement, rest on the notion that the poor performance of many students attending urban schools is at least in part caused by poor school management and instruction. How well do students at schools in central city school districts perform given the percentage of economically disadvantaged students and people without college degrees within them?
Overall, economic disadvantage and education are very important predictors of school district performance. But, there were significant differences in district performance after controlling for socioeconomic factors. In Utica, student performance exceeded expected performance by 7% – 24% passing vs 17% predicted. Performance in Buffalo was close to model predictions – 19% passed vs 22% predicted. Schenectady’s performance was also close to model predictions – 17% passed, compared to the prediction of 20%. The Albany school district fell short of expectations by the largest amount. Had Albany students performed as expected, nearly one-third (31%) would have passed. Only 18% did. In Syracuse, 21% were predicted to pass but only 13% did. In Rochester 17% were predicted to pass. Only 9% did.
The next tables show the fifteen districts within the counties within which Albany, Buffalo, Rochester, Syracuse, Schenectady and Utica are located where student performance exceeded expectations and lagged expectations to the greatest degree.
The performance of school districts outside central cities was quite varied. Among the fifteen school districts where student performance was the lowest compared to the model’s prediction, twelve of fifteen were outside central cities. Among those that exceeded expectations, all but one were outside central cities.
Three upstate cities – Albany, Rochester and Syracuse were in the group of poor performers. Among the school districts where students performed the best compared with the model’s prediction, only one – Utica – was a city school district.
Coleman’s finding continues to be true. Socioeconomic factors explain most of the differences in performance between school districts. But even after controlling for these factors, there are differences in performance between school districts. Some are relatively large.
This data is not a causal analysis. We cannot draw conclusions as to the causes of differences in student performance in school districts compared with model predictions. Other unmeasured variables might explain the differences in performance. However, residents of school districts where performance is lower than predicted by the model might question whether their school district is performing its job effectively.
For residents of three large city school districts – Albany, Rochester and Syracuse, this question is particularly important because the percentage of students passing the state exams is lower than in almost all other school districts, and because controlling for known factors associated with poor performance, they did less well than expected.
(a version of this post also appeared in the Rochester Beacon).