A Caution About the Use of Census Population Estimates
Note – An updated post on this subject may be found here:
New Yorkers were surprised when the Census Bureau announced that instead of gaining 4,271 residents between 2010 and 2020, as reported in the Census Bureau’s Annual Population Estimate, the Decennial Census showed that the state’s population had grown by 823,147. The Annual Estimate was off by 818,876 people. New York was not the only state where the Annual Estimate was significantly off. New Jersey had 398,000 more residents in the Census enumeration than in the annual estimate for 2020, and Pennsylvania had 208,000 more. In one-quarter of the states, the 2020 Annual Population Estimate was off by more than 100,000 residents.
The discrepancies between the Bureau’s 2020 State population estimates and the Decennial Census findings were large. What accounts for the differences?
Errors in the 2020 Estimates
The estimation errors ranged from an overestimate of Arizona’s population change of 3.4% to an underestimate of New Jersey’s change by 4.3% – a spread of 7.7%. In New Jersey, New York, Michigan, Vermont, Connecticut, Mississippi, and Pennsylvania, the percentage error was larger than the estimated percentage population change.
The Bureau’s model produced statistically accurate estimates of population change from 2010 to 2020, with 93% of the variation in actual percentage population change associated with estimated population change. But the estimation errors are large enough in some cases to result in significant differences. In New York’s case, the state’s rank in population change went from 46th to 7th.
The chart above shows the actual percentage change in population from the decennial census between 2010 and 2020 for each state on the vertical axis. The change presented in the Annual Estimate for 2020 is on the horizontal axis. The blue dots show deviations from a one-to-one relationship shown on the dotted trend line.
The errors in estimating population change in the Annual Population Estimates were substantial in many cases. In eight states, the error size was larger than the estimated change. In five other states, the error was more than half the size of the estimate, and in nine other states, the error was larger than one-fifth the size of the estimated change. In only nine states was the estimated population change within five percent of the actual change.
The Bureau’s Methodology
The COVID pandemic created major challenges for the Census Bureau in 2020 that affected the conduct of the Decennial Census and the Annual Estimate program. The Bureau’s field operations were limited by the imposition of stay-at-home orders beginning in March. Follow-up operations were delayed and did not complete operations until October. The Bureau’s American Community Survey program was impacted so severely by the Pandemic that the Bureau did not release official 2020 data because of unrepresentativeness in the sample.
The Bureau checks the quality of the Census enumeration with the Post Enumeration Survey, which is not yet complete – reports will be released this quarter. Some data has already been released, showing that the 2020 Census response metrics were slightly better than those in 2010, mainly because the Bureau posted questionnaires online for the first time in 2020.
The Census Bureau begins its annual estimation process by taking the estimated population for each state from the Annual Population Estimate in the past year. In the year following the Decennial Census, the Bureau uses the Census data for the prior year. The Bureau then incorporates estimates of births, deaths, and migration to arrive at a population estimate for the current year.
Critical steps involve using data that captures only a portion of the population to be measured. For example, federal income tax data is used to estimate domestic migration in the working-age population. But, not all working-age adults file returns. Consequently, the Bureau scales the findings of the return data to reflect the estimated total working-age population. Although federal tax data captures a high percentage of the total population, the characteristics of working-age non-filers could differ from those who aren’t sampled, creating a possible source of error.
The Bureau relies on Medicare recipient data for population estimates of those over 65, a portion of the population it intends to measure. Other data relies on information from prior years that the Bureau trends forward to the current year. For example, birth and death data used by the Bureau has a two-year lag. The Pandemic created new challenges for trending data, affecting prior patterns of births and deaths. Although the Bureau included some provisional data from 2020 to update the estimates, the interim data is of lower quality than final reports because of time lags in reporting.
The estimates are chained. Each year, the new assessment builds on the preceding year’s data with adjustments for births, deaths, and migrations. The process builds prior errors into the latest estimate. The release of new Census data every ten years corrects the errors that develop over the preceding decade.
The Census Bureau’s Annual Population Estimates are closely watched as indicators of population changes at the national, state, and local levels. The Bureau states that “These estimates are used in federal funding allocations, as survey controls, as denominators for vital rates and per capita time series, and as indicators of recent demographic changes.”
The Bureau’s method is a reasonable way to estimate national, state, and local populations without a complete population enumeration. But, errors are possible because the data incorporates assumptions about the relationships between the samples used and the population and the use of trended data from prior years.
The Bureau presents the estimated data without adequate cautions concerning its accuracy. The Annual State Population estimates in 2020 had significant errors. When the Bureau offers an estimate for the state’s population, like its estimate that New York’s 2020 population was 19,382,373, the specificity suggests a degree of accuracy that is not present. The actual number was almost 819,000 larger. The change in population shown in the Annual Estimate diverged from the Decennial Census findings by more than 20% in nearly half the states. In only nine of fifty states was the estimated change from 2010 to 2020 within 5% of the change found in the Census population enumeration.
Although the Census Bureau uses a straightforward methodology to estimate population changes in the Annual State Population Estimates, the available data is not sufficiently robust to produce accurate estimates. The Census Bureau should provide clear statements showing the historical variability of State population estimates compared to the actual Census data and should warn users about the magnitude of potential errors in their use. Because the Estimates are used in federal funding allocations, the Bureau should consider recommending to Congress and Executive Agencies that they reduce reliance on them and that the more accurate Decennial Census be used instead.