An evaluation of population projection errors for census tracts

Methods for Small Area Population Forecasts: State-of-the-Art and Research Needs

Small area population forecasts are widely used by government and business for a variety of planning, research and policy purposes, and often influence major investment decisions. Yet, the toolbox of small area population forecasting methods and techniques is modest relative to that for national and large subnational regional forecasting. In this paper, we assess the current state of small area population forecasting, and suggest areas for further research. The paper provides a review of the literature on small area population forecasting methods published over the period 2001–2020. The key themes covered by the review are extrapolative and comparative methods, simplified cohort-component methods, model averaging and combining, incorporating socioeconomic variables and spatial relationships, ‘downscaling’ and disaggregation approaches, linking population with housing, estimating and projecting small area component input data, microsimulation, machine learning, and forecast uncertainty. Several avenues for further research are then suggested, including more work on model averaging and combining, developing new forecasting methods for situations which current models cannot handle, quantifying uncertainty, exploring methodologies such as machine learning and spatial statistics, creating user-friendly tools for practitioners, and understanding more about how forecasts are used.

Using Synthetic Adjustments and Controlling to Improve County Population Forecasts from the Hamilton–Perry Method

Tayman and Swanson (J Popul Res 34(3):209–231, 2017) found in Washington State counties that a forecast based on the Hamilton–Perry method using a synthetic adjustment (SYN) of cohort change ratios and child-woman ratios had greater accuracy and less bias compared to forecasts holding these ratios constant (CONST). In this paper, we assess the robustness of SYN’s efficacy by evaluating forecast accuracy, bias, and distributional error across age groups in counties nationwide. We also investigate whether forecast errors and their patterns change for SYN and CONST if forecasts by age and gender are adjusted to an independent total population forecast for each county. Our main findings are as follows: (1) SYN lowers forecast error compared to CONST whether the forecasts are controlled or not; (2) controlling also leads to the improvements in forecast error, often exceeding those in SYN; and (3) using SYN and controlling together has the greatest effect in reducing forecast error. These findings remain after controlling for population size and growth rate, but the positive impacts on forecast error of SYN and controlling are most evident in counties with less than 30,000 population and that grow by 15% or more.

Synergy in fertility forecasting: improving forecast accuracy through model averaging

Accuracy in fertility forecasting has proved challenging and warrants renewed attention. One way to improve accuracy is to combine the strengths of a set of existing models through model averaging. The model-averaged forecast is derived using empirical model weights that optimise forecast accuracy at each forecast horizon based on historical data. We apply model averaging to fertility forecasting for the first time, using data for 17 countries and six models. Four model-averaging methods are compared: frequentist, Bayesian, model confidence set, and equal weights. We compute individual-model and model-averaged point and interval forecasts at horizons of one to 20 years. We demonstrate gains in average accuracy of 4–23% for point forecasts and 3–24% for interval forecasts, with greater gains from the frequentist and equal weights approaches at longer horizons. Data for England and Wales are used to illustrate model averaging in forecasting age-specific fertility to 2036. The advantages and further potential of model averaging for fertility forecasting are discussed. As the accuracy of model-averaged forecasts depends on the accuracy of the individual models, there is ongoing need to develop better models of fertility for use in forecasting and model averaging. We conclude that model averaging holds considerable promise for the improvement of fertility forecasting in a systematic way using existing models and warrants further investigation.

The spatio-temporal relationship between alcohol outlets and violence before and after privatization: A natural experiment, Seattle, Wa 2010-2013

Alcohol-related violence is a well-documented public health concern, where various individual and community-level factors contribute to this relationship. The purpose of this study is to examine the impact of a significant policy change at the local level, which privatized liquor sales and distribution. Specifically, we explored the relationship between alcohol and violence in Seattle, WA, 2010-2013, via hierarchical spatio-temporal disease mapping models. To measure and map this complex spatio-temporal relationship at the census block group level (n=567), we examined a variety of models using integrated nested Laplace approximations and used the deviance information criterion to gauge model complexity and fit. For each additional off-premises and on-premises alcohol outlet in a given census block group, we found a significant increase of 8% and 5% for aggravated assaults and 6% and 5% for non-aggravated assaults, respectively. Lastly, our maps showed variation in the estimated relative risks across the city of Seattle.

An evaluation of the accuracy of small-area demographic estimates of population at risk and its effect on prevalence statistics

Demographic estimates of population at risk often underpin epidemiologic research and public health surveillance efforts. In spite of their central importance to epidemiology and public-health practice, little previous attention has been paid to evaluating the magnitude of errors associated with such estimates or the sensitivity of epidemiologic statistics to these effects. In spite of the well-known observation that accuracy in demographic estimates declines as the size of the population to be estimated decreases, demographers continue to face pressure to produce estimates for increasingly fine-grained population characteristics at ever-smaller geographic scales. Unfortunately, little guidance on the magnitude of errors that can be expected in such estimates is currently available in the literature and available for consideration in small-area epidemiology. This paper attempts to fill this current gap by producing a Vintage 2010 set of single-year-of-age estimates for census tracts, then evaluating their accuracy and precision in light of the results of the 2010 Census. These estimates are produced and evaluated for 499 census tracts in New Mexico for single-years of age from 0 to 21 and for each sex individually. The error distributions associated with these estimates are characterized statistically using non-parametric statistics including the median and 2.5th and 97.5th percentiles. The impact of these errors are considered through simulations in which observed and estimated 2010 population counts are used as alternative denominators and simulated event counts are used to compute a realistic range fo prevalence values. The implications of the results of this study for small-area epidemiologic research in cancer and environmental health are considered.

Forecast Accuracy and Uncertainty of Australian Bureau of Statistics State and Territory Population Projections

Errors from past rounds of population projections can provide both diagnostic information to improve future projections as well as information for users on the likely uncertainty of current projections. This paper assesses the forecast accuracy of official Australian Bureau of Statistics (ABS) population projections for the states and territories of Australia and is the first major study to do so. For the states and territories, it is found that, after 10-year projection durations, absolute percentage errors lie between about 1% and 3% for the states and around 6% for the territories. Age-specific population projections are also assessed. It is shown that net interstate migration and net overseas migration are the demographic components of change which contributed most to forecast error. The paper also compares ABS projections of total population against simple linear extrapolation, finding that, overall, ABS projections just outperformed extrapolation. No identifiable trend in accuracy over time is detected. Under the assumption of temporal stability in the magnitude of error, empirical prediction intervals are created from past errors and applied to the current set of ABS projections. The paper concludes with a few ideas for future projection rounds.

Evaluating Population Forecast Accuracy: A Regression Approach Using County Data

Many studies have evaluated the impact of differences in population size and growth rate on population forecast accuracy. Virtually all these studies have been based on aggregate data; that is, they focused on average errors for places with particular size or growth rate characteristics. In this study, we take a different approach by investigating forecast accuracy using regression models based on data for individual places. Using decennial census data from 1900 to 2000 for 2,482 counties in the US, we construct a large number of county population forecasts and calculate forecast errors for 10- and 20-year horizons. Then, we develop and evaluate several alternative functional forms of regression models relating population size and growth rate to forecast accuracy; investigate the impact of adding several other explanatory variables; and estimate the relative contributions of each variable to the discriminatory power of the models. Our results confirm several findings reported in previous studies but uncover several new findings as well. We believe regression models based on data for individual places provide powerful but under-utilized tools for investigating the determinants of population forecast accuracy.

Childhood lymphohematopoietic cancer incidence and hazardous air pollutants in southeast Texas, 1995-2004

BACKGROUND

Cancer is the second leading cause of death among U.S. children with few known risk factors. There is increasing interest in the role of air pollutants, including benzene and 1,3-butadiene, in the etiology of childhood cancers.

OBJECTIVE

Our goal was to assess whether census tracts with the highest benzene or 1,3-butadiene ambient air levels have increased childhood lymphohematopoietic cancer incidence.

METHODS

Our ecologic analysis included 977 cases of childhood lymphohematopoietic cancer diagnosed from 1995-2004. We obtained the U.S. Environmental Protection Agency's 1999 modeled estimates of benzene and 1,3-butadiene for 886 census tracts surrounding Houston, Texas. We ran Poisson regression models by pollutant to explore the associations between pollutant levels and census-tract cancer rates. We adjusted models for age, sex, race/ethnicity, and community-level socioeconomic status (cSES).

RESULTS

Census tracts with the highest benzene levels had elevated rates of all leukemia [rate ratio (RR) = 1.37; 95% confidence interval (CI), 1.05, 1.78]. This association was higher for acute myeloid leukemia (AML) (RR = 2.02; 95% CI, 1.03-3.96) than for acute lymphocytic leukemia (ALL) (RR = 1.24; 95% CI, 0.92-1.66). Among census tracts with the highest 1,3-butadiene levels, we observed RRs of 1.40 (95% CI, 1.07-1.81), 1.68 (95% CI, 0.84-3.35), and 1.32 (95% CI, 0.98-1.77) for all leukemia, AML, and ALL, respectively. We detected no associations between benzene or 1,3-butadiene levels and lymphoma incidence. Results that examined joint exposure to benzene and 1,3-butadiene were similar to those that examined each pollutant separately.

CONCLUSIONS

Our ecologic analysis suggests an association between childhood leukemia and hazardous air pollution; further research using more sophisticated methodology is warranted.

An evaluation of population projections by age

A number of studies have evaluated the accuracy of projections of the size of the total population, but few have considered the accuracy of projections by age group. For many purposes, however, the relevant variable is the population of a particular age group, rather than the population as a whole. We investigated the precision and bias of a variety of age-group projections at the national and state levels in the United States and for counties in Florida. We also compared the accuracy of state and county projections that were derived from full-blown applications of the cohort-component method with the accuracy of projections that were derived from a simpler, less data-intensive version of the method. We found that age-group error patterns are different for national projections than for subnational projections; that errors are substantially larger for some age groups than for others; that differences in errors among age groups decline as the projection horizon becomes longer; and that differences in methodological complexity have no consistent impact on the precision and bias of age-group projections.

On the utility of population forecasts

Many customers demand population forecasts, particularly for small areas. Although the forecast evaluation literature is extensive, it is dominated by a focus on accuracy. We go beyond accuracy by examining the concept of forecast utility in an evaluation of a sample of 2,709 counties and census tracts. Wefind that forecasters provide “value-added” knowledge for areas experiencing rapid change or areas with relatively large populations. For other areas, reduced value is more common than added value. Our results suggest that new forecasting strategies and methods such as composite modeling may substantially improve forecast utility.