Further thoughts on simplicity and complexity in population projection models

Model Complexity and Accuracy: A COVID-19 Case Study

When creating mathematical models for forecasting and decision making, there is a tendency to include more complexity than necessary, in the belief that higher-fidelity models are more accurate than simpler ones. In this paper, we analyze the performance of models that submitted COVID-19 forecasts to the U.S. Centers for Disease Control and Prevention and evaluate them against a simple two-equation model that is specified using simple linear regression. We find that our simple model was comparable in accuracy to highly publicized models and had among the best-calibrated forecasts. This result may be surprising given the complexity of many COVID-19 models and their support by large forecasting teams. However, our result is consistent with the body of research that suggests that simple models perform very well in a variety of settings.

Forecast accuracy hardly improves with method complexity when completing cohort fertility

Information on cohort fertility is critical for the understanding of population dynamics, but only in historical settings can it be calculated without forecasting. Several forecasting methods exist, but their strengths and weaknesses have not been evaluated. Relying on the Human Fertility Database, the largest high-quality fertility dataset to date, and the globally representative United Nations World Population Prospects, we present an assessment of all major methods that complete cohort fertility. This analysis is crucial to advance the understanding of benefits and drawbacks of state-of-the-art methods. We analyze forecast accuracy and uncertainty quantification, identify methodological breakthroughs, and uncover unresolved issues. This study constitutes an evaluation benchmark for cohort fertility forecasting and may inspire establishment of similar evaluation benchmarks in related fields.

Forecasts of completed fertility predict how many children will be born on average by women over their entire reproductive lifetime. These forecasts are important in informing public policy and influencing additional research in the social sciences. However, nothing is known about how to choose a forecasting method from a large basket of variants. We identified 20 major methods, with 162 variants altogether. The approaches range from naive freezing of current age-specific fertility rates to methods that use statistically sophisticated techniques or are grounded in demographic theory. We assess each method by evaluating the overall accuracy and if provided, uncertainty estimates using fertility data of all available birth cohorts and countries of the Human Fertility Database, which covers 1,096 birth cohorts from 29 countries. Across multiple measures of forecast accuracy, we find only four methods that consistently outperform the naive freeze rates method, and only two methods produce uncertainty estimates that are not severely downward biased. Among the top four, there are two simple extrapolation methods and two Bayesian methods. The latter are demanding in terms of input data, statistical techniques, and computational power but do not consistently complete cohort fertility more accurately at all truncation ages than simple extrapolation. This broad picture is unchanged if we base the validation on 201 United Nations countries and six world regions, including Africa, Asia, Europe, Latin America and the Caribbean, northern America, and Oceania.

Population projection accuracy: The impacts of sociodemographics, accessibility, land use, and neighbour characteristics

Population projection is essential to governments, businesses, and research communities for many purposes. Although projection performance is often evaluated, we know very little about what factors affect projection accuracy. It is important to understand these factors in order to utilize the projections knowledgeably. This study fills this gap in the literature by comprehensively investigating the possible factors associated with population projection accuracy in 2010 for the continental US counties. The results indicate that the counties whose populations are more predictable tend to be desirable places-places with abundant employment opportunities, reliable public transportation infrastructure, easy access to work, and/or high land development potential; their neighboring counties tend to have a well-educated population and a higher income level. Also, projection accuracy is highly spatially associated. The findings provide important insights for population projection users to understand the characteristics of counties and their neighboring counties associated with their projection accuracy.

Modelling the effect of urbanization on the transmission of an infectious disease

This paper models the impact of urbanization on infectious disease transmission by integrating a CA land use development model, population projection matrix model and CA epidemic model in S-Plus. The innovative feature of this model lies in both its explicit treatment of spatial land use development, demographic changes, infectious disease transmission and their combination in a dynamic, stochastic model. Heuristically-defined transition rules in cellular automata (CA) were used to capture the processes of both land use development with urban sprawl and infectious disease transmission. A population surface model and dwelling distribution surface were used to bridge the gap between urbanization and infectious disease transmission. A case study is presented involving modelling influenza transmission in Southampton, a dynamically evolving city in the UK. The simulation results for Southampton over a 30-year period show that the pattern of the average number of infection cases per day can depend on land use and demographic changes. The modelling framework presents a useful tool that may be of use in planning applications.