Multilevel regression and poststratification as a modeling approach for estimating population quantities in large population health studies: A simulation study

Improving the representativeness of UK's national COVID-19 Infection Survey through spatio-temporal regression and post-stratification

Population-representative estimates of SARS-CoV-2 infection prevalence and antibody levels in specific geographic areas at different time points are needed to optimise policy responses. However, even population-wide surveys are potentially impacted by biases arising from differences in participation rates across key groups. Here, we used spatio-temporal regression and post-stratification models to UK's national COVID-19 Infection Survey (CIS) to obtain representative estimates of PCR positivity (6,496,052 tests) and antibody prevalence (1,941,333 tests) for different regions, ages and ethnicities (7-December-2020 to 4-May-2022). Not accounting for vaccination status through post-stratification led to small underestimation of PCR positivity, but more substantial overestimations of antibody levels in the population (up to 21 percentage points), particularly in groups with low vaccine uptake in the general population. There was marked variation in the relative contribution of different areas and age-groups to each wave. Future analyses of infectious disease surveys should take into account major drivers of outcomes of interest that may also influence participation, with vaccination being an important factor to consider.

Fixed Effects or Mixed Effects Classifiers? Evidence From Simulated and Archival Data

This study seeks to compare fixed and mixed effects models for the purposes of predictive classification in the presence of multilevel data. The first part of the study utilizes a Monte Carlo simulation to compare fixed and mixed effects logistic regression and random forests. An applied examination of the prediction of student retention in the public-use U.S. PISA data set was considered to verify the simulation findings. Results of this study indicate fixed effects models performed comparably with mixed effects models across both the simulation and PISA examinations. Results broadly suggest that researchers should be cognizant of the type of predictors and data structure being used, as these factors carried more weight than did the model type.

How do we decide how representative our donors are for public health surveillance?

BACKGROUND

Surveillance of blood donors is fundamental to safety of the blood supply. Such data can also be useful for public health policy but tend to be underutilized. When the COVID-19 pandemic arrived, blood centers around the world measured blood donor SARS-CoV-2 seroprevalence to inform public health policy. There is now a movement toward blood centers becoming more involved in public health research and surveillance post-pandemic. However, blood donors are a healthy population and not representative of all segments of the general population. In this article, we explain how blood centers can evaluate their donor base to understand which part of the general population they are representative of.

STUDY DESIGN AND METHODS

Methodologic approaches for evaluating samples relative to the target population were reviewed. Blood donor data that are available to most blood centers were identified and application to assess representativeness of blood donors was evaluated.

RESULTS

Key aspects of blood donor data to compare with general population data include donor selection criteria, health indicators, geography, and demographics. In some cases, statistical adjustment can improve representativeness.

DISCUSSION

Comparing key blood donor data with corresponding general population data can define the subset of the general population for which a particular blood center's donors may be representative of. We suggest that donors are an ideal convenience population for surveillance of infectious agents which are frequently asymptomatic and main routes of transmission are not deferrable, for studying the natural history of disease in an initially well population, and for vaccination serology surveillance.

US county-level estimation for maternal and infant health-related behavior indicators using pregnancy risk assessment monitoring system data, 2016-2018

Background

There is a critical need for maternal and child health data at the local level (for example, county), yet most counties lack sustainable resources or capabilities to collect local-level data. In such case, model-based small area estimation (SAE) could be a feasible approach. SAE for maternal or infant health-related behaviors at small areas has never been conducted or evaluated.

Methods

We applied multilevel regression with post-stratification approach to produce county-level estimates using Pregnancy Risk Assessment Monitoring System (PRAMS) data, 2016–2018 (n = 65,803 from 23 states) for 2 key outcomes, breastfeeding at 8 weeks and infant non-supine sleeping position.

Results

Among the 1,471 counties, the median model estimate of breastfeeding at 8 weeks was 59.8% (ranged from 34.9 to 87.4%), and the median of infant non-supine sleeping position was 16.6% (ranged from 10.3 to 39.0%). Strong correlations were found between model estimates and direct estimates for both indicators at the state level. Model estimates for both indicators were close to direct estimates in magnitude for Philadelphia County, Pennsylvania.

Conclusion

Our findings support this approach being potentially applied to other maternal and infant health and behavioral indicators in PRAMS to facilitate public health decision-making at the local level.

Association of vaccine hesitancy and immunization coverage rates in the European Union

While previous studies have validated vaccine hesitancy scales with uptake behavior at the individual level, the conditions under which aggregated survey data are useful are less clear. We show that vaccine public opinion data aggregated at the subnational level can serve as a valid indicator of aggregate vaccine behaviour. We use a public opinion survey (Eurobarometer EB 91.2) with data on vaccine hesitancy for the EU in 2019. We link this information to (subnational) regional immunization coverage rates for childhood vaccines - DTP3, MCV1, and MCV2 -- obtained from the WHO for 2019. We conduct multilevel regression analyses with data for 177 regions in 20 countries. Given the variation in vaccine hesitancy and immunization rates between countries and within countries, we affirm the valuable role that surveys can play as a public health surveillance tool when it comes to vaccine behavior. We find statistically significantly lower regional vaccine immunization rates in regions where vaccine hesitancy is more pronounced. Our results suggest that different uptake rates across subnational regions are due, at least in part, to differences in attitudes towards vaccines and vaccination. The results are robust to several alternative specifications.

Community prevalence of SARS-CoV-2 in England from April to November, 2020: results from the ONS Coronavirus Infection Survey

BACKGROUND

Decisions about the continued need for control measures to contain the spread of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) rely on accurate and up-to-date information about the number of people testing positive for SARS-CoV-2 and risk factors for testing positive. Existing surveillance systems are generally not based on population samples and are not longitudinal in design.

METHODS

Samples were collected from individuals aged 2 years and older living in private households in England that were randomly selected from address lists and previous Office for National Statistics surveys in repeated cross-sectional household surveys with additional serial sampling and longitudinal follow-up. Participants completed a questionnaire and did nose and throat self-swabs. The percentage of individuals testing positive for SARS-CoV-2 RNA was estimated over time by use of dynamic multilevel regression and poststratification, to account for potential residual non-representativeness. Potential changes in risk factors for testing positive over time were also assessed. The study is registered with the ISRCTN Registry, ISRCTN21086382.

FINDINGS

Between April 26 and Nov 1, 2020, results were available from 1 191 170 samples from 280 327 individuals; 5231 samples were positive overall, from 3923 individuals. The percentage of people testing positive for SARS-CoV-2 changed substantially over time, with an initial decrease between April 26 and June 28, 2020, from 0·40% (95% credible interval 0·29-0·54) to 0·06% (0·04-0·07), followed by low levels during July and August, 2020, before substantial increases at the end of August, 2020, with percentages testing positive above 1% from the end of October, 2020. Having a patient-facing role and working outside your home were important risk factors for testing positive for SARS-CoV-2 at the end of the first wave (April 26 to June 28, 2020), but not in the second wave (from the end of August to Nov 1, 2020). Age (young adults, particularly those aged 17-24 years) was an important initial driver of increased positivity rates in the second wave. For example, the estimated percentage of individuals testing positive was more than six times higher in those aged 17-24 years than in those aged 70 years or older at the end of September, 2020. A substantial proportion of infections were in individuals not reporting symptoms around their positive test (45-68%, dependent on calendar time.

INTERPRETATION

Important risk factors for testing positive for SARS-CoV-2 varied substantially between the part of the first wave that was captured by the study (April to June, 2020) and the first part of the second wave of increased positivity rates (end of August to Nov 1, 2020), and a substantial proportion of infections were in individuals not reporting symptoms, indicating that continued monitoring for SARS-CoV-2 in the community will be important for managing the COVID-19 pandemic moving forwards.

FUNDING

Department of Health and Social Care.

Multilevel regression and poststratification for estimating population quantities from large health studies: a simulation study based on US population structure

INTRODUCTION

Recruiting a representative sample of participants is becoming increasingly difficult in large-scale health surveys. Multilevel regression and poststratification (MRP) has been shown to be effective in estimating population descriptive quantities in non-representative samples. We performed a simulation study, previously applied to an Australian population, this time to a US population, to assess MRP performance.

METHODS

Data were extracted from the 2017 Current Population Survey representing a population of US adult males aged 18-55 years. Simulated datasets of non-representative samples were generated. State-level prevalence estimates for a dichotomous outcome using MRP were compared with the use of sampling weights (with and without raking adjustment). We also investigated the impact on MRP performance of sample size, model misspecification, interactions and the addition of a geographic-level covariate.

RESULTS

MRP was found to achieve generally superior performance, with large gains in precision vastly outweighing the increased accuracy observed for sampling weights with raking adjustment. MRP estimates were generally robust to model misspecification. We found a tendency of MRP to over-pool between-state variation in the outcome, particularly for the least populous states and small sample sizes. The inclusion of a state-level covariate appeared to mitigate this and further improve MRP performance.

DISCUSSION

MRP has been shown to be effective in estimating population descriptive quantities in two different populations. This provides promising evidence for the general applicability of MRP to populations with different geographic structures. MRP appears to be a valuable analytic strategy for addressing potential participation bias from large-scale health surveys.

Multilevel Regression and Poststratification Versus Survey Sample Weighting for Estimating Population Quantities in Large Population Health Studies

Multilevel regression and poststratification (MRP) is a model-based approach for estimating a population parameter of interest, generally from large-scale surveys. It has been shown to be effective in highly selected samples, which is particularly relevant to investigators of large-scale population health and epidemiologic surveys facing increasing difficulties in recruiting representative samples of participants. We aimed to further examine the accuracy and precision of MRP in a context where census data provided reasonable proxies for true population quantities of interest. We considered 2 outcomes from the baseline wave of the Ten to Men study (Australia, 2013-2014) and obtained relevant population data from the 2011 Australian Census. MRP was found to achieve generally superior performance relative to conventional survey weighting methods for the population as a whole and for population subsets of varying sizes. MRP resulted in less variability among estimates across population subsets relative to sample weighting, and there was some evidence of small gains in precision when using MRP, particularly for smaller population subsets. These findings offer further support for MRP as a promising analytical approach for addressing participation bias in the estimation of population descriptive quantities from large-scale health surveys and cohort studies.