Integrating Intensive Longitudinal Data (ILD) to Inform the Development of Dynamic Theories of Behavior Change and Intervention Design: a Case Study of Scientific and Practical Considerations

The increasing sophistication of mobile and sensing technology has enabled the collection of intensive longitudinal data (ILD) concerning dynamic changes in an individual's state and context. ILD can be used to develop dynamic theories of behavior change which, in turn, can be used to provide a conceptual framework for the development of just-in-time adaptive interventions (JITAIs) that leverage advances in mobile and sensing technology to determine when and how to intervene. As such, JITAIs hold tremendous potential in addressing major public health concerns such as cigarette smoking, which can recur and arise unexpectedly. In tandem, a growing number of studies have utilized multiple methods to collect data on a particular dynamic construct of interest from the same individual. This approach holds promise in providing investigators with a significantly more detailed view of how a behavior change processes unfold within the same individual than ever before. However, nuanced challenges relating to coarse data, noisy data, and incoherence among data sources are introduced. In this manuscript, we use a mobile health (mHealth) study on smokers motivated to quit (Break Free; R01MD010362) to illustrate these challenges. Practical approaches to integrate multiple data sources are discussed within the greater scientific context of developing dynamic theories of behavior change and JITAIs.

Estimating the Prevalence of Substance Use Disorders in the US Using the Benchmark Multiplier Method

Importance

Prevalence estimates of substance use disorders in the US rely on general population surveys. However, major population groups, such as homeless individuals and institutionalized individuals, are not captured by these surveys, and participants may underreport substance use.

Objective

To estimate the prevalence of substance use disorders in the US.

Design, Setting, and Participants

The benchmark multiplier method was used to estimate the prevalence of alcohol, cannabis, opioid, and stimulant use disorders based on data from the Transformed Medicaid Statistical Information System (T-MSIS) (the benchmark) and the National Survey on Drug Use and Health (NSDUH) (the multiplier) for 2018 and 2019. T-MSIS collects administrative data on Medicaid beneficiaries 12 years and older with full or comprehensive benefits. NSDUH is a nationally representative annual cross-sectional survey of people 12 years and older. Data were analyzed from February to June 2022.

Main Outcomes and Measures

Prevalence of substance use disorders was estimated using the benchmark multiplier method based on T-MSIS and NSDUH data. Confidence intervals for the multiplier method estimates were computed using Monte Carlo simulations. Sensitivity of prevalence estimates to variations in multiplier values was assessed.

Results

This study included Medicaid beneficiaries 12 years and older accessing treatment services in the past year with diagnoses of alcohol (n = 1 017 308 in 2018; n = 1 041 357 in 2019), cannabis (n = 643 737; n = 644 780), opioid (n = 1 406 455; n = 1 575 219), and stimulant (n = 610 858; n = 657 305) use disorders and NSDUH participants with 12-month DSM-IV alcohol (n = 3390 in 2018; n = 3363 in 2019), cannabis (n = 1426; n = 1604), opioid (n = 448; n = 369), and stimulant (n = 545; n = 559) use disorders. The benchmark multiplier prevalence estimates were higher than NSDUH estimates for every type of substance use disorder in both years and in the combined 2018 to 2019 sample: 20.27% (95% CI, 17.04-24.71) vs 5.34% (95% CI, 5.10-5.58), respectively, for alcohol; 7.57% (95% CI, 5.96-9.93) vs 1.68% (95% CI, 1.59-1.79) for cannabis; 3.46% (95% CI, 2.97-4.12) vs 0.68% (0.60-0.78) for opioid; and 1.91% (95% CI, 1.63-2.30) vs 0.85% (95% CI, 0.75-0.96) for stimulant use disorders. In sensitivity analyses, the differences between the benchmark multiplier method and NSDUH estimates persisted over a wide range of potential multiplier values.

Conclusions and Relevance

The findings in this study reflect a higher national prevalence of substance use disorders than that represented by NSDUH estimates, suggesting a greater burden of these conditions in the US.

Bayesian Bootstrap in Multiple Frames

Multiple frames are becoming increasingly relevant due to the spread of surveys conducted via registers. In this regard, estimators of population quantities have been proposed, including the multiplicity estimator. In all cases, variance estimation still remains a matter of debate. This paper explores the potential of Bayesian bootstrap techniques for computing such estimators. The suitability of the method, which is compared to the existing frequentist bootstrap, is shown by conducting a small-scale simulation study and a case study.

Nonparametric Mass Imputation for Data Integration

Data integration combining a probability sample with another nonprobability sample is an emerging area of research in survey sampling. We consider the case when the study variable of interest is measured only in the nonprobability sample, but comparable auxiliary information is available for both data sources. We consider mass imputation for the probability sample using the nonprobability data as the training set for imputation. The parametric mass imputation is sensitive to parametric model assumptions. To develop improved and robust methods, we consider nonparametric mass imputation for data integration. In particular, we consider kernel smoothing for a low-dimensional covariate and generalized additive models for a relatively high-dimensional covariate for imputation. Asymptotic theories and variance estimation are developed. Simulation studies and real applications show the benefits of our proposed methods over parametric counterparts.

National Health Interview Survey, COVID-19, and Online Data Collection Platforms: Adaptations, Tradeoffs, and New Directions

High-quality data are accurate, relevant, and timely. Large national health surveys have always balanced the implementation of these quality dimensions to meet the needs of diverse users. The COVID-19 pandemic shifted these balances, with both disrupted survey operations and a critical need for relevant and timely health data for decision-making. The National Health Interview Survey (NHIS) responded to these challenges with several operational changes to continue production in 2020. However, data files from the 2020 NHIS were not expected to be publicly available until fall 2021. To fill the gap, the National Center for Health Statistics (NCHS) turned to 2 online data collection platforms-the Census Bureau's Household Pulse Survey (HPS) and the NCHS Research and Development Survey (RANDS)-to collect COVID-19‒related data more quickly. This article describes the adaptations of NHIS and the use of HPS and RANDS during the pandemic in the context of the recently released Framework for Data Quality from the Federal Committee on Statistical Methodology. (Am J Public Health. 2021;111(12):2167-2175. https://doi.org/10.2105/AJPH.2021.306516).

Estimating perceived parental substance use disorder: Using register data to adjust for non-participation in survey research

AIMS

To estimate the prevalence of parental substance use disorder (PSUD) in the general population based on young adults' reports adjusted for non-participation using register-based indicators of PSUD.

DESIGN

A national sample survey study combined with a retrospective register-based study. Setting Denmark. Participants 10,414 young people (aged 15-25 years) invited to two national sample surveys in 2014 and 2015 (5,755 participants and 4,659 non-participants).

MEASUREMENTS

A crude prevalence of PSUD was calculated based on participants' reports. Parental data from medical, mortality, prescription, and treatment registers (from the young adults' birth until the time of the surveys) were used to estimate a register-based prevalence of PSUD for both participants and non-participants. Differences between participants and non-participants were analysed using bivariate comparisons. Inverse probability weighting was used to adjust for bias due to non-participation. The crude prevalence of PSUD based on survey data was adjusted using the ratio of incidence proportion of the register-based PSUD compared with the survey-based PSUD.

FINDINGS

A total of 731 (12.7%) of the 5,755 survey participants reported PSUD. Register-based PSUD was more common among non-participants (856/4,659; 18.4%) compared with participants (738/5,755; 12.8%, OR = 1.53, 95% CI 1.38-1.70). The adjusted estimate of the survey-based PSUD increased by 2.5 percentage points, from 12.7% to 15.2%.

CONCLUSIONS

In the absence of register data, youth-reported PSUD is likely to underestimate the number of young people experiencing PSUD.

FIT FOR PURPOSE IN ACTION: DESIGN, IMPLEMENTATION, AND EVALUATION OF THE NATIONAL INTERNET FLU SURVEY

Researchers strive to design and implement high-quality surveys to maximize the utility of the data collected. The definitions of quality and usefulness, however, vary from survey to survey and depend on the analytic needs. Survey teams must evaluate the trade-offs of various decisions, such as when results are needed and their required level of precision, in addition to practical constraints like budget, before finalizing the design. Characteristics within the concept of fit for purpose (FfP) can provide the framework for considering the trade-offs. Furthermore, this tool can enable an evaluation of quality for the resulting estimates. Implementation of a FfP framework in this context, however, is not straightforward. In this article, we provide the reader with a glimpse of a FfP framework in action for obtaining estimates on early season influenza vaccination coverage estimates and on knowledge, attitudes, behaviors, and barriers related to influenza and influenza prevention among civilian noninstitutionalized adults aged 18 years and older in the United States. The result is the National Internet Flu Survey (NIFS), an annual, two-week internet survey sponsored by the US Centers for Disease Control and Prevention. In addition to critical design decisions, we use the established NIFS FfP framework to discuss the quality of the NIFS in meeting the intended objectives. We highlight aspects that work well and other survey traits requiring further evaluation. Differences found in comparing the NIFS to the National Flu Survey, the National Health Interview Survey, and Behavioral Risk Factor Surveillance System are discussed via their respective FfP characteristics. The findings presented here highlight the importance of the FfP framework for designing surveys, defining data quality, and providing a set a metrics used to advertise the intended use of the survey data and results.

Life in Data Sets: Locating and Accessing Data on the Health of Americans Across the Life Span

CONTEXT

The US government manages a large number of data sets, including federally funded data collection activities that examine infectious and chronic conditions, as well as risk and protective factors for adverse health outcomes. Although there currently is no mature, comprehensive metadata repository of existing data sets, US federal agencies are working to develop and make metadata repositories available that will improve discoverability. However, because these repositories are not yet operating at full capacity, researchers must rely on their own knowledge of the field to identify available data sets.

PROGRAM OR POLICY

We sought to identify and consolidate a practical and annotated listing of those data sets.

IMPLEMENTATION AND/OR DISSEMINATION

Creative use of data resources to address novel questions is an important research skill in a wide range of fields including public health. This report identifies, promotes, and encourages the use of a range of data sources for health, behavior, economic, and policy research efforts across the life span.

EVALUATION

We identified and organized 28 federal data sets by the age-group of primary focus; not all groups are mutually exclusive. These data sets collectively represent a rich source of information that can be used to conduct descriptive epidemiologic studies.

DISCUSSION

The data sets identified in this article are not intended to represent an exhaustive list of all available data sets. Rather, we present an introduction/overview of the current federal data collection landscape and some of its largest and most frequently utilized data sets.

Combining self-reported and objectively measured survey data to improve hypertension prevalence estimates: Portuguese experience

BACKGROUND

Accurate data on hypertension is essential to inform decision-making. Hypertension prevalence may be underestimated by population-based surveys due to misclassification of health status by participants. Therefore, adjustment for misclassification bias is required when relying on self-reports. This study aims to quantify misclassification bias in self-reported hypertension prevalence and prevalence ratios in the Portuguese component of the European Health Interview Survey (INS2014), and illustrate application of multiple imputation (MIME) for bias correction using measured high blood pressure data from the first Portuguese health examination survey (INSEF).

METHODS

We assumed that objectively measured hypertension status was missing for INS2014 participants (n = 13,937) and imputed it using INSEF (n = 4910) as auxiliary data. Self-reported, objectively measured and MIME-corrected hypertension prevalence and prevalence ratios (PR) by sex, age group and education were estimated. Bias in self-reported and MIME-corrected estimates were computed using objectively measured INSEF data as a gold-standard.

RESULTS

Self-reported INS2014 data underestimated hypertension prevalence in all population subgroups, with misclassification bias ranging from 5.2 to 18.6 percentage points (pp). After MIME-correction, prevalence estimates increased and became closer to objectively measured ones, with bias reduction to 0 pp - 5.7 pp. Compared to objectively measured INSEF, self-reported INS2014 data considerably underestimated prevalence ratio by sex (PR = 0.8, 95CI = [0.7, 0.9] vs. PR = 1.2, 95CI = [1.1, 1.4]). MIME successfully corrected direction of association with sex in bivariate (PR = 1.1, 95CI = [1.0, 1.3]) and multivariate analyses (PR = 1.2, 95CI = [1.0, 1.3]). Misclassification bias in hypertension prevalence ratios by education and age group were less pronounced and did not require correction in multivariate analyses.

CONCLUSIONS

Our results highlight the importance of misclassification bias analysis in self-reported hypertension. Multiple imputation is a feasible approach to adjust for misclassification bias in prevalence estimates and exposure-outcomes associations in survey data.

Overview and Initial Results of the National Center for Health Statistics' Research and Development Survey

The National Center for Health Statistics is assessing the usefulness of recruited web panels in multiple research areas. One research area examines the use of close-ended probe questions and split-panel experiments for evaluating question-response patterns. Another research area is the development of statistical methodology to leverage the strength of national survey data to evaluate, and possibly improve, health estimates from recruited panels. Recruited web panels, with their lower cost and faster production cycle, in combination with established population health surveys, may be useful for some purposes for statistical agencies. Our initial results indicate that web survey data from a recruited panel can be used for question evaluation studies without affecting other survey content. However, the success of these data to provide estimates that align with those from large national surveys will depend on many factors, including further understanding of design features of the recruited panel (e.g. coverage and mode effects), the statistical methods and covariates used to obtain the original and adjusted weights, and the health outcomes of interest.

COMBINING INFORMATION FROM MULTIPLE DATA SOURCES TO ASSESS POPULATION HEALTH

Information about an extensive set of health conditions on a well-defined sample of subjects is essential for assessing population health, gauging the impact of various policies, modeling costs, and studying health disparities. Unfortunately, there is no single data source that provides accurate information about health conditions. We combine information from several administrative and survey data sets to obtain model-based dummy variables for 107 health conditions (diseases, preventive measures, and screening for diseases) for elderly (age 65 and older) subjects in the Medicare Current Beneficiary Survey (MCBS) over the fourteen-year period, 1999-2012. The MCBS has prevalence of diseases assessed based on Medicare claims and provides detailed information on all health conditions but is prone to underestimation bias. The National Health and Nutrition Examination Survey (NHANES), on the other hand, collects self-reports and physical/laboratory measures only for a subset of the 107 health conditions. Neither source provides complete information, but we use them together to derive model-based corrected dummy variables in MCBS for the full range of existing health conditions using a missing data and measurement error model framework. We create multiply imputed dummy variables and use them to construct the prevalence rate and trend estimates. The broader goal, however, is to use these corrected or modeled dummy variables for a multitude of policy analysis, cost modeling, and analysis of other relationships either using them as predictors or as outcome variables.

State-level estimation of diabetes and prediabetes prevalence: Combining national and local survey data and clinical data

Many statisticians and policy researchers are interested in using data generated through the normal delivery of health care services, rather than carefully designed and implemented population-representative surveys, to estimate disease prevalence. These larger databases allow for the estimation of smaller geographies, for example, states, at potentially lower expense. However, these health care records frequently do not cover all of the population of interest and may not collect some covariates that are important for accurate estimation. In a recent paper, the authors have described how to adjust for the incomplete coverage of administrative claims data and electronic health records at the state or local level. This article illustrates how to adjust and combine multiple data sets, namely, national surveys, state-level surveys, claims data, and electronic health record data, to improve estimates of diabetes and prediabetes prevalence, along with the estimates of the method's accuracy. We demonstrate and validate the method using data from three jurisdictions (Alabama, California, and New York City). This method can be applied more generally to other areas and other data sources.