Reflection on modern methods: planned missing data designs for epidemiological research

Dealing with missing data in multi-informant studies: A comparison of approaches

Multi-informant studies are popular in social and behavioral science. However, their data analyses are challenging because data from different informants carry both shared and unique information and are often incomplete. Using Monte Carlo Simulation, the current study compares three approaches that can be used to analyze incomplete multi-informant data when there is a distinction between reference and nonreference informants. These approaches include a two-method measurement model for planned missing data (2MM-PMD), treating nonreference informants' reports as auxiliary variables with the full-information maximum likelihood method or multiple imputation, and listwise deletion. The result suggests that 2MM-PMD, when correctly specified and data are missing at random, has the best overall performance among the examined approaches regarding point estimates, type I error rates, and statistical power. In addition, it is also more robust to data that are not missing at random.

On Modeling Missing Data in Structural Investigations Based on Tetrachoric Correlations With Free and Fixed Factor Loadings

In modeling missing data, the missing data latent variable of the confirmatory factor model accounts for systematic variation associated with missing data so that replacement of what is missing is not required. This study aimed at extending the modeling missing data approach to tetrachoric correlations as input and at exploring the consequences of switching between models with free and fixed factor loadings. In a simulation study, confirmatory factor analysis (CFA) models with and without a missing data latent variable were used for investigating the structure of data with and without missing data. In addition, the numbers of columns of data sets with missing data and the amount of missing data were varied. The root mean square error of approximation (RMSEA) results revealed that an additional missing data latent variable recovered the degree-of-model fit characterizing complete data when tetrachoric correlations served as input while comparative fit index (CFI) results showed overestimation of this degree-of-model fit. Whereas the results for fixed factor loadings were in line with the assumptions of modeling missing data, the other results showed only partial agreement. Therefore, modeling missing data with fixed factor loadings is recommended.

Everything has a limit: How intellectual humility lowers the preference for naturalness as reflected in drug choice

OBJECTIVE

Across a broad range of cultures, people demonstrate a strong preference for items that are labeled as natural. Yet, less is known about methods that can reduce the natural-is-better bias. The objective of the present research is to see whether intellectual humility, a moral virtue that can be understood as a more open and curious mindset, reduces naturalness bias in terms of drug-related decisions.

METHODS

We tested our hypotheses across four studies using different populations (university students and community adults) and methods (correlational and experimental). Study 1 involved a survey exploring whether university students choosing a synthetic drug tended to display a higher level of intellectual humility than those choosing a natural drug. Study 2 assessed the link using observation of real-world behavior in non-student adults. Study 3 adopted an experimental approach to test the idea that reflecting on one's intellectual fallibility can at least temporarily reduce naturalness bias on drug choice. Study 4 examined the potential mediating mechanism underlying the observed effect.

RESULTS

We found correlational and experimental evidence that participants higher in intellectual humility were more likely to choose the synthetic drug than those lower in intellectual humility in both self-report and behavioral measures. The results also demonstrate that openness to experience mediated the effect of intellectual humility on naturalness bias.

CONCLUSIONS

These results highlight intellectual humility as a malleable, psychological variable that can combat biased thinking associated with health-related decision-making.

The relationship of COVID-19 traumatic stress, cumulative trauma, and race to posttraumatic stress disorder symptoms

The purpose of this study was to test if coronavirus disease 2019 (COVID-19) traumatic stress predicts posttraumatic stress disorder (PTSD) symptoms after cumulative trauma and whether there is a three-way interaction between COVID-19 traumatic stress, cumulative trauma, and race in the prediction of PTSD. Using a cross-sectional design, a diverse sample of 745 participants completed measures of cumulative trauma, COVID-19 traumatic stress, and PTSD. COVID-19 traumatic stress accounted for a significant amount of the variance in PTSD above and beyond cumulative trauma. A significant interaction effect was found, indicating that the effect of COVID-19 traumatic stress in predicting PTSD varied as a function of cumulative trauma and that the effects of that interaction were different for Asians and Whites. There were generally comparable associations between COVID-19 traumatic stress and PTSD at low and high levels of cumulative trauma across most racial groups. However, for Asians, higher levels of cumulative trauma did not worsen the PTSD outcome as a function of COVID Traumatic Stress but did at low levels of cumulative trauma.

The 3D-Transition Study: Objectives, Methods, and Implementation of an Innovative Planned Missing-Data Design

The prevalence of mental health problems represents a significant burden on school and community health resources as early as preschool. Reducing this burden requires a better understanding of the developmental mechanisms linking children's early vulnerabilities with mental health after the transition to formal schooling. The 3D-Transition Study (2017-2021) follows 939 participants from a pregnancy cohort in the province of Québec, Canada, as they transition to kindergarten and first grade to examine these mechanisms. Biannual assessments include completed questionnaires from 2 parents as well as teachers, parent-child observations, anthropometric measurements, and age-sensitive cognitive assessments. Saliva is also collected on 11 days over a 16-month period in a subsample of 384 participants to examine possible changes in child salivary cortisol levels across the school transition and their role in difficulties observed during the transition. A combination of planned missing-data designs is being implemented to reduce participant burden, where incomplete data are collected without introducing bias after the use of multiple imputation. The 3D-Transition Study will contribute to an evidence-based developmental framework of child mental health from pregnancy to school age. In turn, this framework can help inform prevention programs delivered in health-care settings during pregnancy and in child-care centers, preschools, and schools.

Sampling Strategies for Internal Validation Samples for Exposure Measurement-Error Correction: A Study of Visceral Adipose Tissue Measures Replaced by Waist Circumference Measures

Statistical correction for measurement error in epidemiologic studies is possible, provided that information about the measurement error model and its parameters are available. Such information is commonly obtained from a randomly sampled internal validation sample. It is however unknown whether randomly sampling the internal validation sample is the optimal sampling strategy. We conducted a simulation study to investigate various internal validation sampling strategies in conjunction with regression calibration. Our simulation study showed that for an internal validation study sample of 40% of the main study’s sample size, stratified random and extremes sampling had a small efficiency gain over random sampling (10% and 12% decrease on average over all scenarios, respectively). The efficiency gain was more pronounced in smaller validation samples of 10% of the main study’s sample size (i.e., a 31% and 36% decrease on average over all scenarios, for stratified random and extremes sampling, respectively). To mitigate the bias due to measurement error in epidemiologic studies, small efficiency gains can be achieved for internal validation sampling strategies other than random, but only when measurement error is nondifferential. For regression calibration, the gain in efficiency is, however, at the cost of a higher percentage bias and lower coverage.

Uncovering survivorship bias in longitudinal mental health surveys during the COVID-19 pandemic

AIMS

Markedly elevated adverse mental health symptoms were widely observed early in the coronavirus disease-2019 (COVID-19) pandemic. Unlike the U.S., where cross-sectional data indicate anxiety and depression symptoms have remained elevated, such symptoms reportedly declined in the U.K., according to analysis of repeated measures from a large-scale longitudinal study. However, nearly 40% of U.K. respondents (those who did not complete multiple follow-up surveys) were excluded from analysis, suggesting that survivorship bias might partially explain this discrepancy. We therefore sought to assess survivorship bias among participants in our longitudinal survey study as part of The COVID-19 Outbreak Public Evaluation (COPE) Initiative.

METHODS

Survivorship bias was assessed in 4039 U.S. respondents who completed surveys including the assessment of mental health as part of The COPE Initiative in April 2020 and were invited to complete follow-up surveys. Participants completed validated screening instruments for symptoms of anxiety, depression and insomnia. Survivorship bias was assessed for (1) demographic differences in follow-up survey participation, (2) differences in initial adverse mental health symptom prevalence adjusted for demographic factors and (3) differences in follow-up survey participation based on mental health experiences adjusted for demographic factors.

RESULTS

Adjusting for demographics, individuals who completed only one or two out of four surveys had significantly higher prevalence of anxiety and depression symptoms in April 2020 (e.g. one-survey v. four-survey, anxiety symptoms, adjusted prevalence ratio [aPR]: 1.30, 95% confidence interval [CI]: 1.08-1.55, p = 0.0045; depression symptoms, aPR: 1.43, 95% CI: 1.17-1.75, p = 0.00052). Moreover, individuals who experienced incident anxiety or depression symptoms had significantly higher adjusted odds of not completing follow-up surveys (adjusted odds ratio [aOR]: 1.68, 95% CI: 1.22-2.31, p = 0.0015, aOR: 1.56, 95% CI: 1.15-2.12, p = 0.0046, respectively).

CONCLUSIONS

Our findings reveal significant survivorship bias among longitudinal survey respondents, indicating that restricting analytic samples to only respondents who provide repeated assessments in longitudinal survey studies could lead to overly optimistic interpretations of mental health trends over time. Cross-sectional or planned missing data designs may provide more accurate estimates of population-level adverse mental health symptom prevalence than longitudinal surveys.

A systematic review of how missing data are handled and reported in multi-database pharmacoepidemiologic studies

Purpose

Pharmacoepidemiologic multi‐database studies (MDBS) provide opportunities to better evaluate the safety and effectiveness of medicines. However, the issue of missing data is often exacerbated in MDBS, potentially resulting in bias and precision loss. We sought to measure how missing data are being recorded and addressed in pharmacoepidemiologic MDBS.

Methods

We conducted a systematic literature search in PubMed for pharmacoepidemiologic MDBS published between 1st January 2018 and 31st December 2019. Included studies were those that used ≥2 distinct databases to assess the same safety/effectiveness outcome associated with a drug exposure. Outcome variables extracted from the studies included strategies to execute a MDBS, reporting of missing data (type, bias evaluation) and the methods used to account for missing data.

Results

Two thousand seven hundred and twenty‐six articles were identified, and 62 studies were included: using data from either North America (56%), Europe (31%), multiple regions (11%) or East‐Asia (2%). Thirty‐five (56%) articles reported missing data: 11 of these studies reported that this could have introduced bias and 19 studies reported a method to address missing data. Thirteen (68%) carried out a complete case analysis, 2 (11%) applied multiple imputation, 2 (11%) used both methods, 1 (5%) used mean imputation and 1 (5%) substituted information from a similar variable.

Conclusions

Just over half of the recent pharmacoepidemiologic MDBS reported missing data and two‐thirds of these studies reported how they accounted for it. We should increase our vigilance for database completeness in MDBS by reporting and addressing the missing data that could introduce bias.

Uncovering Survivorship Bias in Longitudinal Mental Health Surveys During the COVID-19 Pandemic

Aims

Markedly elevated adverse mental health symptoms were widely observed early in the coronavirus disease 2019 (COVID-19) pandemic. Unlike the U.S., where cross-sectional data indicate anxiety and depression symptoms have remained elevated, such symptoms reportedly declined in the U.K., according to analysis of repeated measures from a largescale longitudinal study. However, nearly 40% of U.K. respondents (those who did not complete multiple follow-up surveys) were excluded from analysis, suggesting that survivorship bias might partially explain this discrepancy. We therefore sought to assess survivorship bias among participants in our longitudinal survey study as part of The COVID-19 Outbreak Public Evaluation (COPE) Initiative.

Methods

Survivorship bias was assessed 4,039 U.S. respondents who completed surveys including the assessment of mental health as part of The COPE Initiative in April 2020 and were invited to complete follow-up surveys. Participants completed validated screening instruments for symptoms of anxiety, depression, and insomnia. Survivorship bias was assessed for (1) demographic differences in follow-up survey participation, (2) differences in initial adverse mental health symptom prevalences adjusted for demographic factors, and (3) differences in follow-up survey participation based on mental health experiences adjusted for demographic factors.

Results

Adjusting for demographics, individuals who completed only one or two out of four surveys had higher prevalences of anxiety and depression symptoms in April 2020 (e.g., one-survey versus four-survey, anxiety symptoms, adjusted prevalence ratio [aPR]: 1.30, 95% confidence interval [CI]: 1.08–1.55, P=0.0045; depression symptoms, aPR: 1.43, 95% CI: 1.17–1.75, P=0.00052). Moreover, individuals who experienced incident anxiety or depression symptoms had higher odds of not completing follow-up surveys (adjusted odds ratio [aOR]: 1.68, 95% CI: 1.22–2.31, P=0.0015, aOR: 1.56, 95% CI: 1.15–2.12, P=0.0046, respectively).

Conclusions

Our findings revealed significant survivorship bias among longitudinal survey respondents, indicating that restricting analytic samples to only respondents who provide repeated assessments in longitudinal survey studies could lead to overly optimistic interpretations of mental health trends over time. Cross-sectional or planned missing data designs may provide more accurate estimates of population-level adverse mental health symptom prevalences than longitudinal surveys.

On Modeling Missing Data of an Incomplete Design in the CFA Framework

The paper reports an investigation on whether valid results can be achieved in analyzing the structure of datasets although a large percentage of data is missing without replacement. Two types of confirmatory factor analysis (CFA) models were employed for this purpose: the missing data CFA model with an additional latent variable for representing the missing data and the semi-hierarchical CFA model that also includes the additional latent variable and reflects the hierarchical structure assumed to underlie the data. Whereas, the missing data CFA model assumes that the model is equally valid for all participants, the semi-hierarchical CFA model is implicitly specified differently for subgroups of participants with and without omissions. The comparison of these models with the regular one-factor model in investigating simulated binary data revealed that the modeling of missing data prevented negative effects of missing data on model fit. The investigation of the accuracy in estimating the factor loadings yielded the best results for the semi-hierarchical CFA model. The average estimated factor loadings for items with and without omissions showed the expected equal sizes. But even this model tended to underestimate the expected values.