The performance of multiple imputation for missing covariates relative to complete case analysis

Evaluation of Multiple Imputation with Large Proportions of Missing Data: How Much Is Too Much?

Background:

Multiple Imputation (MI) is known as an effective method for handling missing data in public health research. However, it is not clear that the method will be effective when the data contain a high percentage of missing observations on a variable.

Methods:

Using data from “Predictive Study of Coronary Heart Disease” study, this study examined the effectiveness of multiple imputation in data with 20% missing to 80% missing observations using absolute bias (|bias|) and Root Mean Square Error (RMSE) of MI measured under Missing Completely at Random (MCAR), Missing at Random (MAR), and Not Missing at Random (NMAR) assumptions.

Results:

The |bias| and RMSE of MI was much smaller than of the results of CCA under all missing mechanisms, especially with a high percentage of missing. In addition, the |bias| and RMSE of MI were consistent regardless of increasing imputation numbers from M=10 to M=50. Moreover, when comparing imputation mechanisms, MCMC method had universally smaller |bias| and RMSE than those of Regression method and Predictive Mean Matching method under all missing mechanisms.

Conclusion:

As missing percentages become higher, using MI is recommended, because MI produced less biased estimates under all missing mechanisms. However, when large proportions of data are missing, other things need to be considered such as the number of imputations, imputation mechanisms, and missing data mechanisms for proper imputation.

Multiple imputation for patient reported outcome measures in randomised controlled trials: advantages and disadvantages of imputing at the item, subscale or composite score level

Background

Missing data can introduce bias in the results of randomised controlled trials (RCTs), but are typically unavoidable in pragmatic clinical research, especially when patient reported outcome measures (PROMs) are used. Traditionally applied to the composite PROMs score of multi-item instruments, some recent research suggests that multiple imputation (MI) at the item level may be preferable under certain scenarios.

This paper presents practical guidance on the choice of MI models for handling missing PROMs data based on the characteristics of the trial dataset. The comparative performance of complete cases analysis, which is commonly used in the analysis of RCTs, is also considered.

Methods

Realistic missing at random data were simulated using follow-up data from an RCT considering three different PROMs (Oxford Knee Score (OKS), EuroQoL 5 Dimensions 3 Levels (EQ-5D-3L), 12-item Short Form Survey (SF-12)). Data were multiply imputed at the item (using ordinal logit and predicted mean matching models), sub-scale and score level; unadjusted mean outcomes, as well as treatment effects from linear regression models were obtained for 1000 simulations. Performance was assessed by root mean square errors (RMSE) and mean absolute errors (MAE).

Results

Convergence problems were observed for MI at the item level. Performance generally improved with increasing sample sizes and lower percentages of missing data. Imputation at the score and subscale level outperformed imputation at the item level in small sample sizes (n ≤ 200). Imputation at the item level is more accurate for high proportions of item-nonresponse. All methods provided similar results for large sample sizes (≥500) in this particular case study.

Conclusions

Many factors, including the prevalence of missing data in the study, sample size, the number of items within the PROM and numbers of levels within the individual items, and planned analyses need consideration when choosing an imputation model for missing PROMs data.

Electronic supplementary material

The online version of this article (10.1186/s12874-018-0542-6) contains supplementary material, which is available to authorized users.

Associations between race, lifecourse socioeconomic position and prevalence of diabetes among US women and men: results from a population-based panel study

BACKGROUND

Few studies have examined the degree to which racial disparities in the development of diabetes are accounted by differences in lifecourse socioeconomic position (SEP). We assessed the association between race, lifecourse SEP measures and prevalence of diabetes in a representative US sample of black and white adults.

METHODS

A generalised estimating equations approach was used with a sample of 3497 adults from the Americans' Changing Lives study. Sex-specific models were calculated to compute prevalence ratios (PR) for associations of race and SEP with self-reported diagnoses of diabetes.

RESULTS

For men, childhood and adult SEP were unrelated to diabetes, and adjustment for lifecourse SEP had little effect on the excess diabetes in blacks (PR=1.56, 95% CI 1.11 to 2.21). Adjustment for measures of lifecourse SEP reduced the PR for the association between race and diabetes in women from 1.96 (95% CI 1.52 to 2.54) to 1.40 (95% CI 1.04 to 1.87) with the respondent's education responsible for most of the reduction in the association. However, diabetes was also inversely associated with father's education, and low SEP throughout the lifecourse was associated with a nearly threefold increase in diabetes (PR=2.89, 95% CI 2.10 to 3.99).

CONCLUSIONS

Racial disparities in diabetes existed among both men and women, but lifecourse SEP was related to diabetes only among women. The pathway and cumulative hypotheses for lifecourse SEP effects on diabetes may be especially salient for women.

Imputation of response rates from means and standard deviations in schizophrenia

Missing outcome data is a major threat in meta-analytical studies of schizophrenia. Most clinical trials in psychiatry report only continuous outcome measures and express the effect of an intervention as a difference of means. However, these results are difficult to interpret for clinicians. Converting continuous data to binary response rates is one possible solution to the problem. Based on means and standard deviations for a continuous outcome, we examined the performance of an imputation method to define a dichotomous outcome using original individual patients' data from 16 randomized trials (6276 participants) comparing antipsychotic drugs in schizophrenia. We concluded that the imputed values re-captured in a reasonable degree the observed values providing a simple and practical alternative methodological choice for imputation of missing binary data in schizophrenia trials; nevertheless, the imputation method tended to introduce biases, especially for extreme risks and large treatment differences.

Comparison of methods for handling missing data on immunohistochemical markers in survival analysis of breast cancer

BACKGROUND

Tissue micro-arrays (TMAs) are increasingly used to generate data of the molecular phenotype of tumours in clinical epidemiology studies, such as studies of disease prognosis. However, TMA data are particularly prone to missingness. A variety of methods to deal with missing data are available. However, the validity of the various approaches is dependent on the structure of the missing data and there are few empirical studies dealing with missing data from molecular pathology. The purpose of this study was to investigate the results of four commonly used approaches to handling missing data from a large, multi-centre study of the molecular pathological determinants of prognosis in breast cancer.

PATIENTS AND METHODS

We pooled data from over 11,000 cases of invasive breast cancer from five studies that collected information on seven prognostic indicators together with survival time data. We compared the results of a multi-variate Cox regression using four approaches to handling missing data - complete case analysis (CCA), mean substitution (MS) and multiple imputation without inclusion of the outcome (MI-) and multiple imputation with inclusion of the outcome (MI+). We also performed an analysis in which missing data were simulated under different assumptions and the results of the four methods were compared.

RESULTS

Over half the cases had missing data on at least one of the seven variables and 11 percent had missing data on 4 or more. The multi-variate hazard ratio estimates based on multiple imputation models were very similar to those derived after using MS, with similar standard errors. Hazard ratio estimates based on the CCA were only slightly different, but the estimates were less precise as the standard errors were large. However, in data simulated to be missing completely at random (MCAR) or missing at random (MAR), estimates for MI+ were least biased and most accurate, whereas estimates for CCA were most biased and least accurate.

CONCLUSION

In this study, empirical results from analyses using CCA, MS, MI- and MI+ were similar, although results from CCA were less precise. The results from simulations suggest that in general MI+ is likely to be the best. Given the ease of implementing MI in standard statistical software, the results of MI+ and CCA should be compared in any multi-variate analysis where missing data are a problem.