Multiple Imputation for Model Checking: Completed-Data Plots with Missing and Latent Data

Negative variance components and intercept-slope correlations greater than one in magnitude: How do such "non-regular" random intercept and slope models arise, and what should be done when they do?

Statistical models with random intercepts and slopes (RIAS models) are commonly used to analyze longitudinal data. Fitting such models sometimes results in negative estimates of variance components or estimates on parameter space boundaries. This can be an unlucky chance occurrence, but can also occur because certain marginal distributions are mathematically identical to those from RIAS models with negative intercept and/or slope variance components and/or intercept-slope correlations greater than one in magnitude. We term such parameters "pseudo-variances" and "pseudo-correlations," and the models "non-regular." We use eigenvalue theory to explore how and when such non-regular RIAS models arise, showing: (i) A small number of measurements, short follow-up, and large residual variance increase the parameter space for which data (with a positive semidefinite marginal variance-covariance matrix) are compatible with non-regular RIAS models. (ii) Non-regular RIAS models can arise from model misspecification, when non-linearity in fixed effects is ignored or when random effects are omitted. (iii) A non-regular RIAS model can sometimes be interpreted as a regular linear mixed model with one or more additional random effects, which may not be identifiable from the data. (iv) Particular parameterizations of non-regular RIAS models have no generality for all possible numbers of measurements over time. Because of this lack of generality, we conclude that non-regular RIAS models can only be regarded as plausible data-generating mechanisms in some situations. Nevertheless, fitting a non-regular RIAS model can be acceptable, allowing unbiased inference on fixed effects where commonly recommended alternatives such as dropping the random slope result in bias.

A Bayesian joint model for multivariate longitudinal and time-to-event data with application to ALL maintenance studies

The most common type of cancer diagnosed among children is the Acute Lymphocytic Leukemia (ALL). A study was conducted by Tata Translational Cancer Research Center (TTCRC) Kolkata, in which 236 children (diagnosed as ALL patients) were treated for the first two years (approximately) with two standard drugs (6MP and MTx) and were then followed nearly for the next 3 years. The goal is to identify the longitudinal biomarkers that are associated with time-to-relapse, and also to assess the effectiveness of the drugs. We develop a Bayesian joint model in which a linear mixed model is used to jointly model three biomarkers (i.e. white blood cell count, neutrophil count, and platelet count) and a semi-parametric proportional hazards model is used to model the time-to-relapse. Our proposed joint model can assess the effects of different covariates on the progression of the biomarkers, and the effects of the biomarkers (and the covariates) on time-to-relapse. In addition, the proposed joint model can impute the missing longitudinal biomarkers efficiently. Our analysis shows that the white blood cell (WBC) count is not associated with time-to-relapse, but the neutrophil count and the platelet count are significantly associated with it. We also infer that a lower dose of 6MP and a higher dose of MTx jointly result in a lower relapse probability in the follow-up period. Interestingly, we find that relapse probability is the lowest for the patients classified into the "high-risk" group at presentation. The effectiveness of the proposed joint model is assessed through the extensive simulation studies.

Graphical and numerical diagnostic tools to assess multiple imputation models by posterior predictive checking

Problem

The congenial of the imputation model is crucial for valid statistical inferences. Hence, it is important to develop methodologies for diagnosing imputation models.

Aim

We propose and evaluate a new diagnostic method based on posterior predictive checking to diagnose the congeniality of fully conditional imputation models. Our method applies to multiple imputation by chained equations, which is widely used in statistical software.

Methods

The proposed method compares the observed data with their replicates generated under the corresponding posterior predictive distributions to diagnose the performance of imputation models. The method applies to various imputation models, including parametric and semi-parametric approaches and continuous and discrete incomplete variables. We studied the validity of the method through simulation and application.

Results

The proposed diagnostic method based on posterior predictive checking demonstrates its validity in assessing the performance of imputation models. The method can diagnose the consistency of imputation models with the substantive model and can be applied to a broad range of research contexts.

Conclusion

The diagnostic method based on posterior predictive checking provides a valuable tool for researchers who use fully conditional specification to handle missing data. By assessing the performance of imputation models, our method can help researchers improve the accuracy and reliability of their analyzes. Furthermore, our method applies to different imputation models. Hence, it is a versatile and valuable tool for researchers identifying plausible imputation models.

A Bayesian joint model for multivariate longitudinal and time-to-event data with application to ALL maintenance studies

The most common type of cancer diagnosed among children is the acute lymphocytic leukemia (ALL). A study was conducted by Tata Translational Cancer Research Center (TTCRC) Kolkata, in which 236 children (diagnosed as ALL patients) were treated for the first two years (approximately) with two standard drugs (6MP and MTx) and were then followed nearly for the next three years. The goal is to identify the longitudinal biomarkers that are associated with time-to-relapse, and also to assess the effectiveness of the drugs. We develop a Bayesian joint model in which a linear mixed model is used to jointly model three biomarkers (i.e. white blood cell count, neutrophil count, and platelet count) and a semi-parametric proportional hazards model is used to model the time-to-relapse. Our proposed joint model can assess the effects of different covariates on the progression of the biomarkers, and the effects of the biomarkers (and the covariates) on time-to-relapse. In addition, the proposed joint model can impute the missing longitudinal biomarkers efficiently. Our analysis shows that the white blood cell (WBC) count is not associated with time-to-relapse, but the neutrophil count and the platelet count are significantly associated with it. We also infer that a lower dose of 6MP and a higher dose of MTx jointly result in a lower relapse probability in the follow-up period. Interestingly, we find that relapse probability is the lowest for the patients classified into the "high-risk" group at presentation. The effectiveness of the proposed joint model is assessed through the extensive simulation studies.

Assessing conditional causal effect via characteristic score

Observational studies usually include participants representing the wide heterogeneous population. The conditional causal effect, treatment effect conditional on baseline characteristics, is of practical importance. Its estimation is subject to two challenges. First, the causal effect is not observable in any individual due to counterfactuality. Second, high-dimensional baseline variables are involved to satisfy the ignorable treatment selection assumption and to attain better estimation efficiency. In this work, a nonparametric estimation procedure, along with a pseudo-response, is proposed to estimate the conditional treatment effect through "characteristic score"-a parsimonious representation of baseline variable influence on treatment benefit. Adopting sparse dimension reduction with variable prescreening in the proposed estimation, we aim to identify the key baseline variables that impact the conditional treatment effect and to uncover the characteristic score that best predicts the treatment effect. This approach is applied to an HIV study for assessing the benefit of antiretroviral regimens and identifying the beneficiary subpopulation.

Identifying occupancy model inadequacies: can residuals separately assess detection and presence?

Occupancy models are widely applied to estimate species distributions, but few methods exist for model checking. Thorough model assessments can uncover inadequacies and allow for deeper ecological insight by exploring structure in the observed data not accounted for by a model. We introduce occupancy model residual definitions that utilize the posterior distribution of the partially latent occupancy states. Residual-based assessments are valuable because they can target specific assumptions and identify ways to improve a model, such as adding spatial correlation or meaningful covariates. Our approach defines separate residuals for occupancy and detection, and we use simulation to examine whether missing structure for modeling detection probabilities can be distinguished from that for occupancy probabilities. In many scenarios, our residual diagnostics were able to separate inadequacies at the different model levels successfully, but we describe other situations when this may not be the case. Applying Moran's I residual diagnostics to assess models for silver-haired (Lasionycteris noctivagans) and little brown (Myotis lucifugus) bats only provided evidence of residual spatial correlation among detections. Targeting specific model assumptions using carefully chosen residual diagnostics is valuable for any analysis, and we remove previous barriers for occupancy analyses-lack of examples and practical advice.

Use of an mHealth Intervention to Improve Engagement in HIV Community-Based Care Among Persons Recently Released from a Correctional Facility in Washington, DC: A Pilot Study

We examined the preliminary effectiveness of a computerized counseling session plus post-incarceration text messaging intervention (CARE + Corrections) to support ART adherence and linkage/engagement in community care among recently incarcerated persons with HIV in Washington, D.C. Recently incarcerated persons with HIV ≥ 18 years old were recruited from the D.C. jail or community outreach and randomized to CARE + Corrections or control arm. Participants completed assessments at baseline, 3-months and 6-months. Multivariable random effects modeling identified predictors of suppressed viral load (≤ 200 copies/mL) and engagement in HIV care at 6 months. Participants (N = 110) were aged 42 (IQR 30-49); 58% male, 24% female, 18% transgender, 85% Black, and lifetime incarceration was a median of 7 years (IQR 2-15). More controls had a regular healthcare provider at baseline. Although not statistically significant, intervention participants had increased odds of viral suppression versus controls at 6 months (AOR 2.04; 95% CI 0.62, 6.70). Those reporting high ART adherence at baseline had higher odds of viral suppression at follow-up (AOR 10.77; 95% CI 1.83, 63.31). HIV care engagement was similar between the two groups, although both groups reported increased engagement at 6 months versus baseline. We observed a positive but non-significant association of viral suppression in the CARE + Corrections group, and care engagement increased in both groups after 6 months. Further attention to increasing viral suppression among CJ-involved persons with HIV upon community reentry is warranted.

Lack of efficacy in a randomised trial of a brief intervention to reduce drug use and increase drug treatment services utilisation among adult emergency department patients over a 12-month period

OBJECTIVES

Assess the 12-month efficacy of a brief intervention (BI) on reducing drug use and increasing drug treatment services utilisation among adult emergency department (ED) patients.

METHODS

This randomised, controlled trial enrolled 18-64-year-old ED patients needing a drug use intervention. Treatment arm participants received a tailored BI while control arm participants only completed the study questionnaires. Self-reported past 3-month drug use and engagement in drug treatment services were compared by study arm at 3-month intervals over 1 year. Multiple imputations were performed to overcome loss-to-follow-up.

RESULTS

Of the 1030 participants, follow-up completion ranged 55%-64% over the four follow-ups. At 12 months, the two study arms were similar in regards to mean: (1) proportion reporting any drug use (treatment: 67.1% (61.6 to 72.6), control: 74.4% (69.4 to 79.4)); (2) drug use frequency on a five-point scale (treatment: 3.7 (3.3 to 4.2), control: 4.6 (4.0 to 5.2)); (3) total days of drug use (treatment: 28.3 (23.2 to 33.4), control: 33.4 (28.5 to 38.2)); (4) most number of times drugs used/day (treatment: 4.6 (3.6 to 5.5), control: 6.1 (4.8 to 7.3)) and (5) typical number of times drugs used/day (treatment: 3.3 (2.5 to 4.1), control: 5.1 (3.9 to 6.2)). Utilisation of drug treatment services also was similar by study arm. In multivariable regression analyses, patients who were homeless or had higher drug use at baseline continued to have greater drug use in follow-up.

CONCLUSIONS

Among adult ED patients requiring a drug use intervention, this BI did not decrease drug use or increase drug treatment services utilisation over a 12-month period more than the control condition.

TRIAL REGISTRATION NUMBER

NCT01124591; Pre-trial.

Model validation and influence diagnostics for regression models with missing covariates

Missing covariate values are prevalent in regression applications. While an array of methods have been developed for estimating parameters in regression models with missing covariate data for a variety of response types, minimal focus has been given to validation of the response model and influence diagnostics. Previous research has mainly focused on estimating residuals for observations with missing covariates using expected values, after which specialized techniques are needed to conduct proper inference. We suggest a multiple imputation strategy that allows for the use of standard methods for residual analyses on the imputed data sets or a stacked data set. We demonstrate the suggested multiple imputation method by analyzing the Sleep in Mammals data in the context of a linear regression model and the New York Social Indicators Status data with a logistic regression model.

Impact of a brief intervention on reducing alcohol use and increasing alcohol treatment services utilization among alcohol- and drug-using adult emergency department patients

Most previous brief intervention (BI) studies have focused on alcohol or drug use, instead of both substances. Our primary aim was to determine if an alcohol- and drug-use BI reduced alcohol use and increased alcohol treatment services utilization among adult emergency department (ED) patients who drink alcohol and require an intervention for their drug use. Our secondary aims were to assess when the greatest relative reductions in alcohol use occurred, and which patients (stratified by need for an alcohol use intervention) reduced their alcohol use the most. In this secondary analysis, we studied a sub-sample of participants from the Brief Intervention for Drug Misuse in the Emergency Department (BIDMED) randomized, controlled trial of a BI vs. no BI, whose responses to the Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) indicated a need for a BI for any drug use, and who also reported alcohol use. Participants were stratified by their ASSIST alcohol subscore: 1) no BI needed, 2) a BI needed, or 3) an intensive intervention needed for alcohol use. Alcohol use and alcohol treatment services utilization were measured every 3 months for 12 months post-enrollment. Of these 833 participants, median age was 29 years-old, 46% were female; 55% were white/non-Hispanic, 27% black/non-Hispanic, and 15% Hispanic. Although any alcohol use, alcohol use frequency, days of alcohol use, typical drinks consumed/day, and most drinks consumed/day decreased in both the BI and no BI arms, there were no differences between study arms. Few patients sought alcohol use treatment services in follow-up, and utilization also did not differ by study arm. Compared to baseline, alcohol use reduced the most during the first 3 months after enrollment, yet reduced little afterward. Participants whose ASSIST alcohol subscores indicated a need for an intensive intervention generally had the greatest relative decreases in alcohol use. These results indicate that the BI was not efficacious in reducing alcohol use among alcohol- and drug-using adult ED patients than the self-assessments alone, but suggest that self-assessments with or without a BI may confer reductions in alcohol use.

Model checking in multiple imputation: an overview and case study

Background

Multiple imputation has become very popular as a general-purpose method for handling missing data. The validity of multiple-imputation-based analyses relies on the use of an appropriate model to impute the missing values. Despite the widespread use of multiple imputation, there are few guidelines available for checking imputation models.

Analysis

In this paper, we provide an overview of currently available methods for checking imputation models. These include graphical checks and numerical summaries, as well as simulation-based methods such as posterior predictive checking. These model checking techniques are illustrated using an analysis affected by missing data from the Longitudinal Study of Australian Children.

Conclusions

As multiple imputation becomes further established as a standard approach for handling missing data, it will become increasingly important that researchers employ appropriate model checking approaches to ensure that reliable results are obtained when using this method.

Electronic supplementary material

The online version of this article (doi:10.1186/s12982-017-0062-6) contains supplementary material, which is available to authorized users.

Simulation-based sensitivity analysis for non-ignorably missing data

Sensitivity analysis is popular in dealing with missing data problems particularly for non-ignorable missingness, where full-likelihood method cannot be adopted. It analyses how sensitively the conclusions (output) may depend on assumptions or parameters (input) about missing data, i.e. missing data mechanism. We call models with the problem of uncertainty sensitivity models. To make conventional sensitivity analysis more useful in practice we need to define some simple and interpretable statistical quantities to assess the sensitivity models and make evidence based analysis. We propose a novel approach in this paper on attempting to investigate the possibility of each missing data mechanism model assumption, by comparing the simulated datasets from various MNAR models with the observed data non-parametrically, using the K-nearest-neighbour distances. Some asymptotic theory has also been provided. A key step of this method is to plug in a plausibility evaluation system towards each sensitivity parameter, to select plausible values and reject unlikely values, instead of considering all proposed values of sensitivity parameters as in the conventional sensitivity analysis method. The method is generic and has been applied successfully to several specific models in this paper including meta-analysis model with publication bias, analysis of incomplete longitudinal data and mean estimation with non-ignorable missing data.

A note on posterior predictive checks to assess model fit for incomplete data

We examine two posterior predictive distribution based approaches to assess model fit for incomplete longitudinal data. The first approach assesses fit based on replicated complete data as advocated in Gelman et al. (2005). The second approach assesses fit based on replicated observed data. Differences between the two approaches are discussed and an analytic example is presented for illustration and understanding. Both checks are applied to data from a longitudinal clinical trial. The proposed checks can easily be implemented in standard software like (Win)BUGS/JAGS/Stan. Copyright © 2016 John Wiley & Sons, Ltd.

Bayesian methods for latent trait modelling of longitudinal data

Latent trait models have long been used in the social science literature for studying variables that can only be measured indirectly through multiple items. However, such models are also very useful in accounting for correlation in multivariate and longitudinal data, particularly when outcomes have mixed measurement scales. Bayesian methods implemented with Markov chain Monte Carlo provide a flexible framework for routine fitting of a broad class of latent variable (LV) models, including very general structural equation models. However, in considering LV models, a number of challenging issues arise, including identifiability, confounding between the mean and variance, uncertainty in different aspects of the model, and difficulty in computation. Motivated by the problem of modelling multidimensional longitudinal data, this article reviews the recent literature, provides some recommendations and highlights areas in need of additional research, focusing on methods for model uncertainty.

Model-based imputation of latent cigarette counts using data from a calibration study

In addition to dichotomous measures of abstinence, smoking studies may use daily cigarette consumption as an outcome variable. These counts hold the promise of more efficient and detailed analyses than dichotomous measures, but present serious quality issues - measurement error and heaping - if obtained by retrospective recall. A doubly-coded dataset with a retrospective recall measurement (timeline followback, TLFB) and a more precise instantaneous measurement (ecological momentary assessment, EMA) serves as a calibration dataset, allowing us to predict EMA given TLFB and baseline factors. We apply this model to multiply impute precise cigarette counts for a randomized, placebo-controlled trial of bupropion with only TLFB measurements available. To account for repeated measurements on a subject, we induce correlation in the imputed counts. Finally, we analyze the imputed data in a longitudinal model that accommodates random subject effects and zero inflation. Both raw and imputed data show a significant drug effect for reducing the odds of non-abstinence and the number of cigarettes smoked among non-abstainers, but the imputed data provide efficiency gains. This method permits the analysis of daily cigarette consumption data previously deemed suspect due to reporting error and is applicable to other self-reported count data sets for which calibration samples are available. Copyright © 2015 John Wiley & Sons, Ltd.

Posterior predictive checking of multiple imputation models

Multiple imputation is gaining popularity as a strategy for handling missing data, but there is a scarcity of tools for checking imputation models, a critical step in model fitting. Posterior predictive checking (PPC) has been recommended as an imputation diagnostic. PPC involves simulating "replicated" data from the posterior predictive distribution of the model under scrutiny. Model fit is assessed by examining whether the analysis from the observed data appears typical of results obtained from the replicates produced by the model. A proposed diagnostic measure is the posterior predictive "p-value", an extreme value of which (i.e., a value close to 0 or 1) suggests a misfit between the model and the data. The aim of this study was to evaluate the performance of the posterior predictive p-value as an imputation diagnostic. Using simulation methods, we deliberately misspecified imputation models to determine whether posterior predictive p-values were effective in identifying these problems. When estimating the regression parameter of interest, we found that more extreme p-values were associated with poorer imputation model performance, although the results highlighted that traditional thresholds for classical p-values do not apply in this context. A shortcoming of the PPC method was its reduced ability to detect misspecified models with increasing amounts of missing data. Despite the limitations of posterior predictive p-values, they appear to have a valuable place in the imputer's toolkit. In addition to automated checking using p-values, we recommend imputers perform graphical checks and examine other summaries of the test quantity distribution.

The rise of multiple imputation: a review of the reporting and implementation of the method in medical research

Background

Missing data are common in medical research, which can lead to a loss in statistical power and potentially biased results if not handled appropriately. Multiple imputation (MI) is a statistical method, widely adopted in practice, for dealing with missing data. Many academic journals now emphasise the importance of reporting information regarding missing data and proposed guidelines for documenting the application of MI have been published. This review evaluated the reporting of missing data, the application of MI including the details provided regarding the imputation model, and the frequency of sensitivity analyses within the MI framework in medical research articles.

Methods

A systematic review of articles published in the Lancet and New England Journal of Medicine between January 2008 and December 2013 in which MI was implemented was carried out.

Results

We identified 103 papers that used MI, with the number of papers increasing from 11 in 2008 to 26 in 2013. Nearly half of the papers specified the proportion of complete cases or the proportion with missing data by each variable. In the majority of the articles (86%) the imputed variables were specified. Of the 38 papers (37%) that stated the method of imputation, 20 used chained equations, 8 used multivariate normal imputation, and 10 used alternative methods. Very few articles (9%) detailed how they handled non-normally distributed variables during imputation. Thirty-nine papers (38%) stated the variables included in the imputation model. Less than half of the papers (46%) reported the number of imputations, and only two papers compared the distribution of imputed and observed data. Sixty-six papers presented the results from MI as a secondary analysis. Only three articles carried out a sensitivity analysis following MI to assess departures from the missing at random assumption, with details of the sensitivity analyses only provided by one article.

Conclusions

This review outlined deficiencies in the documenting of missing data and the details provided about imputation. Furthermore, only a few articles performed sensitivity analyses following MI even though this is strongly recommended in guidelines. Authors are encouraged to follow the available guidelines and provide information on missing data and the imputation process.

Electronic supplementary material

The online version of this article (doi:10.1186/s12874-015-0022-1) contains supplementary material, which is available to authorized users.

A multiple imputation strategy for sequential multiple assignment randomized trials

Sequential multiple assignment randomized trials (SMARTs) are increasingly being used to inform clinical and intervention science. In a SMART, each patient is repeatedly randomized over time. Each randomization occurs at a critical decision point in the treatment course. These critical decision points often correspond to milestones in the disease process or other changes in a patient's health status. Thus, the timing and number of randomizations may vary across patients and depend on evolving patient-specific information. This presents unique challenges when analyzing data from a SMART in the presence of missing data. This paper presents the first comprehensive discussion of missing data issues typical of SMART studies: we describe five specific challenges and propose a flexible imputation strategy to facilitate valid statistical estimation and inference using incomplete data from a SMART. To illustrate these contributions, we consider data from the Clinical Antipsychotic Trial of Intervention and Effectiveness, one of the most well-known SMARTs to date.

Model selection and diagnostics for joint modeling of survival and longitudinal data with crossing hazard rate functions

Comparison of two hazard rate functions is important for evaluating treatment effect in studies concerning times to some important events. In practice, it may happen that the two hazard rate functions cross each other at one or more unknown time points, representing temporal changes of the treatment effect. Also, besides survival data, there could be longitudinal data available regarding some time-dependent covariates. When jointly modeling the survival and longitudinal data in such cases, model selection and model diagnostics are especially important to provide reliable statistical analysis of the data, which are lacking in the literature. In this paper, we discuss several criteria for assessing model fit that have been used for model selection and apply them to the joint modeling of survival and longitudinal data for comparing two crossing hazard rate functions. We also propose hypothesis testing and graphical methods for model diagnostics of the proposed joint modeling approach. Our proposed methods are illustrated by a simulation study and by a real-data example concerning two early breast cancer treatments.

Testing the difficulty theory of the SON-R 5(1/2)-17, a non-verbal test of intelligence

Fischer's (1973) linear logistic test model can be used to test hypotheses regarding the effect of covariates on item difficulty and to predict the difficulty of newly constructed test items. However, its assumptions of equal discriminatory power across items and a perfect prediction of item difficulty are never absolutely met. The amount of misfit in an application of a Bayesian version of the model to two subtests of the SON-R 5(1/2)-17 is investigated by means of item fit statistics in the framework of posterior predictive checks and by means of a comparison with a model that allows for residual (co)variance in the item parameters. The effect of the degree of residual (co)variance on the robustness of inferences is investigated in a simulation study.

Diagnosing problems with imputation models using the Kolmogorov-Smirnov test: a simulation study

Background

Multiple imputation (MI) is becoming increasingly popular as a strategy for handling missing data, but there is a scarcity of tools for checking the adequacy of imputation models. The Kolmogorov-Smirnov (KS) test has been identified as a potential diagnostic method for assessing whether the distribution of imputed data deviates substantially from that of the observed data. The aim of this study was to evaluate the performance of the KS test as an imputation diagnostic.

Methods

Using simulation, we examined whether the KS test could reliably identify departures from assumptions made in the imputation model. To do this we examined how the p-values from the KS test behaved when skewed and heavy-tailed data were imputed using a normal imputation model. We varied the amount of missing data, the missing data models and the amount of skewness, and evaluated the performance of KS test in diagnosing issues with the imputation models under these different scenarios.

Results

The KS test was able to flag differences between the observations and imputed values; however, these differences did not always correspond to problems with MI inference for the regression parameter of interest. When there was a strong missing at random dependency, the KS p-values were very small, regardless of whether or not the MI estimates were biased; so that the KS test was not able to discriminate between imputed variables that required further investigation, and those that did not. The p-values were also sensitive to sample size and the proportion of missing data, adding to the challenge of interpreting the results from the KS test.

Conclusions

Given our study results, it is difficult to establish guidelines or recommendations for using the KS test as a diagnostic tool for MI. The investigation of other imputation diagnostics and their incorporation into statistical software are important areas for future research.

Bayesian model selection for incomplete data using the posterior predictive distribution

We explore the use of a posterior predictive loss criterion for model selection for incomplete longitudinal data. We begin by identifying a property that most model selection criteria for incomplete data should consider. We then show that a straightforward extension of the Gelfand and Ghosh (1998, Biometrika, 85, 1-11) criterion to incomplete data has two problems. First, it introduces an extra term (in addition to the goodness of fit and penalty terms) that compromises the criterion. Second, it does not satisfy the aforementioned property. We propose an alternative and explore its properties via simulations and on a real dataset and compare it to the deviance information criterion (DIC). In general, the DIC outperforms the posterior predictive criterion, but the latter criterion appears to work well overall and is very easy to compute unlike the DIC in certain classes of models for missing data.

Combining patient-level and summary-level data for Alzheimer's disease modeling and simulation: a β regression meta-analysis

Our objective was to develop a beta regression (BR) model to describe the longitudinal progression of the 11 item Alzheimer's disease (AD) assessment scale cognitive subscale (ADAS-cog) in AD patients in both natural history and randomized clinical trial settings, utilizing both individual patient and summary level literature data. Patient data from the coalition against major diseases database (3,223 patients), the Alzheimer's disease neruroimaging initiative study database (186 patients), and summary data from 73 literature references (representing 17,235 patients) were fit to a BR drug-disease-trial model. Treatment effects for currently available acetyl cholinesterase inhibitors, longitudinal changes in disease severity, dropout rate, placebo effect, and factors influencing these parameters were estimated in the model. Based on predictive checks and external validation, an adequate BR meta-analysis model for ADAS-cog using both summary-level and patient-level data was developed. Baseline ADAS-cog was estimated from baseline MMSE score. Disease progression was dependent on time, ApoE4 status, age, and gender. Study drop out was a function of time, baseline age, and baseline MMSE. The use of the BR constrained simulations to the 0-70 range of the ADAS-cog, even when residuals were incorporated. The model allows for simultaneous fitting of summary and patient level data, allowing for integration of all information available. A further advantage of the BR model is that it constrains values to the range of the original instrument for simulation purposes, in contrast to methodologies that provide appropriate constraints only for conditional expectations.

Truth and Memory: Linking Instantaneous and Retrospective Self-Reported Cigarette Consumption

Studies of smoking behavior commonly use the time-line follow-back (TLFB) method, or periodic retrospective recall, to gather data on daily cigarette consumption. TLFB is considered adequate for identifying periods of abstinence and lapse but not for measurement of daily cigarette consumption, thanks to substantial recall and digit preference biases. With the development of the hand-held electronic diary (ED), it has become possible to collect cigarette consumption data using ecological momentary assessment (EMA), or the instantaneous recording of each cigarette as it is smoked. EMA data, because they do not rely on retrospective recall, are thought to more accurately measure cigarette consumption. In this article we present an analysis of consumption data collected simultaneously by both methods from 236 active smokers in the pre-quit phase of a smoking cessation study. We define a statistical model that describes the genesis of the TLFB records as a two-stage process of mis-remembering and rounding, including fixed and random effects at each stage. We use Bayesian methods to estimate the model, and we evaluate its adequacy by studying histograms of imputed values of the latent remembered cigarette count. Our analysis suggests that both mis-remembering and heaping contribute substantially to the distortion of self-reported cigarette counts. Higher nicotine dependence, white ethnicity and male sex are associated with greater remembered smoking given the EMA count. The model is potentially useful in other applications where it is desirable to understand the process by which subjects remember and report true observations.

Diagnosing imputation models by applying target analyses to posterior replicates of completed data

Multiple imputation fills in missing data with posterior predictive draws from imputation models. To assess the adequacy of imputation models, we can compare completed data with their replicates simulated under the imputation model. We apply analyses of substantive interest to both datasets and use posterior predictive checks of the differences of these estimates to quantify the evidence of model inadequacy. We can further integrate out the imputed missing data and their replicates over the completed-data analyses to reduce variance in the comparison. In many cases, the checking procedure can be easily implemented using standard imputation software by treating re-imputations under the model as posterior predictive replicates. Thus, it can be applied for non-Bayesian imputation methods. We also sketch several strategies for applying the method in the context of practical imputation analyses. We illustrate the method using two real data applications and study its property using a simulation.

A marginalized conditional linear model for longitudinal binary data when informative dropout occurs in continuous time

Within the pattern-mixture modeling framework for informative dropout, conditional linear models (CLMs) are a useful approach to deal with dropout that can occur at any point in continuous time (not just at observation times). However, in contrast with selection models, inferences about marginal covariate effects in CLMs are not readily available if nonidentity links are used in the mean structures. In this article, we propose a CLM for long series of longitudinal binary data with marginal covariate effects directly specified. The association between the binary responses and the dropout time is taken into account by modeling the conditional mean of the binary response as well as the dependence between the binary responses given the dropout time. Specifically, parameters in both the conditional mean and dependence models are assumed to be linear or quadratic functions of the dropout time; and the continuous dropout time distribution is left completely unspecified. Inference is fully Bayesian. We illustrate the proposed model using data from a longitudinal study of depression in HIV-infected women, where the strategy of sensitivity analysis based on the extrapolation method is also demonstrated.

Gaussian-based routines to impute categorical variables in health surveys

The multivariate normal (MVN) distribution is arguably the most popular parametric model used in imputation and is available in most software packages (e.g., SAS PROC MI, R package norm). When it is applied to categorical variables as an approximation, practitioners often either apply simple rounding techniques for ordinal variables or create a distinct 'missing' category and/or disregard the nominal variable from the imputation phase. All of these practices can potentially lead to biased and/or uninterpretable inferences. In this work, we develop a new rounding methodology calibrated to preserve observed distributions to multiply impute missing categorical covariates. The major attractiveness of this method is its flexibility to use any 'working' imputation software, particularly those based on MVN, allowing practitioners to obtain usable imputations with small biases. A simulation study demonstrates the clear advantage of the proposed method in rounding ordinal variables and, in some scenarios, its plausibility in imputing nominal variables. We illustrate our methods on a widely used National Survey of Children with Special Health Care Needs where incomplete values on race posed a valid threat on inferences pertaining to disparities.

Informing sequential clinical decision-making through reinforcement learning: an empirical study

This paper highlights the role that reinforcement learning can play in the optimization of treatment policies for chronic illnesses. Before applying any off-the-shelf reinforcement learning methods in this setting, we must first tackle a number of challenges. We outline some of these challenges and present methods for overcoming them. First, we describe a multiple imputation approach to overcome the problem of missing data. Second, we discuss the use of function approximation in the context of a highly variable observation set. Finally, we discuss approaches to summarizing the evidence in the data for recommending a particular action and quantifying the uncertainty around the Q-function of the recommended policy. We present the results of applying these methods to real clinical trial data of patients with schizophrenia.

Missing data analysis using multiple imputation: getting to the heart of the matter

Missing data are a pervasive problem in health investigations. We describe some background of missing data analysis and criticize ad hoc methods that are prone to serious problems. We then focus on multiple imputation, in which missing cases are first filled in by several sets of plausible values to create multiple completed datasets, then standard complete-data procedures are applied to each completed dataset, and finally the multiple sets of results are combined to yield a single inference. We introduce the basic concepts and general methodology and provide some guidance for application. For illustration, we use a study assessing the effect of cardiovascular diseases on hospice discussion for late stage lung cancer patients.

Impact of non-normal random effects on inference by multiple imputation: A simulation assessment

Multivariate extensions of well-known linear mixed-effects models have been increasingly utilized in inference by multiple imputation in the analysis of multilevel incomplete data. The normality assumption for the underlying error terms and random effects plays a crucial role in simulating the posterior predictive distribution from which the multiple imputations are drawn. The plausibility of this normality assumption on the subject-specific random effects is assessed. Specifically, the performance of multiple imputation created under a multivariate linear mixed-effects model is investigated on a diverse set of incomplete data sets simulated under varying distributional characteristics. Under moderate amounts of missing data, the simulation study confirms that the underlying model leads to a well-calibrated procedure with negligible biases and actual coverage rates close to nominal rates in estimates of the regression coefficients. Estimation quality of the random-effect variance and association measures, however, are negatively affected from both the misspecification of the random-effect distribution and number of incompletely-observed variables. Some of the adverse impacts include lower coverage rates and increased biases.

Multiple imputation in a large-scale complex survey: a practical guide

The Cancer Care Outcomes Research and Surveillance (CanCORS) Consortium is a multisite, multimode, multiwave study of the quality and patterns of care delivered to population-based cohorts of newly diagnosed patients with lung and colorectal cancer. As is typical in observational studies, missing data are a serious concern for CanCORS, following complicated patterns that impose severe challenges to the consortium investigators. Despite the popularity of multiple imputation of missing data, its acceptance and application still lag in large-scale studies with complicated data sets such as CanCORS. We use sequential regression multiple imputation, implemented in public-available software, to deal with non-response in the CanCORS surveys and construct a centralised completed database that can be easily used by investigators from multiple sites. Our work illustrates the feasibility of multiple imputation in a large-scale multiobjective survey, showing its capacity to handle complex missing data. We present the implementation process in detail as an example for practitioners and discuss some of the challenging issues which need further research.

Joint Models for the Association of Longitudinal Binary and Continuous Processes With Application to a Smoking Cessation Trial

Joint models for the association of a longitudinal binary and a longitudinal continuous process are proposed for situations in which their association is of direct interest. The models are parameterized such that the dependence between the two processes is characterized by unconstrained regression coefficients. Bayesian variable selection techniques are used to parsimoniously model these coefficients. A Markov chain Monte Carlo (MCMC) sampling algorithm is developed for sampling from the posterior distribution, using data augmentation steps to handle missing data. Several technical issues are addressed to implement the MCMC algorithm efficiently. The models are motivated by, and are used for, the analysis of a smoking cessation clinical trial in which an important question of interest was the effect of the (exercise) treatment on the relationship between smoking cessation and weight gain.