Specification of random effects in multilevel models: a review

An Investigation of Factored Regression Missing Data Methods for Multilevel Models with Cross-Level Interactions

A growing body of literature has focused on missing data methods that factorize the joint distribution into a part representing the analysis model of interest and a part representing the distributions of the incomplete predictors. Relatively little is known about the utility of this method for multilevel models with interactive effects. This study presents a series of Monte Carlo computer simulations that investigates Bayesian and multiple imputation strategies based on factored regressions. When the model's distributional assumptions are satisfied, these methods generally produce nearly unbiased estimates and good coverage, with few exceptions. Severe misspecifications that arise from substantially non-normal distributions can introduce biased estimates and poor coverage. Follow-up simulations suggest that a Yeo-Johnson transformation can mitigate these biases. A real data example illustrates the methodology, and the paper suggests several avenues for future research.

A highly replicable decline in mood during rest and simple tasks

Does our mood change as time passes? This question is central to behavioural and affective science, yet it remains largely unexamined. To investigate, we intermixed subjective momentary mood ratings into repetitive psychology paradigms. Here we demonstrate that task and rest periods lowered participants' mood, an effect we call 'Mood Drift Over Time'. This finding was replicated in 19 cohorts totalling 28,482 adult and adolescent participants. The drift was relatively large (-13.8% after 7.3 min of rest, Cohen's d = 0.574) and was consistent across cohorts. Behaviour was also impacted: participants were less likely to gamble in a task that followed a rest period. Importantly, the drift slope was inversely related to reward sensitivity. We show that accounting for time using a linear term significantly improves the fit of a computational model of mood. Our work provides conceptual and methodological reasons for researchers to account for time's effects when studying mood and behaviour.

Predictions of machine learning with mixed-effects in analyzing longitudinal data under model misspecification

We consider predictions in longitudinal studies, and investigate the well known statistical mixed-effects model, piecewise linear mixed-effects model and six different popular machine learning approaches: decision trees, bagging, random forest, boosting, support-vector machine and neural network. In order to consider the correlated data in machine learning, the random effects is combined into the traditional tree methods and random forest. Our focus is the performance of statistical modelling and machine learning especially in the cases of the misspecification of the fixed effects and the random effects. Extensive simulation studies have been carried out to evaluate the performance using a number of criteria. Two real datasets from longitudinal studies are analysed to demonstrate our findings. The R code and dataset are freely available at https://github.com/shuwen92/MEML.

Addressing overdispersion and zero-inflation for clustered count data via new multilevel heterogenous hurdle models

Unobserved heterogeneity causing overdispersion and the excessive number of zeros take a prominent place in the methodological development on count modeling. An insight into the mechanisms that induce heterogeneity is required for better understanding of the phenomenon of overdispersion. When the heterogeneity is sourced by the stochastic component of the model, the use of a heterogenous Poisson distribution for this part encounters as an elegant solution. Hierarchical design of the study is also responsible for the heterogeneity as the unobservable effects at various levels also contribute to the overdispersion. Zero-inflation, heterogeneity and multilevel nature in the count data present special challenges in their own respect, however the presence of all in one study adds more challenges to the modeling strategies. This study therefore is designed to merge the attractive features of the separate strand of the solutions in order to face such a comprehensive challenge. This study differs from the previous attempts by the choice of two recently developed heterogeneous distributions, namely Poisson-Lindley (PL) and Poisson-Ailamujia (PA) for the truncated part. Using generalized linear mixed modeling settings, predictive performances of the multilevel PL and PA models and their hurdle counterparts were assessed within a comprehensive simulation study in terms of bias, precision and accuracy measures. Multilevel models were applied to two separate real world examples for the assessment of practical implications of the new models proposed in this study.

Using a mixed-effect model with a parameter-space of heterogenous dimension to evaluate whether accountable care organizations are associated with greater uniformity across constituent practices

Accountable care organization (ACO) legislation was designed to improve patient outcomes by inducing greater coordination of care and adoption of best practices. Therefore, it is of interest to assess whether greater uniformity occurs among practices comprising an ACO post ACO formation. We develop a mixed-effect model with a difference-in-difference design to evaluate the effect of a patient receiving care from an ACO on patient outcomes and adapt this model to examine whether an ACO is associated with increased uniformity across its constituent practices. The task is complicated by the organizations within an ACO forming an additional layer in the multilevel model, due to medical practices and hospitals that form an ACOs being nested within the ACO, making the number of levels of the model variable and the dimension of the parameter space time-varying. We develop the model and a procedure for testing the hypothesis that ACO formation was associated with increased uniformity among its constituent practices. We apply our procedure to a cohort of medicare beneficiaries followed over 2009-2014. Although there is extensive heterogeneity of becoming an ACOs across practices, we find that the formation of an ACO appears to be associated with greater uniformity of patient outcomes among its constituent practices.

A reaction norm perspective on reproducibility

Reproducibility in biomedical research, and more specifically in preclinical animal research, has been seriously questioned. Several cases of spectacular failures to replicate findings published in the primary scientific literature have led to a perceived reproducibility crisis. Diverse threats to reproducibility have been proposed, including lack of scientific rigour, low statistical power, publication bias, analytical flexibility and fraud. An important aspect that is generally overlooked is the lack of external validity caused by rigorous standardization of both the animals and the environment. Here, we argue that a reaction norm approach to phenotypic variation, acknowledging gene-by-environment interactions, can help us seeing reproducibility of animal experiments in a new light. We illustrate how dominating environmental effects can affect inference and effect size estimates of studies and how elimination of dominant factors through standardization affects the nature of the expected phenotype variation through the reaction norms of small effect. Finally, we discuss the consequences of reaction norms of small effect for statistical analysis, specifically for random effect latent variable models and the random lab model.

Effect Partitioning in Cross-Sectionally Clustered Data Without Multilevel Models

Effect partitioning is almost exclusively performed with multilevel models (MLMs) - so much so that some have considered the two to be synonymous. MLMs are able to provide estimates with desirable statistical properties when data come from a hierarchical structure; but the random effects included in MLMs are not always integral to the analysis. As a result, other methods with relaxed assumptions are viable options in many cases. Through empirical examples and simulations, we show how generalized estimating equations (GEEs) can be used to effectively partition effects without random effects. We show that more onerous steps of MLMs such as determining the number of random effects and the structure for their covariance can be bypassed with GEEs while still obtaining identical or near-identical results. Additionally, violations of distributional assumptions adversely affect estimates with MLMs but have no effect on GEEs because no such assumptions are made. This makes GEEs a flexible alternative to MLMs with minimal assumptions that may warrant consideration. Limitations of GEEs for partitioning effects are also discussed.

Variation between Hospitals with Regard to Diagnostic Practice, Coding Accuracy, and Case-Mix. A Retrospective Validation Study of Administrative Data versus Medical Records for Estimating 30-Day Mortality after Hip Fracture

BACKGROUND

The purpose of this study was to assess the validity of patient administrative data (PAS) for calculating 30-day mortality after hip fracture as a quality indicator, by a retrospective study of medical records.

METHODS

We used PAS data from all Norwegian hospitals (2005-2009), merged with vital status from the National Registry, to calculate 30-day case-mix adjusted mortality for each hospital (n = 51). We used stratified sampling to establish a representative sample of both hospitals and cases. The hospitals were stratified according to high, low and medium mortality of which 4, 3, and 5 hospitals were sampled, respectively. Within hospitals, cases were sampled stratified according to year of admission, age, length of stay, and vital 30-day status (alive/dead). The final study sample included 1043 cases from 11 hospitals. Clinical information was abstracted from the medical records. Diagnostic and clinical information from the medical records and PAS were used to define definite and probable hip fracture. We used logistic regression analysis in order to estimate systematic between-hospital variation in unmeasured confounding. Finally, to study the consequences of unmeasured confounding for identifying mortality outlier hospitals, a sensitivity analysis was performed.

RESULTS

The estimated overall positive predictive value was 95.9% for definite and 99.7% for definite or probable hip fracture, with no statistically significant differences between hospitals. The standard deviation of the additional, systematic hospital bias in mortality estimates was 0.044 on the logistic scale. The effect of unmeasured confounding on outlier detection was small to moderate, noticeable only for large hospital volumes.

CONCLUSIONS

This study showed that PAS data are adequate for identifying cases of hip fracture, and the effect of unmeasured case mix variation was small. In conclusion, PAS data are adequate for calculating 30-day mortality after hip-fracture as a quality indicator in Norway.