Effects of covariance model assumptions on hypothesis tests for repeated measurements: analysis of ovarian hormone data and pituitary-pteryomaxillary distance data

Power and Sample Size for Fixed-Effects Inference in Reversible Linear Mixed Models

Despite the popularity of the general linear mixed model for data analysis, power and sample size methods and software are not generally available for commonly used test statistics and reference distributions. Statisticians resort to simulations with homegrown and uncertified programs or rough approximations which are misaligned with the data analysis. For a wide range of designs with longitudinal and clustering features, we provide accurate power and sample size approximations for inference about fixed effects in linear models we call reversible. We show that under widely applicable conditions, the general linear mixed-model Wald test has non-central distributions equivalent to well-studied multivariate tests. In turn, exact and approximate power and sample size results for the multivariate Hotelling-Lawley test provide exact and approximate power and sample size results for the mixed-model Wald test. The calculations are easily computed with a free, open-source product that requires only a web browser to use. Commercial software can be used for a smaller range of reversible models. Simple approximations allow accounting for modest amounts of missing data. A real-world example illustrates the methods. Sample size results are presented for a multicenter study on pregnancy. The proposed study, an extension of a funded project, has clustering within clinic. Exchangeability among participants allows averaging across them to remove the clustering structure. The resulting simplified design is a single level longitudinal study. Multivariate methods for power provide an approximate sample size. All proofs and inputs for the example are in the Supplementary Materials (available online).

Simultaneous small-sample comparisons in longitudinal or multi-endpoint trials using multiple marginal models

Simultaneous inference in longitudinal, repeated-measures, and multi-endpoint designs can be onerous, especially when trying to find a reasonable joint model from which the interesting effects and covariances are estimated. A novel statistical approach known as multiple marginal models greatly simplifies the modelling process: the core idea is to "marginalise" the problem and fit multiple small models to different portions of the data, and then estimate the overall covariance matrix in a subsequent, separate step. Using these estimates guarantees strong control of the family-wise error rate, however only asymptotically. In this paper, we show how to make the approach also applicable to small-sample data problems. Specifically, we discuss the computation of adjusted P values and simultaneous confidence bounds for comparisons of randomised treatment groups as well as for levels of a nonrandomised factor such as multiple endpoints, repeated measures, or a series of points in time or space. We illustrate the practical use of the method with a data example.

The effect of correlation structure on treatment contrasts estimated from incomplete clinical trial data with likelihood-based repeated measures compared with last observation carried forward ANOVA

Valid analyses of longitudinal data can be problematic, particularly when subjects dropout prior to completing the trial for reasons related to the outcome. Regulatory agencies often favor the last observation carried forward (LOCF) approach for imputing missing values in the primary analysis of clinical trials. However, recent evidence suggests that likelihood-based analyses developed under the missing at random framework provide viable alternatives. The within-subject error correlation structure is often the means by which such methods account for the bias from missing data. The objective of this study was to extend previous work that used only one correlation structure by including several common correlation structures in order to assess the effect of the correlation structure in the data, and how it is modeled, on Type I error rates and power from a likelihood-based repeated measures analysis (MMRM), using LOCF for comparison. Data from four realistic clinical trial scenarios were simulated using autoregressive, compound symmetric and unstructured correlation structures. When the correct correlation structure was fit, MMRM provided better control of Type I error and power than LOCF. Although misfitting the correlation structure in MMRM inflated Type I error and altered power, misfitting the structure was typically less deleterious than using LOCF. In fact, simply specifying an unstructured matrix for use in MMRM, regardless of the true correlation structure, yielded superior control of Type I error than LOCF in every scenario. The present and previous investigations have shown that the bias in LOCF is influenced by several factors and interactions between them. Hence, it is difficult to precisely anticipate the direction and magnitude of bias from LOCF in practical situations. However, in scenarios where the overall tendency is for patient improvement, LOCF tends to: 1) overestimate a drug's advantage when dropout is higher in the comparator and underestimate the advantage when dropout is lower in the comparator; 2) overestimate a drug's advantage when the advantage is maximum at intermediate time points and underestimate the advantage when the advantage increases over time; and 3) have a greater likelihood of overestimating a drug's advantage when the advantage is small. In scenarios in which the overall tendency is for patient worsening, the above biases are reversed. In the simulation scenarios considered in this study, which were patterned after acute phase neuropsychiatric clinical trials, the likelihood-based repeated measures approach, implemented with standard software, was more robust to the bias from missing data than LOCF, and choice of correlation structure was not an impediment to its implementation.

A strategy for selecting among alternative models for continuous longitudinal data

Linear mixed models (LMMs) can be used to analyze continuous longitudinal response variables of research studies. Specific aims are then addressed through tests of fixed effects comparing means. However, generated fixed effects results can vary according to the choice of the covariance structure, and so strategies for selecting a model should be utilized first to identify an appropriate covariance structure. We describe alternative LMMs for analyzing continuous longitudinal data and discuss a strategy for using model selection criteria to choose among those models. This is accomplished through the analysis of an exemplar data set considering a wide variety of alternative models for the means, variances, and correlations.

On summary measure analysis of linear trend repeated measures data: performance comparison with two competing methods

Background

The summary measure approach (SMA) is sometimes the only applicable tool for the analysis of repeated measurements in medical research, especially when the number of measurements is relatively large. This study aimed to describe techniques based on summary measures for the analysis of linear trend repeated measures data and then to compare performances of SMA, linear mixed model (LMM), and unstructured multivariate approach (UMA).

Methods

Practical guidelines based on the least squares regression slope and mean of response over time for each subject were provided to test time, group, and interaction effects. Through Monte Carlo simulation studies, the efficacy of SMA vs. LMM and traditional UMA, under different types of covariance structures, was illustrated. All the methods were also employed to analyze two real data examples.

Results

Based on the simulation and example results, it was found that the SMA completely dominated the traditional UMA and performed convincingly close to the best-fitting LMM in testing all the effects. However, the LMM was not often robust and led to non-sensible results when the covariance structure for errors was misspecified. The results emphasized discarding the UMA which often yielded extremely conservative inferences as to such data.

Conclusions

It was shown that summary measure is a simple, safe and powerful approach in which the loss of efficiency compared to the best-fitting LMM was generally negligible. The SMA is recommended as the first choice to reliably analyze the linear trend data with a moderate to large number of measurements and/or small to moderate sample sizes.

Using adaptive tests for the analysis of repeated measurements

Several methods have been proposed for the analysis of data obtained from experimental units that are observed at multiple time points. In this paper we evaluate the performance of an adaptive test for the interaction of group and time, and an adaptive test for the group effect with data sets having measurements at common time points on two groups of experimental units. The results from extensive simulation studies show that the adaptive tests maintain their level of significance and are often more powerful than the corresponding likelihood ratio and mixed model tests when the error distributions and the random effect distributions are nonnormal. When the distributions are normal, the adaptive tests are nearly as powerful as the other tests.

Hormone levels before and after tubal sterilization

OBJECTIVE

The aim of this study was to determine whether women experience significant luteal phase hormonal changes following interval tubal sterilization.

DESIGN

This is a partly randomized, prospective clinical study.

SETTING

This study involved healthy volunteers in an academic research environment.

PATIENTS

This study involved 118 fertile women seeking tubal sterilization and 57 fertile controls with at least three normal cyclic menstrual periods before entry into the study.

INTERVENTIONS

The patients were randomized to bipolar cautery or Hulka clip as sterilization methods. Barrier contraception or abstinence was used by controls.

MAIN OUTCOME MEASURES

The main outcome measures are serum estradiol and progesterone levels and urinary estradiol and pregnanediol levels obtained during the luteal phase before, 1 year and 2 years after sterilization.

RESULTS

The women randomized to the bipolar cautery group had higher midluteal progesterone levels measured between Days 5 and 11 postovulation (15.5 ng/mL before sterilization, 14.5 ng/mL at 1 year and 14.5 ng/mL at 2 years) than did the other two groups. The clip group had progesterone levels of 14.1, 12.0 and 12.5 ng/mL at baseline, 1 year and 2 years, respectively, and the control group had levels of 12.0, 11.9 and 11.3 ng/mL for the same periods. Serum estradiol and progesterone and urinary pregnanediol and estradiol were not significantly changed over the 2-year period, nor were there significant differences between the two groups.

CONCLUSIONS

There were no significant hormonal changes in sterilized women over a period of 2 years when compared with their baseline levels or when compared with unsterilized age-matched controls.

Limitations of Ordinary Least Squares Models in Analyzing Repeated Measures Data

PURPOSE

To a) introduce and present the advantages of linear mixed models using generalized least squares (GLS) when analyzing repeated measures data; and b) show how model misspecification and an inappropriate analysis using repeated measures ANOVA with ordinary least squares (OLS) methodology can negatively impact the probability of occurrence of Type I error.

METHODS

The effects of three strength-training groups were simulated. Strength gains had two slope conditions: null (no gain), and moderate (moderate gain). Ten subjects were hypothetically measured at five time points, and the correlation between measurements within a subject was modeled as compound symmetric (CS), autoregressive lag 1 (AR(1)), and random coefficients (RC). A thousand data sets were generated for each correlation structure. Then, each was analyzed four times--once using OLS, and three times using GLS, assuming the following variance/covariance structures: CS, AR(1), and RC.

RESULTS

OLS produced substantially inflated probabilities of Type I errors when the variance/covariance structure of the data set was not CS. The RC model was less affected by the actual variance/covariance structure of the data set, and gave good estimates across all conditions.

CONCLUSIONS

Using OLS to analyze repeated measures data is inappropriate when the covariance structure is not known to be CS. Random coefficients growth curve models may be useful when the variance/covariance structure of the data set is unknown.