Some new algorithms for computing restricted maximum likelihood estimates of variance components

MM Algorithms For Variance Components Models

Variance components estimation and mixed model analysis are central themes in statistics with applications in numerous scientific disciplines. Despite the best efforts of generations of statisticians and numerical analysts, maximum likelihood estimation and restricted maximum likelihood estimation of variance component models remain numerically challenging. Building on the minorization-maximization (MM) principle, this paper presents a novel iterative algorithm for variance components estimation. Our MM algorithm is trivial to implement and competitive on large data problems. The algorithm readily extends to more complicated problems such as linear mixed models, multivariate response models possibly with missing data, maximum a posteriori estimation, and penalized estimation. We establish the global convergence of the MM algorithm to a Karush-Kuhn-Tucker (KKT) point and demonstrate, both numerically and theoretically, that it converges faster than the classical EM algorithm when the number of variance components is greater than two and all covariance matrices are positive definite.

Adaptive Fitting of Linear Mixed-Effects Models with Correlated Random-effects

Linear mixed-effects model has been widely used in longitudinal data analyses. In practice, the fitting algorithm can fail to converge due to boundary issues of the estimated random-effects covariance matrix G, i.e., being near-singular, non-positive definite, or both. Current available algorithms are not computationally optimal because the condition number of matrix G is unnecessarily increased when the random-effects correlation estimate is not zero. We propose an adaptive fitting (AF) algorithm using an optimal linear transformation of the random-effects design matrix. It is a data-driven adaptive procedure, aiming at reducing subsequent random-effects correlation estimates down to zero in the optimal transformed estimation space. Simulations show that AF significantly improves the convergent properties, especially under small sample size, relative large noise and high correlation settings. One real data for Insulin-like Growth Factor (IGF) protein is used to illustrate the application of this algorithm implemented with software package R (nlme).

Assessing a multiple QTL search using the variance component model

Development of variance component algorithms in genetics has previously mainly focused on animal breeding models or problems in human genetics with a simple data structure. We study alternative methods for constrained likelihood maximization in quantitative trait loci (QTL) analysis for large complex pedigrees. We apply a forward selection scheme to include several QTL and interaction effects, as well as polygenic effects, with up to five variance components in the model. We show that the implemented active set and primal-dual schemes result in accurate solutions and that they are robust. In terms of computational speed, a comparison of two approaches for approximating the Hessian of the log-likelihood shows that the method using an average information matrix is the method of choice for the five-dimensional problem. The active set method, with the average information method for Hessian computation, exhibits the fastest convergence with an average of 20 iterations per tested position, where the change in variance components <0.0001 was used as convergence criterion.

Closed-form approximations to the REML estimator of a variance ratio (or heritability) in a mixed linear model

In this article, we estimate heritability or intraclass correlation in a mixed linear model having two sources of variation. In most applications, the commonly used restricted maximum likelihood (REML) estimator can only be obtained via an iterative approach. In some cases, the algorithm used to compute REML estimates may be slow or may even fail to converge. We develop a set of closed-form approximations to the REML estimator, and the performance of these estimators is compared with that of the REML estimator. We provide guidelines regarding how to choose the estimator that best approximates the REML estimator. Examples presented in the article suggest that the closed-form estimators compete with and, in some cases, outperform the REML estimator.

Tests for gaussian repeated measures with missing data in small samples

For small samples of Gaussian repeated measures with missing data, Barton and Cramer recommended using the EM algorithm for estimation and reducing the degrees of freedom for an analogue of Rao's F approximation to Wilks' test. Computer simulations led to the conclusion that the modified test was slightly conservative for total sample size of N=40. Here we consider additional methods and smaller sample sizes, Nin¿12,24¿. We describe analogues of the Pillai-Bartlett trace, Hotelling-Lawley trace and Geisser-Greenhouse corrected univariate tests which allow for missing data. Eleven sample size adjustments were examined which replace N by some function of the numbers of non-missing pairs of responses in computing error degrees of freedom. Overall, simulation results allowed concluding that an adjusted test can always control test size at or below the nominal rate, even with as few as 12 observations and up to 10 per cent missing data. The choice of method varies with the test statistic. Replacing N by the mean number of non-missing responses per variable works best for the Geisser-Greenhouse test. The Pillai-Bartlett test requires the stronger adjustment of replacing N by the harmonic mean number of non-missing pairs of responses. For Wilks' and Hotelling-Lawley, an even more aggressive adjustment based on the minimum number of non-missing pairs must be used.

Model selection techniques for the covariance matrix for incomplete longitudinal data

In longitudinal studies with incomplete data, where the number of time points can become numerous, it is often advantageous to model the covariance matrix. We describe several covariance models (for example, mixed models, compound symmetry, AR(1)-type models, and combination models) that offer parsimonious alternatives to unstructured sigma. We evaluate each covariance model with longitudinal data concerning cholesterol as the repeated outcome measure. We discuss strategies for deciding the 'best' model and show a graphical technique for judging goodness-of-fit of covariance models.

Intentionally incomplete longitudinal designs: I. Methodology and comparison of some full span designs

Longitudinal designs are important in medical research and in many other disciplines. Complete longitudinal studies, in which each subject is evaluated at each measurement occasion, are often very expensive and motivate a search for more efficient designs. Recently developed statistical methods foster the use of intentionally incomplete longitudinal designs that have the potential to be more efficient than complete designs. Mixed models provide appropriate data analysis tools. Fixed effect hypotheses can be tested via a recently developed test statistic, FH. An accurate approximation of the statistic's small sample non-central distribution makes power computations feasible. After reviewing some longitudinal design terminology and mixed model notation, this paper summarizes the computation of FH and approximate power from its non-central distribution. These methods are applied to obtain a large number of intentionally incomplete full-span designs that are more powerful and/or less costly alternatives to a complete design. The source of the greater efficiency of incomplete designs and potential fragility of incomplete designs to randomly missing data are discussed.