The mle algorithm for the matrix normal distribution

TENSOR REGRESSION FOR INCOMPLETE OBSERVATIONS WITH APPLICATION TO LONGITUDINAL STUDIES

Multivariate longitudinal data are frequently encountered in practice such as in our motivating longitudinal microbiome study. It is of general interest to associate such high-dimensional, longitudinal measures with some univariate continuous outcome. However, incomplete observations are common in a regular study design, as not all samples are measured at every time point, giving rise to the so-called blockwise missing values. Such missing structure imposes significant challenges for association analysis and defies many existing methods that require complete samples. In this paper we propose to represent multivariate longitudinal data as a three-way tensor array (i.e., sample-by-feature-by-time) and exploit a parsimonious scalar-on-tensor regression model for association analysis. We develop a regularized covariance-based estimation procedure that effectively leverages all available observations without imputation. The method achieves variable selection and smooth estimation of time-varying effects. The application to the motivating microbiome study reveals interesting links between the preterm infant's gut microbiome dynamics and their neurodevelopment. Additional numerical studies on synthetic data and a longitudinal aging study further demonstrate the efficacy of the proposed method.

Procrustes Analysis for High-Dimensional Data

The Procrustes-based perturbation model (Goodall in J R Stat Soc Ser B Methodol 53(2):285-321, 1991) allows minimization of the Frobenius distance between matrices by similarity transformation. However, it suffers from non-identifiability, critical interpretation of the transformed matrices, and inapplicability in high-dimensional data. We provide an extension of the perturbation model focused on the high-dimensional data framework, called the ProMises (Procrustes von Mises-Fisher) model. The ill-posed and interpretability problems are solved by imposing a proper prior distribution for the orthogonal matrix parameter (i.e., the von Mises-Fisher distribution) which is a conjugate prior, resulting in a fast estimation process. Furthermore, we present the Efficient ProMises model for the high-dimensional framework, useful in neuroimaging, where the problem has much more than three dimensions. We found a great improvement in functional magnetic resonance imaging connectivity analysis because the ProMises model permits incorporation of topological brain information in the alignment's estimation process.

Simultaneous differential network analysis and classification for matrix-variate data with application to brain connectivity

Growing evidence has shown that the brain connectivity network experiences alterations for complex diseases such as Alzheimer's disease (AD). Network comparison, also known as differential network analysis, is thus particularly powerful to reveal the disease pathologies and identify clinical biomarkers for medical diagnoses (classification). Data from neurophysiological measurements are multidimensional and in matrix-form. Naive vectorization method is not sufficient as it ignores the structural information within the matrix. In the article, we adopt the Kronecker product covariance matrices framework to capture both spatial and temporal correlations of the matrix-variate data while the temporal covariance matrix is treated as a nuisance parameter. By recognizing that the strengths of network connections may vary across subjects, we develop an ensemble-learning procedure, which identifies the differential interaction patterns of brain regions between the case group and the control group and conducts medical diagnosis (classification) of the disease simultaneously. Simulation studies are conducted to assess the performance of the proposed method. We apply the proposed procedure to the functional connectivity analysis of an functional magnetic resonance imaging study on AD. The hub nodes and differential interaction patterns identified are consistent with existing experimental studies, and satisfactory out-of-sample classification performance is achieved for medical diagnosis of AD.

Quantitative matching of forensic evidence fragments utilizing 3D microscopy analysis of fracture surface replicas

Silicone casts are widely used by practitioners in the comparative analysis of forensic items. Fractured surfaces carry unique details that can provide accurate quantitative comparisons of forensic fragments. In this study, a statistical analysis comparison protocol was applied to a set of 3D topological images of fractured surface pairs and their replicas to provide confidence in the quantitative statistical comparison between fractured items and their silicone cast replicas. A set of 10 fractured stainless steel samples were fractured from the same metal rod under controlled conditions and were replicated using a standard forensic casting technique. Six 3D topological maps with 50% overlap were acquired for each fractured pair. Spectral analyses were utilized to identify the correlation between topological surface features at different length scales of the surface topology. We selected two frequency bands over the critical wavelength (greater than two-grain diameters) for statistical comparison. Our statistical model utilized a matrix-variate t-distribution that accounts for overlap between images to model match and non-match population densities. A decision rule identified the probability of matched and unmatched pairs of surfaces. The proposed methodology correctly classified the fractured steel surfaces and their replicas with a posterior probability of match exceeding 99.96%. Moreover, the replication technique shows potential in accurately replicating fracture surface topological details with a wavelength greater than 20 μm, which far exceeds the feature comparison range on most metallic alloy surfaces. Our framework establishes the basis and limits for forensic comparison of fractured articles and their replicas while providing a reliable fracture mechanics-based quantitative statistical forensic comparison.

Estimating High-Order Brain Functional Networks in Bayesian View for Autism Spectrum Disorder Identification

Brain functional network (BFN) has become an increasingly important tool to understand the inherent organization of the brain and explore informative biomarkers of neurological disorders. Pearson’s correlation (PC) is the most widely accepted method for constructing BFNs and provides a basis for designing new BFN estimation schemes. Particularly, a recent study proposes to use two sequential PC operations, namely, correlation’s correlation (CC), for constructing the high-order BFN. Despite its empirical effectiveness in identifying neurological disorders and detecting subtle changes of connections in different subject groups, CC is defined intuitively without a solid and sustainable theoretical foundation. For understanding CC more rigorously and providing a systematic BFN learning framework, in this paper, we reformulate it in the Bayesian view with a prior of matrix-variate normal distribution. As a result, we obtain a probabilistic explanation of CC. In addition, we develop a Bayesian high-order method (BHM) to automatically and simultaneously estimate the high- and low-order BFN based on the probabilistic framework. An efficient optimization algorithm is also proposed. Finally, we evaluate BHM in identifying subjects with autism spectrum disorder (ASD) from typical controls based on the estimated BFNs. Experimental results suggest that the automatically learned high- and low-order BFNs yield a superior performance over the artificially defined BFNs via conventional CC and PC.

Mean Equality Tests for High-Dimensional and Higher-Order Data with k-Self Similar Compound Symmetry Covariance Structure

An extension of the D2 test statistic to test the equality of mean for high-dimensional and k-th order array-variate data using k-self similar compound symmetry (k-SSCS) covariance structure is derived. The k-th order data appear in many scientific fields including agriculture, medical, environmental and engineering applications. We discuss the property of this k-SSCS covariance structure, namely, the property of Jordan algebra. We formally show that our D2 test statistic for k-th order data is an extension or the generalization of the D2 test statistic for second-order data and for third-order data, respectively. We also derive the D2 test statistic for third-order data and illustrate its application using a medical dataset from a clinical trial study of the eye disease glaucoma. The new test statistic is very efficient for high-dimensional data where the estimation of unstructured variance-covariance matrix is not feasible due to small sample size.

A Penalized Matrix Normal Mixture Model for Clustering Matrix Data

Along with advances in technology, matrix data, such as medical/industrial images, have emerged in many practical fields. These data usually have high dimensions and are not easy to cluster due to their intrinsic correlated structure among rows and columns. Most approaches convert matrix data to multi dimensional vectors and apply conventional clustering methods to them, and thus, suffer from an extreme high-dimensionality problem as well as a lack of interpretability of the correlated structure among row/column variables. Recently, a regularized model was proposed for clustering matrix-valued data by imposing a sparsity structure for the mean signal of each cluster. We extend their approach by regularizing further on the covariance to cope better with the curse of dimensionality for large size images. A penalized matrix normal mixture model with lasso-type penalty terms in both mean and covariance matrices is proposed, and then an expectation maximization algorithm is developed to estimate the parameters. The proposed method has the competence of both parsimonious modeling and reflecting the proper conditional correlation structure. The estimators are consistent, and their limiting distributions are derived. We applied the proposed method to simulated data as well as real datasets and measured its clustering performance with the clustering accuracy (ACC) and the adjusted rand index (ARI). The experiment results show that the proposed method performed better with higher ACC and ARI than those of conventional methods.

Tensor envelope mixture model for simultaneous clustering and multiway dimension reduction

In the form of multidimensional arrays, tensor data have become increasingly prevalent in modern scientific studies and biomedical applications such as computational biology, brain imaging analysis, and process monitoring system. These data are intrinsically heterogeneous with complex dependencies and structure. Therefore, ad-hoc dimension reduction methods on tensor data may lack statistical efficiency and can obscure essential findings. Model-based clustering is a cornerstone of multivariate statistics and unsupervised learning; however, existing methods and algorithms are not designed for tensor-variate samples. In this article, we propose a tensor envelope mixture model (TEMM) for simultaneous clustering and multiway dimension reduction of tensor data. TEMM incorporates tensor-structure-preserving dimension reduction into mixture modeling and drastically reduces the number of free parameters and estimative variability. An expectation-maximization-type algorithm is developed to obtain likelihood-based estimators of the cluster means and covariances, which are jointly parameterized and constrained onto a series of lower dimensional subspaces known as the tensor envelopes. We demonstrate the encouraging empirical performance of the proposed method in extensive simulation studies and a real data application in comparison with existing vector and tensor clustering methods.

Regularized matrix data clustering and its application to image analysis

We propose a novel regularized mixture model for clustering matrix-valued data. The proposed method assumes a separable covariance structure for each cluster and imposes a sparsity structure (eg, low rankness, spatial sparsity) for the mean signal of each cluster. We formulate the problem as a finite mixture model of matrix-normal distributions with regularization terms, and then develop an expectation maximization type of algorithm for efficient computation. In theory, we show that the proposed estimators are strongly consistent for various choices of penalty functions. Simulation and two applications on brain signal studies confirm the excellent performance of the proposed method including a better prediction accuracy than the competitors and the scientific interpretability of the solution.

Matrix-variate logistic regression with measurement error

Measurement error in covariates has been extensively studied in many conventional regression settings where covariate information is typically expressed in a vector form. However, there has been little work on error-prone matrix-variate data, which commonly arise from studies with imaging, spatial-temporal structures, etc. We consider analysis of error-contaminated matrix-variate data. We particularly focus on matrix-variate logistic measurement error models. We examine the biases induced from naive analysis which ignores measurement error in matrix-variate data. Two measurement error correction methods are developed to adjust for measurement error effects. The proposed methods are justified both theoretically and empirically. We analyse an electroencephalography dataset with the proposed methods.

Kinship Solutions for Partially Observed Multiphenotype Data

Current work for multivariate analysis of phenotypes in genome-wide association studies often requires that genetic similarity matrices be inverted or decomposed. This can be a computational bottleneck when many phenotypes are presented, each with a different missingness pattern. A usual method in this case is to perform decompositions on subsets of the kinship matrix for each phenotype, with each subset corresponding to the set of observed samples for that phenotype. We provide a new method for decomposing these kinship matrices that can reduce the computational complexity by an order of magnitude by propagating low-rank modifications along a tree spanning the phenotypes. We demonstrate that our method provides speed improvements of around 40% under reasonable conditions.

ML and MAP estimation of parameters for the Kalman filter and smoother applied to electrocardiographic imaging

In electrocardiographic imaging (ECGI), one solves the inverse problem of electrocardiography (ECG) to reconstruct equivalent cardiac sources based on the body surface potential measurements and a mathematical model of the torso. Due to attenuation and spatial smoothing within the torso, this inverse problem is ill-posed. Among many regularization approaches used in the ECG literature to overcome this ill-posedness, statistical techniques have received great attention because of their flexibility to represent the data, and ability to provide performance evaluation tools for quantification of uncertainties and errors in the model. However, despite their potential to accurately reconstruct the equivalent cardiac sources, one major challenge in these methods is how to best utilize the prior information available in terms of training data. In this paper, we address the question of how to define the prior probability distributions (pdf) of the sources and the error terms so that we can obtain more accurate and robust inverse solutions. We employ two methods, maximum likelihood (ML) and maximum a posteriori (MAP), for estimating the model parameters such as the prior pdfs, error pdfs, and the state-transition matrix, based on the same training data. These model parameters are then used for the state-space representation and estimation of the epicardial potentials, which constitute the equivalent cardiac sources in this study. The performances of ML- and MAP-based model parameter estimation methods are evaluated qualitatively and quantitatively at various noise levels and geometric disturbances using two different simulated datasets. Bayesian MAP estimation, which is also a well-known statistical inversion technique, and Tikhonov regularization, which can be formulated as a special and simplified version of Bayesian MAP estimation, have been included here for comparison with the Kalman filtering method. Our results show that the state-space approach outperforms Bayesian MAP estimation in all cases; ML yields accurate results when the test and training beats come from the same physiological model, but MAP is superior to ML, especially if the test and training beats are from different physiological models. Graphical Abstract ML and MAP estimation of parameters for the Kalman filter and smoother applied to electrocardiographic imaging.

Gaussian process emulation to accelerate parameter estimation in a mechanical model of the left ventricle: a critical step towards clinical end-user relevance

In recent years, we have witnessed substantial advances in the mathematical modelling of the biomechanical processes underlying the dynamics of the cardiac soft-tissue. Gao et al. (Gao et al. 2017 J. R. Soc. Interface 14, 20170203 ( doi:10.1098/rsif.2017.0203 )) demonstrated that the parameters underlying the biomechanical model have diagnostic value for prognosticating the risk of myocardial infarction. However, the computational costs of parameter estimation are prohibitive when the goal lies in building real-time clinical decision support systems. This is due to the need to repeatedly solve the mathematical equations numerically using finite-element discretization during an iterative optimization routine. The present article presents a method for accelerating the inference of the constitutive parameters by using statistical emulation with Gaussian processes. We demonstrate how the computational costs can be reduced by about three orders of magnitude, with hardly any loss in accuracy, and we assess various alternative techniques in a comparative evaluation study based on simulated data obtained by solving the left ventricular model with the finite-element method, and real magnetic resonance images data for a human volunteer.

Permutation based testing on covariance separability

Separability is an attractive feature of covariance matrices or matrix variate data, which can improve and simplify many multivariate procedures. Due to its importance, testing separability has attracted much attention in the past. The procedures in the literature are of two types, likelihood ratio test (LRT) and Rao's score test (RST). Both are based on the normality assumption or the large-sample asymptotic properties of the test statistics. In this paper, we develop a new approach that is very different from existing ones. We propose to reformulate the null hypothesis (the separability of a covariance matrix of interest) into many sub-hypotheses (the separability of the sub-matrices of the covariance matrix), which are testable using a permutation based procedure.We then combine the testing results of sub-hypotheses using the Bonferroni and two-stage additive procedures. Our permutation based procedures are inherently distribution free; thus it is robust to non-normality of the data. In addition, unlike the LRT, they are applicable to situations when the sample size is smaller than the number of unknown parameters in the covariance matrix. Our numerical study and data examples show the advantages of our procedures over the existing LRT and RST.

Separable covariance models for health care quality measures across years and topics

Public quality reports for Medicare Advantage health plans include 11 measures of patient experiences reported in the annual Consumer Assessment of Healthcare Providers and Systems surveys. Computing summaries at the health plan level (of multiple measures in multiple years) yields an array-structured random variable. To summarize associations among measures and years, we model the variance-covariance matrix governing the plan-level vectors of yearly quality measures as a Kronecker product of an across-measure matrix and an across-year matrix, or a sum of such Kronecker products. This approach extends separable covariance structure to Fay-Herriot models. In addition, we develop linear combinations of Kronecker products similar to principal components for array random variables. To each Kronecker-product term, we apply post hoc analyses suited to the corresponding dimension of the cross-classification: 1-way factor analysis for the across-measure factor and time-series analysis to the across-year factor. These methods draw out key patterns of variation in the quality measures over time and suggest new strategies for reporting quality information to consumers.

Simultaneous Estimation of Low- and High-Order Functional Connectivity for Identifying Mild Cognitive Impairment

Functional connectivity (FC) network has been becoming an increasingly useful tool for understanding the cerebral working mechanism and mining sensitive biomarkers for neural/mental disease diagnosis. Currently, Pearson's Correlation (PC) is the simplest and most commonly used scheme in FC estimation. Despite its empirical effectiveness, PC only encodes the low-order (i.e., second-order) statistics by calculating the pairwise correlations between network nodes (brain regions), which fails to capture the high-order information involved in FC (e.g., the correlations among different edges in a network). To address this issue, we propose a novel FC estimation method based on Matrix Variate Normal Distribution (MVND), which can capture both low- and high-order correlations simultaneously with a clear mathematical interpretability. Specifically, we first generate a set of BOLD subseries by the sliding window scheme, and for each subseries we construct a temporal FC network by PC. Then, we employ the constructed FC networks as samples to estimate the final low- and high-order FC networks by maximizing the likelihood of MVND. To illustrate the effectiveness of the proposed method, we conduct experiments to identify subjects with Mild Cognitive Impairment (MCI) from Normal Controls (NCs). Experimental results show that the fusion of low- and high-order FCs can generally help to improve the final classification performance, even though the high-order FC may contain less discriminative information than its low-order counterpart. Importantly, the proposed method for simultaneous estimation of low- and high-order FCs can achieve better classification performance than the two baseline methods, i.e., the original PC method and a recent high-order FC estimation method.

Kronecker-Markov Prior for Dynamic 3D Reconstruction

Recovering dynamic 3D structures from 2D image observations is highly under-constrained because of projection and missing data, motivating the use of strong priors to constrain shape deformation. In this paper, we empirically show that the spatiotemporal covariance of natural deformations is dominated by a Kronecker pattern. We demonstrate that this pattern arises as the limit of a spatiotemporal autoregressive process, and derive a Kronecker Markov Random Field as a prior distribution over dynamic structures. This distribution unifies shape and trajectory models of prior art and has the individual models as its marginals. The key assumption of the Kronecker MRF is that the spatiotemporal covariance is separable into the product of a temporal and a shape covariance, and can therefore be modeled using the matrix normal distribution. Analysis on motion capture data validates that this distribution is an accurate approximation with significantly fewer free parameters. Using the trace-norm, we present a convex method to estimate missing data from a single sequence when the marginal shape distribution is unknown. The Kronecker-Markov distribution, fit to a single sequence, outperforms state-of-the-art methods at inferring missing 3D data, and additionally provides covariance estimates of the uncertainty.