Efficient Regularized Regression with L0 Penalty for Variable Selection and Network Construction

Variable selection for mixed panel count data under the proportional mean model

Mixed panel count data have attracted increasing attention in medical research based on event history studies. When such data arise, one either observes the number of event occurrences or only knows whether the event has happened or not over an observation period. In this article, we discuss variable selection in event history studies given such complex data, for which there does not seem to exist an established procedure. For the problem, we propose a penalized likelihood variable selection procedure and for the implementation, an expectation-maximization algorithm is developed with the use of the coordinate descent algorithm in the M-step. Furthermore, the oracle property of the proposed method is established, and a simulation study is performed and indicates that the proposed method works well in practical scenarios. Finally, the method is applied to identify the risk factors associated with medical non-adherence arising from the Sequenced Treatment Alternatives to Relieve Depression Study.

Variable selection for bivariate interval-censored failure time data under linear transformation models

Variable selection is needed and performed in almost every field and a large literature on it has been established, especially under the context of linear models or for complete data. Many authors have also investigated the variable selection problem for incomplete data such as right-censored failure time data. In this paper, we discuss variable selection when one faces bivariate interval-censored failure time data arising from a linear transformation model, for which it does not seem to exist an established procedure. For the problem, a penalized maximum likelihood approach is proposed and in particular, a novel Poisson-based EM algorithm is developed for the implementation. The oracle property of the proposed method is established, and the numerical studies suggest that the method works well for practical situations.

Variable Selection for Generalized Linear Models with Interval-Censored Failure Time Data

Variable selection is often needed in many fields and has been discussed by many authors in various situations. This is especially the case under linear models and when one observes complete data. Among others, one common situation where variable selection is required is to identify important risk factors from a large number of covariates. In this paper, we consider the problem when one observes interval-censored failure time data arising from generalized linear models, for which there does not seem to exist an established method. To address this, we propose a penalized least squares method with the use of an unbiased transformation and the oracle property of the method is established along with the asymptotic normality of the resulting estimators of regression parameters. Simulation studies were conducted and demonstrated that the proposed method performed well for practical situations. In addition, the method was applied to a motivating example about children’s mortality data of Nigeria.

Risk of Developing Breast Reconstruction Complications: A Machine-Learning Nomogram for Individualized Risk Estimation with and without Postmastectomy Radiation Therapy

BACKGROUND

The purpose of this study was to create a nomogram using machine learning models predicting risk of breast reconstruction complications with or without postmastectomy radiation therapy.

METHODS

Between 1997 and 2017, 1617 breast cancer patients undergoing mastectomy and breast reconstruction were analyzed. Those with autologous, tissue expander/implant, and single-stage direct-to-implant reconstruction were included. Postmastectomy radiation therapy was delivered either with three-dimensional conformal photon or proton therapy. Complication endpoints were defined based on surgical reintervention operative notes as infection/necrosis requiring débridement. For implant-based patients, complications were defined as capsular contracture requiring capsulotomy and implant failure. For each complication endpoint, least absolute shrinkage and selection operator-penalized regression was used to select the subset of predictors associated with the smallest prediction error from 10-fold cross-validation. Nomograms were built using the least absolute shrinkage and selection operator-selected predictors, and internal validation using cross-validation was performed.

RESULTS

Median follow-up was 6.6 years. Among 1617 patients, 23 percent underwent autologous reconstruction, 39 percent underwent direct-to-implant reconstruction, and 37 percent underwent tissue expander/implant reconstruction. Among 759 patients who received postmastectomy radiation therapy, 8.3 percent received proton-therapy to the chest wall and nodes and 43 percent received chest wall boost. Internal validation for each model showed an area under the receiver operating characteristic curve of 73 percent for infection, 75 percent for capsular contracture, 76 percent for absolute implant failure, and 68 percent for overall implant failure. Periareolar incisions and complete implant muscle coverage were found to be important predictors for infection and capsular contracture, respectively. In a multivariable analysis, we found that protons compared to no postmastectomy radiation therapy significantly increased capsular contracture risk (OR, 15.3; p < 0.001). This was higher than the effect of photons with electron boost versus no postmastectomy radiation therapy (OR, 2.5; p = 0.01).

CONCLUSION

Using machine learning, these nomograms provided prediction of postmastectomy breast reconstruction complications with and without radiation therapy.

CLINICAL QUESTION/LEVEL OF EVIDENCE

Risk, III.

Simultaneous variable selection and estimation for joint models of longitudinal and failure time data with interval censoring

This paper discusses variable selection in the context of joint analysis of longitudinal data and failure time data. A large literature has been developed for either variable selection or the joint analysis but there exists only limited literature for variable selection in the context of the joint analysis when failure time data are right censored. Corresponding to this, we will consider the situation where instead of right-censored data, one observes interval-censored failure time data, a more general and commonly occurring form of failure time data. For the problem, a class of penalized likelihood-based procedures will be developed for simultaneous variable selection and estimation of relevant covariate effects for both longitudinal and failure time variables of interest. In particular, a Monte Carlo EM (MCEM) algorithm is presented for the implementation of the proposed approach. The proposed method allows for the number of covariates to be diverging with the sample size and is shown to have the oracle property. An extensive simulation study is conducted to assess the finite sample performance of the proposed approach and indicates that it works well in practical situations. An application is also provided.

Variable selection for high-dimensional partly linear additive Cox model with application to Alzheimer's disease

Variable selection has been discussed under many contexts and especially, a large literature has been established for the analysis of right-censored failure time data. In this article, we discuss an interval-censored failure time situation where there exist two sets of covariates with one being low-dimensional and having possible nonlinear effects and the other being high-dimensional. For the problem, we present a penalized estimation procedure for simultaneous variable selection and estimation, and in the method, Bernstein polynomials are used to approximate the involved nonlinear functions. Furthermore, for implementation, a coordinate-wise optimization algorithm, which can accommodate most commonly used penalty functions, is developed. A numerical study is performed for the evaluation of the proposed approach and suggests that it works well in practical situations. Finally the method is applied to an Alzheimer's disease study that motivated this investigation.

Penalized estimation of semiparametric transformation models with interval-censored data and application to Alzheimer's disease

Variable selection or feature extraction is fundamental to identify important risk factors from a large number of covariates and has applications in many fields. In particular, its applications in failure time data analysis have been recognized and many methods have been proposed for right-censored data. However, developing relevant methods for variable selection becomes more challenging when one confronts interval censoring that often occurs in practice. In this article, motivated by an Alzheimer's disease study, we develop a variable selection method for interval-censored data with a general class of semiparametric transformation models. Specifically, a novel penalized expectation-maximization algorithm is developed to maximize the complex penalized likelihood function, which is shown to perform well in the finite-sample situation through a simulation study. The proposed methodology is then applied to the interval-censored data arising from the Alzheimer's disease study mentioned above.

Sparse support vector machines with L0 approximation for ultra-high dimensional omics data

Omics data usually have ultra-high dimension (p) and small sample size (n). Standard support vector machines (SVMs), which minimize the L₂ norm for the primal variables, only lead to sparse solutions for the dual variables. L₁ based SVMs, directly minimizing the L₁ norm, have been used for feature selection with omics data. However, most current methods directly solve the primal formulations of the problem, which are not computationally scalable. The computational complexity increases with the number of features. In addition, L₁ norm is known to be asymptotically biased and not consistent for feature selection. In this paper, we develop an efficient method for sparse support vector machines with L₀ norm approximation. The proposed method approximates the L₀ minimization through solving a series of L₂ optimization problems, which can be formulated with dual variables. It finds the optimal solution for p primal variables through estimating n dual variables, which is more efficient as long as the sample size is small. L₀ approximation leads to sparsity in both dual and primal variables, and can be used for both feature and sample selections. The proposed method identifies much less number of features and achieves similar performances in simulations. We apply the proposed method to feature selections with metagenomic sequencing and gene expression data. It can identify biologically important genes and taxa efficiently.

Simultaneous Estimation and Variable Selection for Interval-Censored Data with Broken Adaptive Ridge Regression

The simultaneous estimation and variable selection for Cox model has been discussed by several authors (Fan and Li, 2002; Huang and Ma, 2010; Tibshirani, 1997) when one observes right-censored failure time data. However, there does not seem to exist an established procedure for interval-censored data, a more general and complex type of failure time data, except two parametric procedures given in Scolas et al. (2016) and Wu and Cook (2015). To address this, we propose a broken adaptive ridge (BAR) regression procedure that combines the strengths of the quadratic regularization and the adaptive weighted bridge shrinkage. In particular, the method allows for the number of covariates to be diverging with the sample size. Under some weak regularity conditions, unlike most of the existing variable selection methods, we establish both the oracle property and the grouping effect of the proposed BAR procedure. An extensive simulation study is conducted and indicates that the proposed approach works well in practical situations and deals with the collinearity problem better than the other oracle-like methods. An application is also provided.

Broken adaptive ridge regression and its asymptotic properties

This paper studies the asymptotic properties of a sparse linear regression estimator, referred to as broken adaptive ridge (BAR) estimator, resulting from an L 0-based iteratively reweighted L 2 penalization algorithm using the ridge estimator as its initial value. We show that the BAR estimator is consistent for variable selection and has an oracle property for parameter estimation. Moreover, we show that the BAR estimator possesses a grouping effect: highly correlated covariates are naturally grouped together, which is a desirable property not known for other oracle variable selection methods. Lastly, we combine BAR with a sparsity-restricted least squares estimator and give conditions under which the resulting two-stage sparse regression method is selection and estimation consistent in addition to having the grouping property in high- or ultrahigh-dimensional settings. Numerical studies are conducted to investigate and illustrate the operating characteristics of the BAR method in comparison with other methods.

Variable selection for recurrent event data with broken adaptive ridge regression

Recurrent event data occur in many areas such as medical studies and social sciences and a great deal of literature has been established for their analysis. On the other hand, only limited research exists on the variable selection for recurrent event data, and the existing methods can be seen as direct generalizations of the available penalized procedures for linear models and may not perform as well as expected. This article discusses simultaneous parameter estimation and variable selection and presents a new method with a new penalty function, which will be referred to as the broken adaptive ridge regression approach. In addition to the establishment of the oracle property, we also show that the proposed method has the clustering or grouping effect when covariates are highly correlated. Furthermore, a numerical study is performed and indicates that the method works well for practical situations and can outperform existing methods. An application is provided.

Sparse generalized linear model with L0 approximation for feature selection and prediction with big omics data

Background

Feature selection and prediction are the most important tasks for big data mining. The common strategies for feature selection in big data mining are L₁, SCAD and MC+. However, none of the existing algorithms optimizes L₀, which penalizes the number of nonzero features directly.

Results

In this paper, we develop a novel sparse generalized linear model (GLM) with L₀ approximation for feature selection and prediction with big omics data. The proposed approach approximate the L₀ optimization directly. Even though the original L₀ problem is non-convex, the problem is approximated by sequential convex optimizations with the proposed algorithm. The proposed method is easy to implement with only several lines of code. Novel adaptive ridge algorithms (L₀ADRIDGE) for L₀ penalized GLM with ultra high dimensional big data are developed. The proposed approach outperforms the other cutting edge regularization methods including SCAD and MC+ in simulations. When it is applied to integrated analysis of mRNA, microRNA, and methylation data from TCGA ovarian cancer, multilevel gene signatures associated with suboptimal debulking are identified simultaneously. The biological significance and potential clinical importance of those genes are further explored.

Conclusions

The developed Software L₀ADRIDGE in MATLAB is available at https://github.com/liuzqx/L0adridge.

Electronic supplementary material

The online version of this article (doi:10.1186/s13040-017-0159-z) contains supplementary material, which is available to authorized users.