A guide for kernel generalized regression methods for genomic-enabled prediction

Stacked generalization as a computational method for the genomic selection

As genomic selection emerges as a promising breeding method for both plants and animals, numerous methods have been introduced and applied to various real and simulated data sets. Research suggests that no single method is universally better than others; rather, performance is highly dependent on the characteristics of the data and the nature of the prediction task. This implies that each method has its strengths and weaknesses. In this study, we exploit this notion and propose a different approach. Rather than comparing multiple methods to determine the best one for a particular study, we advocate combining multiple methods to achieve better performance than each method in isolation. In pursuit of this goal, we introduce and develop a computational method of the stacked generalization within ensemble methods. In this method, the meta-model merges predictions from multiple base models to achieve improved performance. We applied this method to plant and animal data and compared its performance with currently available methods using standard performance metrics. We found that the proposed method yielded a lower or comparable mean squared error in predicting phenotypes compared to the current methods. In addition, the proposed method showed greater resistance to overfitting compared to the current methods. Further analysis included statistical hypothesis testing, which showed that the proposed method outperformed or matched the current methods. In summary, the proposed stacked generalization integrates currently available methods to achieve stable and better performance. In this context, our study provides general recommendations for effective practices in genomic selection.

Benchmarking machine learning and parametric methods for genomic prediction of feed efficiency-related traits in Nellore cattle

Genomic selection (GS) offers a promising opportunity for selecting more efficient animals to use consumed energy for maintenance and growth functions, impacting profitability and environmental sustainability. Here, we compared the prediction accuracy of multi-layer neural network (MLNN) and support vector regression (SVR) against single-trait (STGBLUP), multi-trait genomic best linear unbiased prediction (MTGBLUP), and Bayesian regression (BayesA, BayesB, BayesC, BRR, and BLasso) for feed efficiency (FE) traits. FE-related traits were measured in 1156 Nellore cattle from an experimental breeding program genotyped for ~ 300 K markers after quality control. Prediction accuracy (Acc) was evaluated using a forward validation splitting the dataset based on birth year, considering the phenotypes adjusted for the fixed effects and covariates as pseudo-phenotypes. The MLNN and SVR approaches were trained by randomly splitting the training population into fivefold to select the best hyperparameters. The results show that the machine learning methods (MLNN and SVR) and MTGBLUP outperformed STGBLUP and the Bayesian regression approaches, increasing the Acc by approximately 8.9%, 14.6%, and 13.7% using MLNN, SVR, and MTGBLUP, respectively. Acc for SVR and MTGBLUP were slightly different, ranging from 0.62 to 0.69 and 0.62 to 0.68, respectively, with empirically unbiased for both models (0.97 and 1.09). Our results indicated that SVR and MTGBLUBP approaches were more accurate in predicting FE-related traits than Bayesian regression and STGBLUP and seemed competitive for GS of complex phenotypes with various degrees of inheritance.

(Quasi) multitask support vector regression with heuristic hyperparameter optimization for whole-genome prediction of complex traits: a case study with carcass traits in broilers

This study investigates nonlinear kernels for multitrait (MT) genomic prediction using support vector regression (SVR) models. We assessed the predictive ability delivered by single-trait (ST) and MT models for 2 carcass traits (CT1 and CT2) measured in purebred broiler chickens. The MT models also included information on indicator traits measured in vivo [Growth and feed efficiency trait (FE)]. We proposed an approach termed (quasi) multitask SVR (QMTSVR), with hyperparameter optimization performed via genetic algorithm. ST and MT Bayesian shrinkage and variable selection models [genomic best linear unbiased predictor (GBLUP), BayesC (BC), and reproducing kernel Hilbert space (RKHS) regression] were employed as benchmarks. MT models were trained using 2 validation designs (CV1 and CV2), which differ if the information on secondary traits is available in the testing set. Models' predictive ability was assessed with prediction accuracy (ACC; i.e. the correlation between predicted and observed values, divided by the square root of phenotype accuracy), standardized root-mean-squared error (RMSE*), and inflation factor (b). To account for potential bias in CV2-style predictions, we also computed a parametric estimate of accuracy (ACCpar). Predictive ability metrics varied according to trait, model, and validation design (CV1 or CV2), ranging from 0.71 to 0.84 for ACC, 0.78 to 0.92 for RMSE*, and between 0.82 and 1.34 for b. The highest ACC and smallest RMSE* were achieved with QMTSVR-CV2 in both traits. We observed that for CT1, model/validation design selection was sensitive to the choice of accuracy metric (ACC or ACCpar). Nonetheless, the higher predictive accuracy of QMTSVR over MTGBLUP and MTBC was replicated across accuracy metrics, besides the similar performance between the proposed method and the MTRKHS model. Results showed that the proposed approach is competitive with conventional MT Bayesian regression models using either Gaussian or spike-slab multivariate priors.

Application of linear and machine learning models to genomic prediction of fatty acid composition in Japanese Black cattle

We collected 3180 records of oleic acid (C18:1) and monounsaturated fatty acid (MUFA) measured using gas chromatography (GC) and 6960 records of C18:1 and MUFA measured using near-infrared spectroscopy (NIRS) in intermuscular fat samples of Japanese Black cattle. We compared genomic prediction performance for four linear models (genomic best linear unbiased prediction [GBLUP], kinship-adjusted multiple loci [KAML], BayesC, and BayesLASSO) and five machine learning models (Gaussian kernel [GK], deep kernel [DK], random forest [RF], extreme gradient boost [XGB], and convolutional neural network [CNN]). For GC-based C18:1 and MUFA, KAML showed the highest accuracies, followed by BayesC, XGB, DK, GK, and BayesLASSO, with more than 6% gain of accuracy by KAML over GBLUP. Meanwhile, DK had the highest prediction accuracy for NIRS-based C18:1 and MUFA, but the difference in accuracies between DK and KAML was slight. For all traits, accuracies of RF and CNN were lower than those of GBLUP. The KAML extends GBLUP methods, of which marker effects are weighted, and involves only additive genetic effects; whereas machine learning methods capture non-additive genetic effects. Thus, KAML is the most suitable method for breeding of fatty acid composition in Japanese Black cattle.

A Comparison between Three Tuning Strategies for Gaussian Kernels in the Context of Univariate Genomic Prediction

Genomic prediction is revolutionizing plant breeding since candidate genotypes can be selected without the need to measure their trait in the field. When a reference population contains both phenotypic and genotypic information, it is trained by a statistical machine learning method that is subsequently used for making predictions of breeding or phenotypic values of candidate genotypes that were only genotyped. Nevertheless, the successful implementation of the genomic selection (GS) methodology depends on many factors. One key factor is the type of statistical machine learning method used since some are unable to capture nonlinear patterns available in the data. While kernel methods are powerful statistical machine learning algorithms that capture complex nonlinear patterns in the data, their successful implementation strongly depends on the careful tuning process of the involved hyperparameters. As such, in this paper we compare three methods of tuning (manual tuning, grid search, and Bayesian optimization) for the Gaussian kernel under a Bayesian best linear unbiased predictor model. We used six real datasets of wheat (Triticum aestivum L.) to compare the three strategies of tuning. We found that if we want to obtain the major benefits of using Gaussian kernels, it is very important to perform a careful tuning process. The best prediction performance was observed when the tuning process was performed with grid search and Bayesian optimization. However, we did not observe relevant differences between the grid search and Bayesian optimization approach. The observed gains in terms of prediction performance were between 2.1% and 27.8% across the six datasets under study.

Toward Precision Radiotherapy: A Nonlinear Optimization Framework and an Accelerated Machine Learning Algorithm for the Deconvolution of Tumor-Infiltrating Immune Cells

Tumor-infiltrating immune cells (TIICs) form a critical part of the ecosystem surrounding a cancerous tumor. Recent advances in radiobiology have shown that, in addition to damaging cancerous cells, radiotherapy drives the upregulation of immunosuppressive and immunostimulatory TIICs, which in turn impacts treatment response. Quantifying TIICs in tumor samples could form an important predictive biomarker guiding patient stratification and the design of radiotherapy regimens and combined immune-radiation treatments. As a result of several limitations associated with experimental methods for quantifying TIICs and the availability of extensive gene sequencing data, deconvolution-based computational methods have appeared as a suitable alternative for quantifying TIICs. Accordingly, we introduce and discuss a nonlinear regression approach (remarkably different from the traditional linear modeling approach of current deconvolution-based methods) and a machine learning algorithm for approximating the solution of the resulting constrained optimization problem. This way, the deconvolution problem is treated naturally, given that the gene expression levels of pure and heterogenous samples do not have a strictly linear relationship. When applied across transcriptomics datasets, our approach, which also allows the coupling of different loss functions, yields results that closely match ground-truth values from experimental methods and exhibits superior performance over popular deconvolution-based methods.

Exploring methods to summarize gut microbiota composition for microbiability estimation and phenotypic prediction in swine

The microbial composition resemblance among individuals in a group can be summarized in a square covariance matrix and fitted in linear models. We investigated eight approaches to create the matrix that quantified the resemblance between animals based on the gut microbiota composition. We aimed to compare the performance of different methods in estimating trait microbiability and predicting growth and body composition traits in three pig breeds. This study included 651 purebred boars from either breed: Duroc (n = 205), Landrace (n = 226), and Large White (n = 220). Growth and body composition traits, including body weight (BW), ultrasound backfat thickness (BF), ultrasound loin depth (LD), and ultrasound intramuscular fat (IMF) content, were measured on live animals at the market weight (156 ± 2.5 days of age). Rectal swabs were taken from each animal at 158 ± 4 days of age and subjected to 16S rRNA gene sequencing. Eight methods were used to create the microbial similarity matrices, including four kernel functions (Linear Kernel, LK; Polynomial Kernel, PK; Gaussian Kernel, GK; Arc-cosine Kernel with one hidden layer, AK1), two dissimilarity methods (Bray-Curtis, BC; Jaccard, JA), and two ordination methods (Metric Multidimensional Scaling, MDS; Detrended Correspondence analysis, DCA). Based on the matrix used, microbiability estimates ranged from 0.07 to 0.21 and 0.12 to 0.53 for Duroc, 0.03 to 0.21 and 0.05 to 0.44 for Landrace, and 0.02 to 0.24 and 0.05 to 0.52 for Large White pigs averaged over traits in the model with sire, pen, and microbiome, and model with the only microbiome, respectively. The GK, JA, BC, and AK1 obtained greater microbiability estimates than the remaining methods across traits and breeds. Predictions were made within each breed group using four-fold cross-validation based on the relatedness of sires in each breed group. The prediction accuracy ranged from 0.03 to 0.18 for BW, 0.08 to 0.31 for BF, 0.21 to 0.48 for LD, and 0.04 to 0.16 for IMF when averaged across breeds. The BC, MDS, LK, and JA achieved better accuracy than other methods in most predictions. Overall, the PK and DCA exhibited the worst performance compared to other microbiability estimation and prediction methods. The current study shows how alternative approaches summarized the resemblance of gut microbiota composition among animals and contributed this information to variance component estimation and phenotypic prediction in swine.

A new approach fits multivariate genomic prediction models efficiently

Background

Fast, memory-efficient, and reliable algorithms for estimating genomic estimated breeding values (GEBV) for multiple traits and environments are needed to make timely decisions in breeding. Multivariate genomic prediction exploits genetic correlations between traits and environments to increase accuracy of GEBV compared to univariate methods. These genetic correlations are estimated simultaneously with GEBV, because they are specific to year, environment, and management. However, estimating genetic parameters is computationally demanding with restricted maximum likelihood (REML) and Bayesian samplers, and canonical transformations or orthogonalizations cannot be used for unbalanced experimental designs.

Methods

We propose a multivariate randomized Gauss–Seidel algorithm for simultaneous estimation of model effects and genetic parameters. Two previously proposed methods for estimating genetic parameters were combined with a Gauss–Seidel (GS) solver, and were called Tilde-Hat-GS (THGS) and Pseudo-Expectation-GS (PEGS). Balanced and unbalanced experimental designs were simulated to compare runtime, bias and accuracy of GEBV, and bias and standard errors of estimates of heritabilities and genetic correlations of THGS, PEGS, and REML. Models with 10 to 400 response variables, 1279 to 42,034 genetic markers, and 5990 to 1.85 million observations were fitted.

Results

Runtime of PEGS and THGS was a fraction of REML. Accuracies of GEBV were slightly lower than those from REML, but higher than those from the univariate approach, hence THGS and PEGS exploited genetic correlations. For 500 to 600 observations per response variable, biases of estimates of genetic parameters of THGS and PEGS were small, but standard errors of estimates of genetic correlations were higher than for REML. Bias and standard errors decreased as sample size increased. For balanced designs, GEBV and estimates of genetic correlations from THGS were unbiased when only an intercept and eigenvectors of genotype scores were fitted.

Conclusions

THGS and PEGS are fast and memory-efficient algorithms for multivariate genomic prediction for balanced and unbalanced experimental designs. They are scalable for increasing numbers of environments and genetic markers. Accuracy of GEBV was comparable to REML. Estimates of genetic parameters had little bias, but their standard errors were larger than for REML. More studies are needed to evaluate the proposed methods for datasets that contain selection.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12711-022-00730-w.

A General-Purpose Machine Learning R Library for Sparse Kernels Methods With an Application for Genome-Based Prediction

The adoption of machine learning frameworks in areas beyond computer science have been facilitated by the development of user-friendly software tools that do not require an advanced understanding of computer programming. In this paper, we present a new package (sparse kernel methods, SKM) software developed in R language for implementing six (generalized boosted machines, generalized linear models, support vector machines, random forest, Bayesian regression models and deep neural networks) of the most popular supervised machine learning algorithms with the optional use of sparse kernels. The SKM focuses on user simplicity, as it does not try to include all the available machine learning algorithms, but rather the most important aspects of these six algorithms in an easy-to-understand format. Another relevant contribution of this package is a function for the computation of seven different kernels. These are Linear, Polynomial, Sigmoid, Gaussian, Exponential, Arc-Cosine 1 and Arc-Cosine L (with L = 2, 3, … ) and their sparse versions, which allow users to create kernel machines without modifying the statistical machine learning algorithm. It is important to point out that the main contribution of our package resides in the functionality for the computation of the sparse version of seven basic kernels, which is indispensable for reducing computational resources to implement kernel machine learning methods without a significant loss in prediction performance. Performance of the SKM is evaluated in a genome-based prediction framework using both a maize and wheat data set. As such, the use of this package is not restricted to genome prediction problems, and can be used in many different applications.

Bayesian multitrait kernel methods improve multienvironment genome-based prediction

When multitrait data are available, the preferred models are those that are able to account for correlations between phenotypic traits because when the degree of correlation is moderate or large, this increases the genomic prediction accuracy. For this reason, in this article, we explore Bayesian multitrait kernel methods for genomic prediction and we illustrate the power of these models with three-real datasets. The kernels under study were the linear, Gaussian, polynomial, and sigmoid kernels; they were compared with the conventional Ridge regression and GBLUP multitrait models. The results show that, in general, the Gaussian kernel method outperformed conventional Bayesian Ridge and GBLUP multitrait linear models by 2.2–17.45% (datasets 1–3) in terms of prediction performance based on the mean square error of prediction. This improvement in terms of prediction performance of the Bayesian multitrait kernel method can be attributed to the fact that the proposed model is able to capture nonlinear patterns more efficiently than linear multitrait models. However, not all kernels perform well in the datasets used for evaluation, which is why more than one kernel should be evaluated to be able to choose the best kernel.