Transformer fault diagnosis method based on SMOTE and NGO-GBDT

In order to improve the accuracy of transformer fault diagnosis and improve the influence of unbalanced samples on the low accuracy of model identification caused by insufficient model training, this paper proposes a transformer fault diagnosis method based on SMOTE and NGO-GBDT. Firstly, the Synthetic Minority Over-sampling Technique (SMOTE) was used to expand the minority samples. Secondly, the non-coding ratio method was used to construct multi-dimensional feature parameters, and the Light Gradient Boosting Machine (LightGBM) feature optimization strategy was introduced to screen the optimal feature subset. Finally, Northern Goshawk Optimization (NGO) algorithm was used to optimize the parameters of Gradient Boosting Decision Tree (GBDT), and then the transformer fault diagnosis was realized. The results show that the proposed method can reduce the misjudgment of minority samples. Compared with other integrated models, the proposed method has high fault identification accuracy, low misjudgment rate and stable performance.

Predicting Risk of Bullying Victimization among Primary and Secondary School Students: Based on a Machine Learning Model

School bullying among primary and secondary school students has received increasing attention, and identifying relevant factors is a crucial way to reduce the risk of bullying victimization. Machine learning methods can help researchers predict and identify individual risk behaviors. Through a machine learning approach (i.e., the gradient boosting decision tree model, GBDT), the present longitudinal study aims to systematically examine individual, family, and school environment factors that can predict the risk of bullying victimization among primary and secondary school students a year later. A total of 2767 participants (2065 secondary school students, 702 primary school students, 55.20% female students, mean age at T1 was 12.22) completed measures of 24 predictors at the first wave, including individual factors (e.g., self-control, gender, grade), family factors (family cohesion, parental control, parenting style), peer factor (peer relationship), and school factors (teacher-student relationship, learning capacity). A year later (i.e., T2), they completed the Olweus Bullying Questionnaire. The GBDT model predicted whether primary and secondary school students would be exposed to school bullying after one year by training a series of base learners and outputting the importance ranking of predictors. The GBDT model performed well. The GBDT model yielded the top 6 predictors: teacher-student relationship, peer relationship, family cohesion, negative affect, anxiety, and denying parenting style. The protective factors (i.e., teacher-student relationship, peer relationship, and family cohesion) and risk factors (i.e., negative affect, anxiety, and denying parenting style) associated with the risk of bullying victimization a year later among primary and secondary school students are identified by using a machine learning approach. The GBDT model can be used as a tool to predict the future risk of bullying victimization for children and adolescents and to help improve the effectiveness of school bullying interventions.

Machine learning approach for carbon disclosure in the Korean market: The role of environmental performance

Over the past few decades, scholars have employed a wide range of methodologies to determine the factors influencing firms' voluntary carbon disclosure. Most of these studies have been conducted in advanced markets. This article aims to examine the trend of voluntary carbon disclosure in the Korean financial market by utilizing machine learning models such as Random Forest and Gradient Boosted Decision Tree. Based on a set of hand-collected carbon disclosure data, we initially demonstrated significantly better performance of machine learning models compared to the traditional logistic model. Regarding the factors influencing disclosure, we consistently find the importance of environmental scores, emphasizing the role of the emerging mega-trend of ESG management practices in disclosure decisions. However, in contrast to recent studies, we do not find that the unique Korean governance structure, chaebol, has any significantly different implications in terms of prediction performance and variable importance in carbon disclosure decisions.

GBDT_KgluSite: An improved computational prediction model for lysine glutarylation sites based on feature fusion and GBDT classifier

BACKGROUND

Lysine glutarylation (Kglu) is one of the most important Post-translational modifications (PTMs), which plays significant roles in various cellular functions, including metabolism, mitochondrial processes, and translation. Therefore, accurate identification of the Kglu site is important for elucidating protein molecular function. Due to the time-consuming and expensive limitations of traditional biological experiments, computational-based Kglu site prediction research is gaining more and more attention.

RESULTS

In this paper, we proposed GBDT_KgluSite, a novel Kglu site prediction model based on GBDT and appropriate feature combinations, which achieved satisfactory performance. Specifically, seven features including sequence-based features, physicochemical property-based features, structural-based features, and evolutionary-derived features were used to characterize proteins. NearMiss-3 and Elastic Net were applied to address data imbalance and feature redundancy issues, respectively. The experimental results show that GBDT_KgluSite has good robustness and generalization ability, with accuracy and AUC values of 93.73%, and 98.14% on five-fold cross-validation as well as 90.11%, and 96.75% on the independent test dataset, respectively.

CONCLUSION

GBDT_KgluSite is an effective computational method for identifying Kglu sites in protein sequences. It has good stability and generalization ability and could be useful for the identification of new Kglu sites in the future. The relevant code and dataset are available at https://github.com/flyinsky6/GBDT_KgluSite .

Prediction of diabetic kidney disease risk using machine learning models: A population-based cohort study of Asian adults

Background

Machine learning (ML) techniques improve disease prediction by identifying the most relevant features in multidimensional data. We compared the accuracy of ML algorithms for predicting incident diabetic kidney disease (DKD).

Methods

We utilized longitudinal data from 1365 Chinese, Malay, and Indian participants aged 40-80 y with diabetes but free of DKD who participated in the baseline and 6-year follow-up visit of the Singapore Epidemiology of Eye Diseases Study (2004-2017). Incident DKD (11.9%) was defined as an estimated glomerular filtration rate (eGFR) <60 mL/min/1.73 m2 with at least 25% decrease in eGFR at follow-up from baseline. A total of 339 features, including participant characteristics, retinal imaging, and genetic and blood metabolites, were used as predictors. Performances of several ML models were compared to each other and to logistic regression (LR) model based on established features of DKD (age, sex, ethnicity, duration of diabetes, systolic blood pressure, HbA1c, and body mass index) using area under the receiver operating characteristic curve (AUC).

Results

ML model Elastic Net (EN) had the best AUC (95% CI) of 0.851 (0.847-0.856), which was 7.0% relatively higher than by LR 0.795 (0.790-0.801). Sensitivity and specificity of EN were 88.2 and 65.9% vs. 73.0 and 72.8% by LR. The top 15 predictors included age, ethnicity, antidiabetic medication, hypertension, diabetic retinopathy, systolic blood pressure, HbA1c, eGFR, and metabolites related to lipids, lipoproteins, fatty acids, and ketone bodies.

Conclusions

Our results showed that ML, together with feature selection, improves prediction accuracy of DKD risk in an asymptomatic stable population and identifies novel risk factors, including metabolites.

Funding

This study was supported by the National Medical Research Council, NMRC/OFLCG/001/2017 and NMRC/HCSAINV/MOH-001019-00. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Bayesian model averaging for predicting factors associated with length of COVID-19 hospitalization

INTRODUCTION

The length of hospital stay (LOHS) caused by COVID-19 has imposed a financial burden, and cost on the healthcare service system and a high psychological burden on patients and health workers. The purpose of this study is to adopt the Bayesian model averaging (BMA) based on linear regression models and to determine the predictors of the LOHS of COVID-19.

METHODS

In this historical cohort study, from 5100 COVID-19 patients who had registered in the hospital database, 4996 patients were eligible to enter the study. The data included demographic, clinical, biomarkers, and LOHS. Factors affecting the LOHS were fitted in six models, including the stepwise method, AIC, BIC in classical linear regression models, two BMA using Occam's Window and Markov Chain Monte Carlo (MCMC) methods, and GBDT algorithm, a new method of machine learning.

RESULTS

The average length of hospitalization was 6.7 ± 5.7 days. In fitting classical linear models, both stepwise and AIC methods (R 2 = 0.168 and adjusted R 2 = 0.165) performed better than BIC (R 2 = 0.160 and adjusted = 0.158). In fitting the BMA, Occam's Window model has performed better than MCMC with R 2 = 0.174. The GBDT method with the value of R 2 = 0.64, has performed worse than the BMA in the testing dataset but not in the training dataset. Based on the six fitted models, hospitalized in ICU, respiratory distress, age, diabetes, CRP, PO2, WBC, AST, BUN, and NLR were associated significantly with predicting LOHS of COVID-19.

CONCLUSION

The BMA with Occam's Window method has a better fit and better performance in predicting affecting factors on the LOHS in the testing dataset than other models.

CED: A case-level explainable paramedical diagnosis via AdaGBDT

OBJECTIVE

The rapid growth of medical data has greatly promoted the wide exploitation of machine learning for paramedical diagnosis. Inversely proportional to their performance, most machine learning models generally suffer from the lack of explainability, especially the local explainability of the model, that is, the case-specific explainability.

MATERIALS AND METHODS

In this paper, we proposed a GBDT (Gradient Boosting Decision Tree)-based explainable model for case-specific paramedical diagnostics, and mainly make the following contributions: (1) an adaptive gradient boosting decision tree (AdaGBDT) model is proposed to boost the path-mining for decision effectively; (2) to learn a case-specific feature importance embedding for a specific patient, the bi-side mutual information is applied to characterize the backtracking on the decision path; (3) through the collaborative decision-making by globally explainable AdaGBDT with case-based reasoning (CBR) in the case-specific metric space, some hard cases can be identified by the means of visualized interpretation. The performance of our model is evaluated on the Wisconsin diagnostic breast cancer dataset and the UCI heart disease dataset.

RESULTS

Experiments conducted on two datasets show that our AdaGBDT achieves the best performance, with the F1-value of 0.9647 and 0.8405 respectively. Moreover, a series of experimental analyses and case studies further illustrate the excellent performance of feature importance embedding.

CONCLUSION

The proposed case-specific explainable paramedical diagnosis via AdaGBDT has excellent predictive performance, with both promising case-level and consistent global explainability.

An objective model for diagnosing comorbid cognitive impairment in patients with epilepsy based on the clinical-EEG functional connectivity features

Objective

Cognitive impairment (CI) is a common disorder in patients with epilepsy (PWEs). Objective assessment method for diagnosing CI in PWEs would be beneficial in reality. This study proposed to construct a diagnostic model for CI in PWEs using the clinical and the phase locking value (PLV) functional connectivity features of the electroencephalogram (EEG).

Methods

PWEs who met the inclusion and exclusion criteria were divided into a cognitively normal (CON) group (n = 55) and a CI group (n = 76). The 23 clinical features and 684 PLV _EEG features at the time of patient visit were screened and ranked using the Fisher score. Adaptive Boosting (AdaBoost) and Gradient Boosting Decision Tree (GBDT) were used as algorithms to construct diagnostic models of CI in PWEs either with pure clinical features, pure PLV _EEG features, or combined clinical and PLV _EEG features. The performance of these models was assessed using a five-fold cross-validation method.

Results

GBDT-built model with combined clinical and PLV _EEG features performed the best with accuracy, precision, recall, F1-score, and an area under the curve (AUC) of 90.11, 93.40, 89.50, 91.39, and 0.95%. The top 5 features found to influence the model performance based on the Fisher scores were the magnetic resonance imaging (MRI) findings of the head for abnormalities, educational attainment, PLV _EEG in the beta (β)-band C3-F4, seizure frequency, and PLV _EEG in theta (θ)-band Fp1-Fz. A total of 12 of the top 5% of features exhibited statistically different PLV _EEG features, while eight of which were PLV _EEG features in the θ band.

Conclusion

The model constructed from the combined clinical and PLV _EEG features could effectively identify CI in PWEs and possess the potential as a useful objective evaluation method. The PLV _EEG in the θ band could be a potential biomarker for the complementary diagnosis of CI comorbid with epilepsy.

M6A-GSMS: Computational identification of N6-methyladenosine sites with GBDT and stacking learning in multiple species

N⁶-methyladenosine (m⁶A) is one of the most abundant forms of RNA methylation modifications currently known. It involves a wide range of biological processes, including degradation, stability, alternative splicing, etc. Therefore, the development of convenient and efficient m⁶A prediction technologies are urgent. In this work, a novel predictor based on GBDT and stacking learning is developed to identify m⁶A sites, which is called M6A-GSMS. To achieve accurate prediction, we explore RNA sequence information from four aspects: correlation, structure, physicochemical properties and pseudo ribonucleic acid composition. After using the GBDT algorithm for feature selection, a stacking model is constructed by combining seven basic classifiers. Compared with other state-of-the-art methods, the results show that M6A-GSMS can obtain excellent performance for identifying the m⁶A sites. The prediction accuracy of A.thaliana, D.melanogaster, M.musculus, S.cerevisiae and Human reaches 88.4%, 60.8%, 80.5%, 92.4% and 61.8%, respectively. This method provides an effective prediction for the investigation of m⁶A sites. In addition, all the datasets and codes are currently available at https://github.com/Wang-Jinyue/M6A-GSMS.Communicated by Ramaswamy H. Sarma.

Epilepsy Seizures Prediction Based on Nonlinear Features of EEG Signal and Gradient Boosting Decision Tree

Epilepsy is a common neurological disorder with sudden and recurrent seizures. Early prediction of seizures and effective intervention can significantly reduce the harm suffered by patients. In this paper, a method based on nonlinear features of EEG signal and gradient boosting decision tree (GBDT) is proposed for early prediction of epilepsy seizures. First, the EEG signals were divided into two categories: those that had seizures onset over a period of time (represented by InT) and those that did not. Second, the noise in the EEG was removed using complementary ensemble empirical mode decomposition (CEEMD) and wavelet threshold denoising. Third, the nonlinear features of the two categories of EEG were extracted, including approximate entropy, sample entropy, permutation entropy, spectral entropy and wavelet entropy. Fourth, a GBDT classifier with random forest as the initial result was designed to distinguish the two categories of EEG. Fifth, a two-step "k of n" method was used to reduce the number of false alarms. The proposed method was evaluated on 13 patients' EEG data from the CHB-MIT Scalp EEG Database. Based on ten-fold cross validation, the average accuracy was 91.76% when the InT was taken at 30 min, and 38 out of 39 seizures were successfully predicted. When the InT was taken for 40 min, the average accuracy was 92.50% and all 42 seizures selected were successfully predicted. The results indicate the effectiveness of the proposed method for predicting epilepsy seizures.

Construction of a Risk Prediction Model for Hospital-Acquired Pulmonary Embolism in Hospitalized Patients

The purpose of this study is to establish a novel pulmonary embolism (PE) risk prediction model based on machine learning (ML) methods and to evaluate the predictive performance of the model and the contribution of variables to the predictive performance. We conducted a retrospective study at the Shanghai Tenth People's Hospital and collected the clinical data of in-patients that received pulmonary computed tomography imaging between January 1, 2014 and December 31, 2018. We trained several ML models, including logistic regression (LR), support vector machine (SVM), random forest (RF), and gradient boosting decision tree (GBDT), compared the models with representative baseline algorithms, and investigated their predictability and feature interpretation. A total of 3619 patients were included in the study. We discovered that the GBDT model demonstrated the best prediction with an area under the curve value of 0.799, whereas those of the RF, LR, and SVM models were 0.791, 0.716, and 0.743, respectively. The sensibilities of the GBDT, LR, RF, and SVM models were 63.9%, 68.1%, 71.5%, and 75%, respectively; the specificities were 81.1%, 66.1, 72.7%, and 65.1%, respectively; and the accuracies were 77.8%, 66.5%, 72.5%, and 67%, respectively. We discovered that the maximum D-dimer level contributed the most to the outcome prediction, followed by the extreme growth rate of the plasma fibrinogen level, in-hospital duration, and extreme growth rate of the D-dimer level. The study demonstrates the superiority of the GBDT model in predicting the risk of PE in hospitalized patients. However, in order to be applied in clinical practice and provide support for clinical decision-making, the predictive performance of the model needs to be prospectively verified.

What Factors Are Most Closely Associated With Mood Disorders in Adolescents During the COVID-19 Pandemic? A Cross-Sectional Study Based on 1,771 Adolescents in Shandong Province, China

Background and Aims: COVID-19 has been proven to harm adolescents' mental health, and several psychological influence factors have been proposed. However, the importance of these factors in the development of mood disorders in adolescents during the pandemic still eludes researchers, and practical strategies for mental health education are limited. Methods: We constructed a sample of 1,771 adolescents from three junior high middle schools, three senior high middle schools, and three independent universities in Shandong province, China. The sample stratification was set as 5:4:3 for adolescent aged from 12 - 15, 15 - 18, 18 - 19. We examined the subjects' anxiety, depression, psychological resilience, perceived social support, coping strategies, subjective social/school status, screen time, and sleep quality with suitable psychological scales. We chose four widely used classification models-k-nearest neighbors, logistic regression, gradient-boosted decision tree (GBDT), and a combination of the GBDT and LR (GBDT + LR)-to construct machine learning models, and we utilized the Shapley additive explanations value (SHAP) to measure how the features affected the dependent variables. The area under the curve (AUC) of the receiver operating characteristic (ROC) curves was used to evaluate the performance of the models. Results: The current rates of occurrence of symptoms of anxiety and depression were 28.3 and 30.8% among the participants. The descriptive and univariate analyses showed that all of the factors included were statistically related to mood disorders. Among the four machine learning algorithms, the GBDT+LR algorithm achieved the best performance for anxiety and depression with average AUC values of 0.819 and 0.857. We found that the poor sleep quality was the most significant risk factor for mood disorders among Chinese adolescents. In addition, according to the feature importance (SHAP) of the psychological factors, we proposed a five-step mental health education strategy to be used during the COVID-19 pandemic (sleep quality-resilience-coping strategy-social support-perceived social status). Conclusion: In this study, we performed a cross-sectional investigation to examine the psychological impact of COVID-19 on adolescents. We applied machine learning algorithms to quantify the importance of each factor. In addition, we proposed a five-step mental health education strategy for school psychologists.

PRG: A Distance Measurement Algorithm Based on Phase Regeneration

With the booming development of the Internet of things (IoT) industry, the demand of positioning technology in various IoT application scenarios is also greatly increased. To meet the positioning requirements of the IoT application, we propose a distance measurement method based on phase regeneration that can provide positioning capability for IoT applications in indoor and outdoor environments. The PRG algorithm consists of two phases: coarse ranging phase and fine ranging phase. Fingerprint positioning algorithm based on Gradient Boost Decision Tree (GBDT) is used to determine coarse distance. The host machine measures the difference between the transmitted carrier phase and the received regenerative carrier phase to fix the fine distance and then the coarse distance is used to determine the carrier phase integer ambiguity. Finally, high precision ranging is realized. Simulation results show that the PRG method can achieve range finding with decimeter level precision under the 10 MHz subcarrier frequency.

NucPosPred: Predicting species-specific genomic nucleosome positioning via four different modes of general PseKNC

The nucleosome is the basic structure of chromatin in eukaryotic cells, with essential roles in the regulation of many biological processes, such as DNA transcription, replication and repair, and RNA splicing. Because of the importance of nucleosomes, the factors that determine their positioning within genomes should be investigated. High-resolution nucleosome-positioning maps are now available for organisms including Saccharomyces cerevisiae, Drosophila melanogaster and Caenorhabditis elegans, enabling the identification of nucleosome positioning by application of computational tools. Here, we describe a novel predictor called NucPosPred, which was specifically designed for large-scale identification of nucleosome positioning in C. elegans and D. melanogaster genomes. NucPosPred was separately optimized for each species for four types of DNA sequence feature extraction, with consideration of two classification algorithms (gradient-boosting decision tree and support vector machine). The overall accuracy obtained with NucPosPred was 92.29% for C. elegans and 88.26% for D. melanogaster, outperforming previous methods and demonstrating the potential for species-specific prediction of nucleosome positioning. For the convenience of most experimental scientists, a web-server for the predictor NucPosPred is available at http://121.42.167.206/NucPosPred/index.jsp.