Analysis of sampling techniques for imbalanced data: An n = 648 ADNI study

Generative AI with WGAN-GP for boosting seizure detection accuracy

Background

Imbalanced datasets pose challenges for developing accurate seizure detection systems based on electroencephalogram (EEG) data. Generative AI techniques may help augment minority class data to facilitate automatic epileptic seizure detection.

New method

This study investigates the impact of various data augmentation (DA) approaches, including Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP), Vanilla GAN, Conditional GAN (CGAN), and Cramer GAN, on classification performance with Random Forest models. The best-performing GAN variant, WGAN-GP, was then integrated with a bidirectional Long Short-Term Memory (LSTM) architecture and compared against traditional and synthetic oversampling methods.

Results

The evaluation of different GAN variants for data augmentation with Random Forest classifiers identified WGAN-GP as the most effective approach. The integration of WGAN-GP with bidirectional LSTM yielded substantial performance improvements, outperforming traditional oversampling methods and achieving an accuracy of 91.73% on the augmented data, compared to 86% accuracy on real data without augmentation.

Comparison with existing methods

The proposed generative AI approach combining WGAN-GP and recurrent neural network models outperforms comparative synthetic oversampling methods on metrics relevant for reliable seizure detection from imbalanced EEG datasets.

Conclusions

Incorporating the WGAN-GP generative AI technique for data augmentation and integrating it with bidirectional LSTM elevates seizure detection accuracy for imbalanced EEG datasets, surpassing the performance of traditional oversampling and class weight adjustment methods. This approach shows promise for improving epilepsy monitoring and management through enhanced automated detection system effectiveness.

Understanding machine learning applications in dementia research and clinical practice: a review for biomedical scientists and clinicians

Several (inter)national longitudinal dementia observational datasets encompassing demographic information, neuroimaging, biomarkers, neuropsychological evaluations, and muti-omics data, have ushered in a new era of potential for integrating machine learning (ML) into dementia research and clinical practice. ML, with its proficiency in handling multi-modal and high-dimensional data, has emerged as an innovative technique to facilitate early diagnosis, differential diagnosis, and to predict onset and progression of mild cognitive impairment and dementia. In this review, we evaluate current and potential applications of ML, including its history in dementia research, how it compares to traditional statistics, the types of datasets it uses and the general workflow. Moreover, we identify the technical barriers and challenges of ML implementations in clinical practice. Overall, this review provides a comprehensive understanding of ML with non-technical explanations for broader accessibility to biomedical scientists and clinicians.

Artificial intelligence in salivary biomarker discovery and validation for oral diseases

Salivary biomarkers can improve the efficacy, efficiency, and timeliness of oral and maxillofacial disease diagnosis and monitoring. Oral and maxillofacial conditions in which salivary biomarkers have been utilized for disease-related outcomes include periodontal diseases, dental caries, oral cancer, temporomandibular joint dysfunction, and salivary gland diseases. However, given the equivocal accuracy of salivary biomarkers during validation, incorporating contemporary analytical techniques for biomarker selection and operationalization from the abundant multi-omics data available may help improve biomarker performance. Artificial intelligence represents one such advanced approach that may optimize the potential of salivary biomarkers to diagnose and manage oral and maxillofacial diseases. Therefore, this review summarized the role and current application of techniques based on artificial intelligence for salivary biomarker discovery and validation in oral and maxillofacial diseases.

Undersampling and cumulative class re-decision methods to improve detection of agitation in people with dementia

Agitation is one of the most prevalent symptoms in people with dementia (PwD) that can place themselves and the caregiver's safety at risk. Developing objective agitation detection approaches is important to support health and safety of PwD living in a residential setting. In a previous study, we collected multimodal wearable sensor data from 17 participants for 600 days and developed machine learning models for detecting agitation in 1-min windows. However, there are significant limitations in the dataset, such as imbalance problem and potential imprecise labels as the occurrence of agitation is much rarer in comparison to the normal behaviours. In this paper, we first implemented different undersampling methods to eliminate the imbalance problem, and came to the conclusion that only 20% of normal behaviour data were adequate to train a competitive agitation detection model. Then, we designed a weighted undersampling method to evaluate the manual labeling mechanism given the ambiguous time interval assumption. After that, the postprocessing method of cumulative class re-decision (CCR) was proposed based on the historical sequential information and continuity characteristic of agitation, improving the decision-making performance for the potential application of agitation detection system. The results showed that a combination of undersampling and CCR improved F1-score and other metrics to varying degrees with less training time and data.

Supplementary Information

The online version contains supplementary material available at 10.1007/s13534-023-00313-8.

A metric learning-based method for biomedical entity linking

Biomedical entity linking task is the task of mapping mention(s) that occur in a particular textual context to a unique concept or entity in a knowledge base, e.g., the Unified Medical Language System (UMLS). One of the most challenging aspects of the entity linking task is the ambiguity of mentions, i.e., (1) mentions whose surface forms are very similar, but which map to different entities in different contexts, and (2) entities that can be expressed using diverse types of mentions. Recent studies have used BERT-based encoders to encode mentions and entities into distinguishable representations such that their similarity can be measured using distance metrics. However, most real-world biomedical datasets suffer from severe imbalance, i.e., some classes have many instances while others appear only once or are completely absent from the training data. A common way to address this issue is to down-sample the dataset, i.e., to reduce the number instances of the majority classes to make the dataset more balanced. In the context of entity linking, down-sampling reduces the ability of the model to comprehensively learn the representations of mentions in different contexts, which is very important. To tackle this issue, we propose a metric-based learning method that treats a given entity and its mentions as a whole, regardless of the number of mentions in the training set. Specifically, our method uses a triplet loss-based function in conjunction with a clustering technique to learn the representation of mentions and entities. Through evaluations on two challenging biomedical datasets, i.e., MedMentions and BC5CDR, we show that our proposed method is able to address the issue of imbalanced data and to perform competitively with other state-of-the-art models. Moreover, our method significantly reduces computational cost in both training and inference steps. Our source code is publicly available here.

Class-Balanced Deep Learning with Adaptive Vector Scaling Loss for Dementia Stage Detection

Alzheimer's disease (AD) leads to irreversible cognitive decline, with Mild Cognitive Impairment (MCI) as its prodromal stage. Early detection of AD and related dementia is crucial for timely treatment and slowing disease progression. However, classifying cognitive normal (CN), MCI, and AD subjects using machine learning models faces class imbalance, necessitating the use of balanced accuracy as a suitable metric. To enhance model performance and balanced accuracy, we introduce a novel method called VS-Opt-Net. This approach incorporates the recently developed vector-scaling (VS) loss into a machine learning pipeline named STREAMLINE. Moreover, it employs Bayesian optimization for hyperparameter learning of both the model and loss function. VS-Opt-Net not only amplifies the contribution of minority examples in proportion to the imbalance level but also addresses the challenge of generalization in training deep networks. In our empirical study, we use MRI-based brain regional measurements as features to conduct the CN vs MCI and AD vs MCI binary classifications. We compare the balanced accuracy of our model with other machine learning models and deep neural network loss functions that also employ class-balanced strategies. Our findings demonstrate that after hyperparameter optimization, the deep neural network using the VS loss function substantially improves balanced accuracy. It also surpasses other models in performance on the AD dataset. Moreover, our feature importance analysis highlights VS-Opt-Net's ability to elucidate biomarker differences across dementia stages.

XGBoost-SHAP-based interpretable diagnostic framework for alzheimer's disease

BACKGROUND

Due to the class imbalance issue faced when Alzheimer's disease (AD) develops from normal cognition (NC) to mild cognitive impairment (MCI), present clinical practice is met with challenges regarding the auxiliary diagnosis of AD using machine learning (ML). This leads to low diagnosis performance. We aimed to construct an interpretable framework, extreme gradient boosting-Shapley additive explanations (XGBoost-SHAP), to handle the imbalance among different AD progression statuses at the algorithmic level. We also sought to achieve multiclassification of NC, MCI, and AD.

METHODS

We obtained patient data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database, including clinical information, neuropsychological test results, neuroimaging-derived biomarkers, and APOE-ε4 gene statuses. First, three feature selection algorithms were applied, and they were then included in the XGBoost algorithm. Due to the imbalance among the three classes, we changed the sample weight distribution to achieve multiclassification of NC, MCI, and AD. Then, the SHAP method was linked to XGBoost to form an interpretable framework. This framework utilized attribution ideas that quantified the impacts of model predictions into numerical values and analysed them based on their directions and sizes. Subsequently, the top 10 features (optimal subset) were used to simplify the clinical decision-making process, and their performance was compared with that of a random forest (RF), Bagging, AdaBoost, and a naive Bayes (NB) classifier. Finally, the National Alzheimer's Coordinating Center (NACC) dataset was employed to assess the impact path consistency of the features within the optimal subset.

RESULTS

Compared to the RF, Bagging, AdaBoost, NB and XGBoost (unweighted), the interpretable framework had higher classification performance with accuracy improvements of 0.74%, 0.74%, 1.46%, 13.18%, and 0.83%, respectively. The framework achieved high sensitivity (81.21%/74.85%), specificity (92.18%/89.86%), accuracy (87.57%/80.52%), area under the receiver operating characteristic curve (AUC) (0.91/0.88), positive clinical utility index (0.71/0.56), and negative clinical utility index (0.75/0.68) on the ADNI and NACC datasets, respectively. In the ADNI dataset, the top 10 features were found to have varying associations with the risk of AD onset based on their SHAP values. Specifically, the higher SHAP values of CDRSB, ADAS13, ADAS11, ventricle volume, ADASQ4, and FAQ were associated with higher risks of AD onset. Conversely, the higher SHAP values of LDELTOTAL, mPACCdigit, RAVLT_immediate, and MMSE were associated with lower risks of AD onset. Similar results were found for the NACC dataset.

CONCLUSIONS

The proposed interpretable framework contributes to achieving excellent performance in imbalanced AD multiclassification tasks and provides scientific guidance (optimal subset) for clinical decision-making, thereby facilitating disease management and offering new research ideas for optimizing AD prevention and treatment programs.

Class imbalance should not throw you off balance: Choosing the right classifiers and performance metrics for brain decoding with imbalanced data

Machine learning (ML) is increasingly used in cognitive, computational and clinical neuroscience. The reliable and efficient application of ML requires a sound understanding of its subtleties and limitations. Training ML models on datasets with imbalanced classes is a particularly common problem, and it can have severe consequences if not adequately addressed. With the neuroscience ML user in mind, this paper provides a didactic assessment of the class imbalance problem and illustrates its impact through systematic manipulation of data imbalance ratios in (i) simulated data and (ii) brain data recorded with electroencephalography (EEG), magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI). Our results illustrate how the widely-used Accuracy (Acc) metric, which measures the overall proportion of successful predictions, yields misleadingly high performances, as class imbalance increases. Because Acc weights the per-class ratios of correct predictions proportionally to class size, it largely disregards the performance on the minority class. A binary classification model that learns to systematically vote for the majority class will yield an artificially high decoding accuracy that directly reflects the imbalance between the two classes, rather than any genuine generalizable ability to discriminate between them. We show that other evaluation metrics such as the Area Under the Curve (AUC) of the Receiver Operating Characteristic (ROC), and the less common Balanced Accuracy (BAcc) metric - defined as the arithmetic mean between sensitivity and specificity, provide more reliable performance evaluations for imbalanced data. Our findings also highlight the robustness of Random Forest (RF), and the benefits of using stratified cross-validation and hyperprameter optimization to tackle data imbalance. Critically, for neuroscience ML applications that seek to minimize overall classification error, we recommend the routine use of BAcc, which in the specific case of balanced data is equivalent to using standard Acc, and readily extends to multi-class settings. Importantly, we present a list of recommendations for dealing with imbalanced data, as well as open-source code to allow the neuroscience community to replicate and extend our observations and explore alternative approaches to coping with imbalanced data.

Leveraging History to Predict Infrequent Abnormal Transfers in Distributed Workflows

Scientific computing heavily relies on data shared by the community, especially in distributed data-intensive applications. This research focuses on predicting slow connections that create bottlenecks in distributed workflows. In this study, we analyze network traffic logs collected between January 2021 and August 2022 at the National Energy Research Scientific Computing Center (NERSC). Based on the observed patterns, we define a set of features primarily based on history for identifying low-performing data transfers. Typically, there are far fewer slow connections on well-maintained networks, which creates difficulty in learning to identify these abnormally slow connections from the normal ones. We devise several stratified sampling techniques to address the class-imbalance challenge and study how they affect the machine learning approaches. Our tests show that a relatively simple technique that undersamples the normal cases to balance the number of samples in two classes (normal and slow) is very effective for model training. This model predicts slow connections with an F1 score of 0.926.

A self-inspected adaptive SMOTE algorithm (SASMOTE) for highly imbalanced data classification in healthcare

In many healthcare applications, datasets for classification may be highly imbalanced due to the rare occurrence of target events such as disease onset. The SMOTE (Synthetic Minority Over-sampling Technique) algorithm has been developed as an effective resampling method for imbalanced data classification by oversampling samples from the minority class. However, samples generated by SMOTE may be ambiguous, low-quality and non-separable with the majority class. To enhance the quality of generated samples, we proposed a novel self-inspected adaptive SMOTE (SASMOTE) model that leverages an adaptive nearest neighborhood selection algorithm to identify the "visible" nearest neighbors, which are used to generate samples likely to fall into the minority class. To further enhance the quality of the generated samples, an uncertainty elimination via self-inspection approach is introduced in the proposed SASMOTE model. Its objective is to filter out the generated samples that are highly uncertain and inseparable with the majority class. The effectiveness of the proposed algorithm is compared with existing SMOTE-based algorithms and demonstrated through two real-world case studies in healthcare, including risk gene discovery and fatal congenital heart disease prediction. By generating the higher quality synthetic samples, the proposed algorithm is able to help achieve better prediction performance (in terms of F1 score) on average compared to the other methods, which is promising to enhance the usability of machine learning models on highly imbalanced healthcare data.

Machine learning to improve frequent emergency department use prediction: a retrospective cohort study

Frequent emergency department use is associated with many adverse events, such as increased risk for hospitalization and mortality. Frequent users have complex needs and associated factors are commonly evaluated using logistic regression. However, other machine learning models, especially those exploiting the potential of large databases, have been less explored. This study aims at comparing the performance of logistic regression to four machine learning models for predicting frequent emergency department use in an adult population with chronic diseases, in the province of Quebec (Canada). This is a retrospective population-based study using medical and administrative databases from the Régie de l'assurance maladie du Québec. Two definitions were used for frequent emergency department use (outcome to predict): having at least three and five visits during a year period. Independent variables included sociodemographic characteristics, healthcare service use, and chronic diseases. We compared the performance of logistic regression with gradient boosting machine, naïve Bayes, neural networks, and random forests (binary and continuous outcome) using Area under the ROC curve, sensibility, specificity, positive predictive value, and negative predictive value. Out of 451,775 ED users, 43,151 (9.5%) and 13,676 (3.0%) were frequent users with at least three and five visits per year, respectively. Random forests with a binary outcome had the lowest performances (ROC curve: 53.8 [95% confidence interval 53.5-54.0] and 51.4 [95% confidence interval 51.1-51.8] for frequent users 3 and 5, respectively) while the other models had superior and overall similar performance. The most important variable in prediction was the number of emergency department visits in the previous year. No model outperformed the others. Innovations in algorithms may slightly refine current predictions, but access to other variables may be more helpful in the case of frequent emergency department use prediction.

The framing of time-dependent machine learning models improves risk estimation among young individuals with acute coronary syndromes

Acute coronary syndrome (ACS) is a common cause of death in individuals older than 55 years. Although younger individuals are less frequently seen with ACS, this clinical event has increasing incidence trends, shows high recurrence rates and triggers considerable economic burden. Young individuals with ACS (yACS) are usually underrepresented and show idiosyncratic epidemiologic features compared to older subjects. These differences may justify why available risk prediction models usually penalize yACS with higher false positive rates compared to older subjects. We hypothesized that exploring temporal framing structures such as prediction time, observation windows and subgroup-specific prediction, could improve time-dependent prediction metrics. Among individuals who have experienced ACS (n_{global_cohort} = 6341 and n_yACS = 2242), the predictive accuracy for adverse clinical events was optimized by using specific rules for yACS and splitting short-term and long-term prediction windows, leading to the detection of 80% of events, compared to 69% by using a rule designed for the global cohort.

A theoretical distribution analysis of synthetic minority oversampling technique (SMOTE) for imbalanced learning

Class imbalance occurs when the class distribution is not equal. Namely, one class is under-represented (minority class), and the other class has significantly more samples in the data (majority class). The class imbalance problem is prevalent in many real world applications. Generally, the under-represented minority class is the class of interest. The synthetic minority over-sampling technique (SMOTE) method is considered the most prominent method for handling unbalanced data. The SMOTE method generates new synthetic data patterns by performing linear interpolation between minority class samples and their K nearest neighbors. However, the SMOTE generated patterns do not necessarily conform to the original minority class distribution. This paper develops a novel theoretical analysis of the SMOTE method by deriving the probability distribution of the SMOTE generated samples. To the best of our knowledge, this is the first work deriving a mathematical formulation for the SMOTE patterns’ probability distribution. This allows us to compare the density of the generated samples with the true underlying class-conditional density, in order to assess how representative the generated samples are. The derived formula is verified by computing it on a number of densities versus densities computed and estimated empirically.

Deep ROC Analysis and AUC as Balanced Average Accuracy, for Improved Classifier Selection, Audit and Explanation

Optimal performance is desired for decision-making in any field with binary classifiers and diagnostic tests, however common performance measures lack depth in information. The area under the receiver operating characteristic curve (AUC) and the area under the precision recall curve are too general because they evaluate all decision thresholds including unrealistic ones. Conversely, accuracy, sensitivity, specificity, positive predictive value and the F1 score are too specific-they are measured at a single threshold that is optimal for some instances, but not others, which is not equitable. In between both approaches, we propose deep ROC analysis to measure performance in multiple groups of predicted risk (like calibration), or groups of true positive rate or false positive rate. In each group, we measure the group AUC (properly), normalized group AUC, and averages of: sensitivity, specificity, positive and negative predictive value, and likelihood ratio positive and negative. The measurements can be compared between groups, to whole measures, to point measures and between models. We also provide a new interpretation of AUC in whole or part, as balanced average accuracy, relevant to individuals instead of pairs. We evaluate models in three case studies using our method and Python toolkit and confirm its utility.

Subpopulation-specific machine learning prognosis for underrepresented patients with double prioritized bias correction

Background

Many clinical datasets are intrinsically imbalanced, dominated by overwhelming majority groups. Off-the-shelf machine learning models that optimize the prognosis of majority patient types (e.g., healthy class) may cause substantial errors on the minority prediction class (e.g., disease class) and demographic subgroups (e.g., Black or young patients). In the typical one-machine-learning-model-fits-all paradigm, racial and age disparities are likely to exist, but unreported. In addition, some widely used whole-population metrics give misleading results.

Methods

We design a double prioritized (DP) bias correction technique to mitigate representational biases in machine learning-based prognosis. Our method trains customized machine learning models for specific ethnicity or age groups, a substantial departure from the one-model-predicts-all convention. We compare with other sampling and reweighting techniques in mortality and cancer survivability prediction tasks.

Results

We first provide empirical evidence showing various prediction deficiencies in a typical machine learning setting without bias correction. For example, missed death cases are 3.14 times higher than missed survival cases for mortality prediction. Then, we show DP consistently boosts the minority class recall for underrepresented groups, by up to 38.0%. DP also reduces relative disparities across race and age groups, e.g., up to 88.0% better than the 8 existing sampling solutions in terms of the relative disparity of minority class recall. Cross-race and cross-age-group evaluation also suggests the need for subpopulation-specific machine learning models.

Conclusions

Biases exist in the widely accepted one-machine-learning-model-fits-all-population approach. We invent a bias correction method that produces specialized machine learning prognostication models for underrepresented racial and age groups. This technique may reduce potentially life-threatening prediction mistakes for minority populations.

Recognition of the Multi-class Schizophrenia Based on the Resting-State EEG Network Topology

The clinical therapy of schizophrenia (SCZ) replies on the corresponding accurate and reliable recognition. Although efforts have been paid, the diagnosis of SCZ is still roughly subjective, it is thus urgent to search for related objective physiological parameters. Motivated by the great potential of resting-state networks in underling the brain deficits among different SCZ groups, in this study, we then developed a multi-class feature extraction approach that could effectively extract the spatial network topology and facilitate the recognition of the SCZ, by combining a network structure based supervised learning with an ensemble co-decision strategy. The results demonstrated that the multi-class spatial pattern of the network (MSPN) features outperformed the other conventional electrophysiological features, such as relative power spectrums and network properties, and achieved the highest classification accuracy of 71.58% in the alpha band. These findings did validate that the resting-state MSPN is a promising tool for the clinical assessment of the SCZ.

Developing a bioinformatics pipeline for comparative protein classification analysis

BACKGROUND

Protein classification is a task of paramount importance in various fields of biology. Despite the great momentum of modern implementation of protein classification, machine learning techniques such as Random Forest and Neural Network could not always be used for several reasons: data collection, unbalanced classification or labelling of the data.As an alternative, I propose the use of a bioinformatics pipeline to search for and classify information from protein databases. Hence, to evaluate the efficiency and accuracy of the pipeline, I focused on the carotenoid biosynthetic genes and developed a filtering approach to retrieve orthologs clusters in two well-studied plants that belong to the Brassicaceae family: Arabidopsis thaliana and Brassica rapa Pekinensis group. The result obtained has been compared with previous studies on carotenoid biosynthetic genes in B. rapa where phylogenetic analysis was conducted.

RESULTS

The developed bioinformatics pipeline relies on commercial software and multiple databeses including the use of phylogeny, Gene Ontology terms (GOs) and Protein Families (Pfams) at a protein level. Furthermore, the phylogeny is coupled with "population analysis" to evaluate the potential orthologs. All the steps taken together give a final table of potential orthologs. The phylogenetic tree gives a result of 43 putative orthologs conserved in B. rapa Pekinensis group. Different A. thaliana proteins have more than one syntenic ortholog as also shown in a previous finding (Li et al., BMC Genomics 16(1):1-11, 2015).

CONCLUSIONS

This study demonstrates that, when the biological features of proteins of interest are not specific, I can rely on a computational approach in filtering steps for classification purposes. The comparison of the results obtained here for the carotenoid biosynthetic genes with previous research confirmed the accuracy of the developed pipeline which can therefore be applied for filtering different types of datasets.

Machine Learning for the Prediction of Antiviral Compounds Targeting Avian Influenza A/H9N2 Viral Proteins

Avian influenza subtype A/H9N2—which infects chickens, reducing egg production by up to 80%—may be transmissible to humans. In humans, this virus is very harmful since it attacks the respiratory system and reproductive tract, replicating in both. Previous attempts to find antiviral candidates capable of inhibiting influenza A/H9N2 transmission were unsuccessful. This study aims to better characterize A/H9N2 to facilitate the discovery of antiviral compounds capable of inhibiting its transmission. The Symmetry of this study is to apply several machine learning methods to perform virtual screening to identify H9N2 antivirus candidates. The parameters used to measure the machine learning model’s quality included accuracy, sensitivity, specificity, balanced accuracy, and receiver operating characteristic score. We found that the extreme gradient boosting method yielded better results in classifying compounds predicted to be suitable antiviral compounds than six other machine learning methods, including logistic regression, k-nearest neighbor analysis, support vector machine, multilayer perceptron, random forest, and gradient boosting. Using this algorithm, we identified 10 candidate synthetic compounds with the highest scores. These high scores predicted that the molecular fingerprint may involve strong bonding characteristics. Thus, we were able to find significant candidates for synthetic H9N2 antivirus compounds and identify the best machine learning method to perform virtual screenings.

Prediction of Emergency Cesarean Section Using Machine Learning Methods: Development and External Validation of a Nationwide Multicenter Dataset in Republic of Korea

This study was a multicenter retrospective cohort study of term nulliparous women who underwent labor, and was conducted to develop an automated machine learning model for prediction of emergent cesarean section (CS) before onset of labor. Nine machine learning methods of logistic regression, random forest, Support Vector Machine (SVM), gradient boosting, extreme gradient boosting (XGBoost), light gradient boosting machine (LGBM), k-nearest neighbors (KNN), Voting, and Stacking were applied and compared for prediction of emergent CS during active labor. External validation was performed using a nationwide multicenter dataset for Korean fetal growth. A total of 6549 term nulliparous women was included in the analysis, and the emergent CS rate was 16.1%. The C-statistics values for KNN, Voting, XGBoost, Stacking, gradient boosting, random forest, LGBM, logistic regression, and SVM were 0.6, 0.69, 0.64, 0.59, 0.66, 0.68, 0.68, 0.7, and 0.69, respectively. The logistic regression model showed the best predictive performance with an accuracy of 0.78. The machine learning model identified nine significant variables of maternal age, height, weight at pre-pregnancy, pregnancy-associated hypertension, gestational age, and fetal sonographic findings. The C-statistic value for the logistic regression machine learning model in the external validation set (1391 term nulliparous women) was 0.69, with an overall accuracy of 0.68, a specificity of 0.83, and a sensitivity of 0.41. Machine learning algorithms with clinical and sonographic parameters at near term could be useful tools to predict individual risk of emergent CS during active labor in nulliparous women.

Assessing performance of artificial neural networks and re-sampling techniques for healthcare datasets

Re-sampling methods to solve class imbalance problems have shown to improve classification accuracy by mitigating the bias introduced by differences in class size. However, it is possible that a model which uses a specific re-sampling technique prior to Artificial neural networks (ANN) training may not be suitable for aid in classifying varied datasets from the healthcare industry. Five healthcare-related datasets were used across three re-sampling conditions: under-sampling, over-sampling and combi-sampling. Within each condition, different algorithmic approaches were applied to the dataset and the results were statistically analysed for a significant difference in ANN performance. The combi-sampling condition showed that four out of the five datasets did not show significant consistency for the optimal re-sampling technique between the f1-score and Area Under the Receiver Operating Characteristic Curve performance evaluation methods. Contrarily, the over-sampling and under-sampling condition showed all five datasets put forward the same optimal algorithmic approach across performance evaluation methods. Furthermore, the optimal combi-sampling technique (under-, over-sampling and convergence point), were found to be consistent across evaluation measures in only two of the five datasets. This study exemplifies how discrete ANN performances on datasets from the same industry can occur in two ways: how the same re-sampling technique can generate varying ANN performance on different datasets, and how different re-sampling techniques can generate varying ANN performance on the same dataset.

Multi-scale semi-supervised clustering of brain images: Deriving disease subtypes

Disease heterogeneity is a significant obstacle to understanding pathological processes and delivering precision diagnostics and treatment. Clustering methods have gained popularity for stratifying patients into subpopulations (i.e., subtypes) of brain diseases using imaging data. However, unsupervised clustering approaches are often confounded by anatomical and functional variations not related to a disease or pathology of interest. Semi-supervised clustering techniques have been proposed to overcome this and, therefore, capture disease-specific patterns more effectively. An additional limitation of both unsupervised and semi-supervised conventional machine learning methods is that they typically model, learn and infer from data using a basis of feature sets pre-defined at a fixed anatomical or functional scale (e.g., atlas-based regions of interest). Herein we propose a novel method, "Multi-scAle heteroGeneity analysIs and Clustering" (MAGIC), to depict the multi-scale presentation of disease heterogeneity, which builds on a previously proposed semi-supervised clustering method, HYDRA. It derives multi-scale and clinically interpretable feature representations and exploits a double-cyclic optimization procedure to effectively drive identification of inter-scale-consistent disease subtypes. More importantly, to understand the conditions under which the clustering model can estimate true heterogeneity related to diseases, we conducted extensive and systematic semi-simulated experiments to evaluate the proposed method on a sizeable healthy control sample from the UK Biobank (N = 4403). We then applied MAGIC to imaging data from Alzheimer's disease (ADNI, N = 1728) and schizophrenia (PHENOM, N = 1166) patients to demonstrate its potential and challenges in dissecting the neuroanatomical heterogeneity of common brain diseases. Taken together, we aim to provide guidance regarding when such analyses can succeed or should be taken with caution. The code of the proposed method is publicly available at https://github.com/anbai106/MAGIC.

Machine learning meets omics: applications and perspectives

The innovation of biotechnologies has allowed the accumulation of omics data at an alarming rate, thus introducing the era of 'big data'. Extracting inherent valuable knowledge from various omics data remains a daunting problem in bioinformatics. Better solutions often need some kind of more innovative methods for efficient handlings and effective results. Recent advancements in integrated analysis and computational modeling of multi-omics data helped address such needs in an increasingly harmonious manner. The development and application of machine learning have largely advanced our insights into biology and biomedicine and greatly promoted the development of therapeutic strategies, especially for precision medicine. Here, we propose a comprehensive survey and discussion on what happened, is happening and will happen when machine learning meets omics. Specifically, we describe how artificial intelligence can be applied to omics studies and review recent advancements at the interface between machine learning and the ever-widest range of omics including genomics, transcriptomics, proteomics, metabolomics, radiomics, as well as those at the single-cell resolution. We also discuss and provide a synthesis of ideas, new insights, current challenges and perspectives of machine learning in omics.

Predicting Brain Amyloid Using Multivariate Morphometry Statistics, Sparse Coding, and Correntropy: Validation in 1,101 Individuals From the ADNI and OASIS Databases

Biomarker assisted preclinical/early detection and intervention in Alzheimer’s disease (AD) may be the key to therapeutic breakthroughs. One of the presymptomatic hallmarks of AD is the accumulation of beta-amyloid (Aβ) plaques in the human brain. However, current methods to detect Aβ pathology are either invasive (lumbar puncture) or quite costly and not widely available (amyloid PET). Our prior studies show that magnetic resonance imaging (MRI)-based hippocampal multivariate morphometry statistics (MMS) are an effective neurodegenerative biomarker for preclinical AD. Here we attempt to use MRI-MMS to make inferences regarding brain Aβ burden at the individual subject level. As MMS data has a larger dimension than the sample size, we propose a sparse coding algorithm, Patch Analysis-based Surface Correntropy-induced Sparse-coding and Max-Pooling (PASCS-MP), to generate a low-dimensional representation of hippocampal morphometry for each individual subject. Then we apply these individual representations and a binary random forest classifier to predict brain Aβ positivity for each person. We test our method in two independent cohorts, 841 subjects from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) and 260 subjects from the Open Access Series of Imaging Studies (OASIS). Experimental results suggest that our proposed PASCS-MP method and MMS can discriminate Aβ positivity in people with mild cognitive impairment (MCI) [Accuracy (ACC) = 0.89 (ADNI)] and in cognitively unimpaired (CU) individuals [ACC = 0.79 (ADNI) and ACC = 0.81 (OASIS)]. These results compare favorably relative to measures derived from traditional algorithms, including hippocampal volume and surface area, shape measures based on spherical harmonics (SPHARM) and our prior Patch Analysis-based Surface Sparse-coding and Max-Pooling (PASS-MP) methods.

Predicting MCI to AD Conversation Using Integrated sMRI and rs-fMRI: Machine Learning and Graph Theory Approach

BACKGROUND

Graph theory and machine learning have been shown to be effective ways of classifying different stages of Alzheimer's disease (AD). Most previous studies have only focused on inter-subject classification with single-mode neuroimaging data. However, whether this classification can truly reflect the changes in the structure and function of the brain region in disease progression remains unverified. In the current study, we aimed to evaluate the classification framework, which combines structural Magnetic Resonance Imaging (sMRI) and resting-state functional Magnetic Resonance Imaging (rs-fMRI) metrics, to distinguish mild cognitive impairment non-converters (MCInc)/AD from MCI converters (MCIc) by using graph theory and machine learning.

METHODS

With the intra-subject (MCInc vs. MCIc) and inter-subject (MCIc vs. AD) design, we employed cortical thickness features, structural brain network features, and sub-frequency (full-band, slow-4, slow-5) functional brain network features for classification. Three feature selection methods [random subset feature selection algorithm (RSFS), minimal redundancy maximal relevance (mRMR), and sparse linear regression feature selection algorithm based on stationary selection (SS-LR)] were used respectively to select discriminative features in the iterative combinations of MRI and network measures. Then support vector machine (SVM) classifier with nested cross-validation was employed for classification. We also compared the performance of multiple classifiers (Random Forest, K-nearest neighbor, Adaboost, SVM) and verified the reliability of our results by upsampling.

RESULTS

We found that in the classifications of MCIc vs. MCInc, and MCIc vs. AD, the proposed RSFS algorithm achieved the best accuracies (84.71, 89.80%) than the other algorithms. And the high-sensitivity brain regions found with the two classification groups were inconsistent. Specifically, in MCIc vs. MCInc, the high-sensitivity brain regions associated with both structural and functional features included frontal, temporal, caudate, entorhinal, parahippocampal, and calcarine fissure and surrounding cortex. While in MCIc vs. AD, the high-sensitivity brain regions associated only with functional features included frontal, temporal, thalamus, olfactory, and angular.

CONCLUSIONS

These results suggest that our proposed method could effectively predict the conversion of MCI to AD, and the inconsistency of specific brain regions provides a novel insight for clinical AD diagnosis.

The 30-days hospital readmission risk in diabetic patients: predictive modeling with machine learning classifiers

BACKGROUND AND OBJECTIVES

Diabetes mellitus is a major chronic disease that results in readmissions due to poor disease control. Here we established and compared machine learning (ML)-based readmission prediction methods to predict readmission risks of diabetic patients.

METHODS

The dataset analyzed in this study was acquired from the Health Facts Database, which includes over 100,000 records of diabetic patients from 1999 to 2008. The basic data distribution characteristics of this dataset were summarized and then analyzed. In this study, 30-days readmission was defined as a readmission period of less than 30 days. After data preprocessing and normalization, multiple risk factors in the dataset were examined for classifier training to predict the probability of readmission using ML models. Different ML classifiers such as random forest, Naive Bayes, and decision tree ensemble were adopted to improve the clinical efficiency of the classification. In this study, the Konstanz Information Miner platform was used to preprocess and model the data, and the performances of the different classifiers were compared.

RESULTS

A total of 100,244 records were included in the model construction after the data preprocessing and normalization. A total of 23 attributes, including race, sex, age, admission type, admission location, length of stay, and drug use, were finally identified as modeling risk factors. Comparison of the performance indexes of the three algorithms revealed that the RF model had the best performance with a higher area under receiver operating characteristic curve (AUC) than the other two algorithms, suggesting that its use is more suitable for making readmission predictions.

CONCLUSION

The factors influencing 30-days readmission predictions in diabetic patients, including number of inpatient admissions, age, diagnosis, number of emergencies, and sex, would help healthcare providers to identify patients who are at high risk of short-term readmission and reduce the probability of 30-days readmission. The RF algorithm with the highest AUC is more suitable for making 30-days readmission predictions and  deserves further validation in clinical trials.

Computational methods for integrative evaluation of confidence, accuracy, and reaction time in facial affect recognition in schizophrenia

People with schizophrenia (SZ) process emotions less accurately than do healthy comparators (HC), and emotion recognition have expanded beyond accuracy to performance variables like reaction time (RT) and confidence. These domains are typically evaluated independently, but complex inter-relationships can be evaluated through machine learning at an item-by-item level. Using a mix of ranking and machine learning tools, we investigated item-by-item discrimination of facial affect with two emotion recognition tests (BLERT and ER-40) between SZ and HC. The best performing multi-domain model for ER40 had a large effect size in differentiating SZ and HC (d = 1.24) compared to a standard comparison of accuracy alone (d = 0.48); smaller increments in effect sizes were evident for the BLERT (d = 0.87 vs. d = 0.58). Almost half of the selected items were confidence ratings. Within SZ, machine learning models with ER40 (generally accuracy and reaction time) items predicted severity of depression and overconfidence in social cognitive ability, but not psychotic symptoms. Pending independent replication, the results support machine learning, and the inclusion of confidence ratings, in characterizing the social cognitive deficits in SZ. This moderate-sized study (n = 372) included subjects with schizophrenia (SZ, n = 218) and healthy controls (HC, n = 154).

•

This paper explores the value of integrative evaluation of confidence, accuracy, and reaction time by way of machine learning in understanding the unique aspects of facial affect recognition in schizophrenia.

•

Machine learning models better separated schizophrenia from healthy comparators that standard statistical comparison, confidence ratings contributed to this separation in a disproportionate manner.

•

Machine learning approaches provide a novel way to analyze item-by-item associations with social cognition measures, or potentially other tests, where multiple overlapping dimensions exist.

•

Aberrant confidence ratings interact with performance variables in complex ways to contribute to social cognitive deficits in schizophrenia.

Reproducible Evaluation of Diffusion MRI Features for Automatic Classification of Patients with Alzheimer's Disease

Diffusion MRI is the modality of choice to study alterations of white matter. In past years, various works have used diffusion MRI for automatic classification of Alzheimer's disease. However, classification performance obtained with different approaches is difficult to compare because of variations in components such as input data, participant selection, image preprocessing, feature extraction, feature rescaling (FR), feature selection (FS) and cross-validation (CV) procedures. Moreover, these studies are also difficult to reproduce because these different components are not readily available. In a previous work (Samper-González et al. 2018), we propose an open-source framework for the reproducible evaluation of AD classification from T1-weighted (T1w) MRI and PET data. In the present paper, we first extend this framework to diffusion MRI data. Specifically, we add: conversion of diffusion MRI ADNI data into the BIDS standard and pipelines for diffusion MRI preprocessing and feature extraction. We then apply the framework to compare different components. First, FS has a positive impact on classification results: highest balanced accuracy (BA) improved from 0.76 to 0.82 for task CN vs AD. Secondly, voxel-wise features generally gives better performance than regional features. Fractional anisotropy (FA) and mean diffusivity (MD) provided comparable results for voxel-wise features. Moreover, we observe that the poor performance obtained in tasks involving MCI were potentially caused by the small data samples, rather than by the data imbalance. Furthermore, no extensive classification difference exists for different degree of smoothing and registration methods. Besides, we demonstrate that using non-nested validation of FS leads to unreliable and over-optimistic results: 5% up to 40% relative increase in BA. Lastly, with proper FR and FS, the performance of diffusion MRI features is comparable to that of T1w MRI. All the code of the framework and the experiments are publicly available: general-purpose tools have been integrated into the Clinica software package ( www.clinica.run ) and the paper-specific code is available at: https://github.com/aramis-lab/AD-ML .

Use of Machine Learning to Determine the Information Value of a BMI Screening Program

INTRODUCTION

Childhood obesity continues to be a significant public health issue in the U.S. and is associated with short- and long-term adverse health outcomes. A number of states have implemented school-based BMI screening programs. However, these programs have been criticized for not being effective in improving students' BMI or reducing childhood obesity. One potential benefit, however, of screening programs is the identification of younger children at risk of obesity as they age.

METHODS

This study used a unique panel data set from the BMI screening program for public school children in the state of Arkansas collected from 2003 to 2004 through the 2018-2019 academic years and analyzed in 2020. Machine learning algorithms were applied to understand the informational value of BMI screening. Specifically, this study evaluated the importance of BMI information during kindergarten to the accurate prediction of childhood obesity by the 4th grade.

RESULTS

Kindergarten BMI z-score is the most important predictor of obesity by the 4th grade and is much more important to prediction than sociodemographic and socioeconomic variables that would otherwise be available to policymakers in the absence of the screening program. Including the kindergarten BMI z-score of students in the model meaningfully increases the accuracy of the prediction.

CONCLUSIONS

Data from the Arkansas BMI screening program greatly improve the ability to identify children at greatest risk of future obesity to the extent that better prediction can be translated into more effective policy and better health outcomes. This is a heretofore unexamined benefit of school-based BMI screening.

Injury risk assessment based on pre-crash variables: The role of closing velocity and impact eccentricity

Thorough evaluations on injury risk (IR) are fundamental for guiding interventions toward the enhancement of both the road infrastructure and the active/passive safety of vehicles. Well-established estimates are currently based on IR functions modeled on post-crash variables, such as velocity change sustained by the vehicle (ΔV); thence, these analyses do not directly suggest how pre-crash conditions can be modified to allow for IR reduction. Nevertheless, ΔV can be disaggregated into two contributions which enable its apriori calculation, based only on the information available at the impact instant: the Crash Momentum Index (CMI), representing impact eccentricity at collision, and the closing velocity at collision (V_r). By extensively employing the CMI indicator, this work assesses the overall influence of impact eccentricity and closing velocity on the risk for occupants to sustain a serious injury. As CMI synthesizes indications regarding ΔV, its use can be disjointed from the ΔV itself for the derivation of high-quality IR models. This feature distinguishes CMI from the other eccentricity indicators available at the state-of-the-art, allowing for the contribution of eccentricity on IR to be completely isolated. Because of this element of originality, special attention is given to the CMI variable throughout the present work. Based on data extracted from the NASS/CDS database, the influence of the CMI and V_r variables on IR is specifically highlighted and analyzed from several perspectives. The feature ranking algorithm ReliefF, whose use is unprecedented in the accident analysis field, is first employed to assess importance of such impact-related variables in determining the injury outcome: if compared to vehicle-related and occupant-related variables (as category and age, respectively), the higher influence of CMI and V_r is initially highlighted. Secondly, the relevance of CMI and V_r is confirmed by fitting different predictive models: the fitted models which include the CMI predictor perform better than models which neglect the CMI, in terms of classical evaluation metrics. As a whole, considering the high predictive power of the proposed CMI-based models, this work provides valuable tools for the apriori assessment of IR.

Development of a periodontitis risk assessment model for primary care providers in an interdisciplinary setting

BACKGROUND

Periodontitis (PD), a form of gum disease, is a major public health concern as it is globally prevalent and harms both individual quality of life and economic productivity. Global cost in lost productivity is estimated at US$54 billion annually. Moreover, current PD assessment applies only after the damage has already occurred.

OBJECTIVE

This study proposes and tests a new PD risk assessment model applicable at point-of-care, using supervised machine learning methods.

METHODS

We compare the performance of five algorithms using retrospective clinical data: Naïve Bayes (NB), Logistic Regression (LR), Support Vector Machine (SVM), Artificial Neural Network (ANN), and Decision Tree (DT).

RESULTS

DT and ANN demonstrated higher accuracy in classifying the patients with high or low PD risk as compared to NB, LR and SVM. The resultant model with DT showed a sensitivity of 87.08% (95% CI 84.12% to 89.76%) and specificity of 93.5% (95% CI 91% to 95.49%).

CONCLUSIONS

A predictive model with high sensitivity and specificity to stratify individuals into low and high PD risk tiers was developed. Validation in other populations will inform translational value of this approach and its potential applicability as clinical decision support tool.

Multiset Feature Learning for Highly Imbalanced Data Classification

With the expansion of data, increasing imbalanced data has emerged. When the imbalance ratio (IR) of data is high, most existing imbalanced learning methods decline seriously in classification performance. In this paper, we systematically investigate the highly imbalanced data classification problem, and propose an uncorrelated cost-sensitive multiset learning (UCML) approach for it. Specifically, UCML first constructs multiple balanced subsets through random partition, and then employs the multiset feature learning (MFL) to learn discriminant features from the constructed multiset. To enhance the usability of each subset and deal with the non-linearity issue existed in each subset, we further propose a deep metric based UCML (DM-UCML) approach. DM-UCML introduces the generative adversarial network technique into the multiset constructing process, such that each subset can own similar distribution with the original dataset. To cope with the non-linearity issue, DM-UCML integrates deep metric learning with MFL, such that more favorable performance can be achieved. In addition, DM-UCML designs a new discriminant term to enhance the discriminability of learned metrics. Experiments on eight traditional highly class-imbalanced datasets and two large-scale datasets indicate that: the proposed approaches outperform state-of-the-art highly imbalanced learning methods and are more robust to high IR.

Dysfunctional Architecture Underlies White Matter Hyperintensities with and without Cognitive Impairment

BACKGROUND

White matter hyperintensities (WMH) are common in older adults and are associated with cognitive decline. However, little is known about the functional changes underlying cognitive decline in WMH subjects.

OBJECTIVES

To investigate whole-brain functional connectivity (FC) underpinnings of cognitive decline in WMH subjects using univariate and multivariate analyses.

METHODS

Twenty-three WMH subjects with mild cognitive impairment (WMH-MCI), 43 WMH subjects with no cognitive impairment (WMH-nCI), and 55 healthy controls underwent resting-state functional MRI scans. Whole-brain FC was calculated using the fine-grained human Brainnetome Atlas, followed by performance of between-group comparisons and FC-cognition correlation analysis. A multivariate analysis using support vector machine (SVM) was performed to classify WMH-MCI and WMH-nCI subjects based on FC.

RESULTS

Both the WMH-MCI and WMH-nCI subjects exhibited characteristic impaired FC patterns. Markedly reduced FC involving subcortical nuclei and cortical hub regions of cognitive networks, especially the cingulate cortex, was identified in the WMH-MCI patients. In the WMH-MCI group, several connections involving the cingulate cortex were associated with cognitive decline. The exploratory mediation analyses indicated that FC alterations could partially explain the association between WMH and cognition. Furthermore, an SVM classifier based on FC distinguished WMH-MCI and WMH-nCI subjects with 78.8% accuracy. Connections that contributed most to the classification showed a similar distribution as the connections identified in the univariate analysis.

CONCLUSIONS

This study provides a new window into the pathophysiology of cognitive impairment in WMH subjects and offer a novel and potential approach for early detection of the cognitive impairment in WMH subjects at the individual level.

Prediction of physical violence in schizophrenia with machine learning algorithms

Patients with schizophrenia have been shown to have an increased risk for physical violence. While certain features have been identified as risk factors, it has been difficult to integrate these variables to identify violent patients. The present study thus attempts to develop a clinically-relevant predictive tool. In a population of 275 schizophrenia patients, we identified 103 participants as violent and 172 as non-violent through electronic medical documentation, and conducted cross-sectional assessments to identify demographic, clinical, and sociocultural variables. Using these predictors, we utilized seven machine learning classification algorithms to predict for past instances of physical violence. Our classification algorithms predicted with significant accuracy compared to random discrimination alone, and had varying degrees of predictive power, as described by various performance measures. We determined that the random forest model performed marginally better than other algorithms, with an accuracy of 62% and an area under the receiver operator characteristic curve (AUROC) of 0.63. To summarize, machine learning classification algorithms are becoming increasingly valuable, though, optimization of these models is needed to better complement diagnostic decisions regarding early interventional measures to predict instances of physical violence.

Machine learning analyses of methylation profiles uncovers tissue-specific gene expression patterns in wheat

DNA methylation is a mechanism of epigenetic modification in eukaryotic organisms. Generally, methylation within genes promoter inhibits regulatory protein binding and represses transcription, whereas gene body methylation is associated with actively transcribed genes. However, it remains unclear whether there is interaction between methylation levels across genic regions and which site has the biggest impact on gene regulation. We investigated and used the methylation patterns of the bread wheat cultivar Chinese Spring to uncover differentially expressed genes (DEGs) between roots and leaves, using six machine learning algorithms and a deep neural network. As anticipated, genes with higher expression in leaves were mainly involved in photosynthesis and pigment biosynthesis processes whereas genes that were not differentially expressed between roots and leaves were involved in protein processes and membrane structures. Methylation occurred preponderantly (60%) in the CG context, whereas 35 and 5% of methylation occurred in CHG and CHH contexts, respectively. Methylation levels were highly correlated (r = 0.7 to 0.9) between all genic regions, except within the promoter (r = 0.4 to 0.5). Machine learning models gave a high (0.81) prediction accuracy of DEGs. There was a strong correlation (p-value = 9.20×10^-10 ) between all features and gene expression, suggesting that methylation across all genic regions contribute to gene regulation. However, the methylation of the promoter, the CDS and the exon in CG context was the most impactful. Our study provides more insights into the interplay between DNA methylation and gene expression and paves the way for identifying tissue-specific genes using methylation profiles.

Hellinger distance-based stable sparse feature selection for high-dimensional class-imbalanced data

Background

Feature selection in class-imbalance learning has gained increasing attention in recent years due to the massive growth of high-dimensional class-imbalanced data across many scientific fields. In addition to reducing model complexity and discovering key biomarkers, feature selection is also an effective method of combating overlapping which may arise in such data and become a crucial aspect for determining classification performance. However, ordinary feature selection techniques for classification can not be simply used for addressing class-imbalanced data without any adjustment. Thus, more efficient feature selection technique must be developed for complicated class-imbalanced data, especially in the context of high-dimensionality.

Results

We proposed an algorithm called sssHD to achieve stable sparse feature selection applied it to complicated class-imbalanced data. sssHD is based on the Hellinger distance (HD) coupled with sparse regularization techniques. We stated that Hellinger distance is not only class-insensitive but also translation-invariant. Simulation result indicates that HD-based selection algorithm is effective in recognizing key features and control false discoveries for class-imbalance learning. Five gene expression datasets are also employed to test the performance of the sssHD algorithm, and a comparison with several existing selection procedures is performed. The result shows that sssHD is highly competitive in terms of five assessment metrics. In addition, sssHD presents limited differences between performing and not performing re-balance preprocessing.

Conclusions

sssHD is a practical feature selection method for high-dimensional class-imbalanced data, which is simple and can be an alternative for performing feature selection in class-imbalanced data. sssHD can be easily extended by connecting it with different re-balance preprocessing, different sparse regularization structures as well as different classifiers. As such, the algorithm is extremely general and has a wide range of applicability.

Data Imbalance in Autism Pre-Diagnosis Classification Systems: An Experimental Study

Machine learning (ML) is a branch of computer science that is rapidly gaining popularity within the healthcare arena due to its ability to explore large datasets to discover useful patterns that can be interepreted for decision-making and prediction. ML techniques are used for the analysis of clinical parameters and their combinations for prognosis, therapy planning and support and patient management and wellbeing. In this research, we investigate a crucial problem associated with medical applications such as autism spectrum disorder (ASD) data imbalances in which cases are far more than just controls in the dataset. In autism diagnosis data, the number of possible instances is linked with one class, i.e. the no ASD is larger than the ASD, and this may cause performance issues such as models favouring the majority class and undermining the minority class. This research experimentally measures the impact of class imbalance issue on the performance of different classifiers on real autism datasets when various data imbalance approaches are utilised in the pre-processing phase. We employ oversampling techniques, such as Synthetic Minority Oversampling (SMOTE), and undersampling with different classifiers including Naive Bayes, RIPPER, C4.5 and Random Forest to measure the impact of these on the performance of the models derived in terms of area under curve and other metrics. Results pinpoint that oversampling techniques are superior to undersampling techniques, at least for the toddlers’ autism dataset that we consider, and suggest that further work should look at incorporating sampling techniques with feature selection to generate models that do not overfit the dataset.

Identification of newborns at risk for autism using electronic medical records and machine learning

BACKGROUND

Current approaches for early identification of individuals at high risk for autism spectrum disorder (ASD) in the general population are limited, and most ASD patients are not identified until after the age of 4. This is despite substantial evidence suggesting that early diagnosis and intervention improves developmental course and outcome. The aim of the current study was to test the ability of machine learning (ML) models applied to electronic medical records (EMRs) to predict ASD early in life, in a general population sample.

METHODS

We used EMR data from a single Israeli Health Maintenance Organization, including EMR information for parents of 1,397 ASD children (ICD-9/10) and 94,741 non-ASD children born between January 1st, 1997 and December 31st, 2008. Routinely available parental sociodemographic information, parental medical histories, and prescribed medications data were used to generate features to train various ML algorithms, including multivariate logistic regression, artificial neural networks, and random forest. Prediction performance was evaluated with 10-fold cross-validation by computing the area under the receiver operating characteristic curve (AUC; C-statistic), sensitivity, specificity, accuracy, false positive rate, and precision (positive predictive value [PPV]).

RESULTS

All ML models tested had similar performance. The average performance across all models had C-statistic of 0.709, sensitivity of 29.93%, specificity of 98.18%, accuracy of 95.62%, false positive rate of 1.81%, and PPV of 43.35% for predicting ASD in this dataset.

CONCLUSIONS

We conclude that ML algorithms combined with EMR capture early life ASD risk as well as reveal previously unknown features to be associated with ASD-risk. Such approaches may be able to enhance the ability for accurate and efficient early detection of ASD in large populations of children.

DeepSnap-Deep Learning Approach Predicts Progesterone Receptor Antagonist Activity With High Performance

The progesterone receptor (PR) is important therapeutic target for many malignancies and endocrine disorders due to its role in controlling ovulation and pregnancy via the reproductive cycle. Therefore, the modulation of PR activity using its agonists and antagonists is receiving increasing interest as novel treatment strategy. However, clinical trials using the PR modulators have not yet been found conclusive evidences. Recently, increasing evidence from several fields shows that the classification of chemical compounds, including agonists and antagonists, can be done with recent improvements in deep learning (DL) using deep neural network. Therefore, we recently proposed a novel DL-based quantitative structure-activity relationship (QSAR) strategy using transfer learning to build prediction models for agonists and antagonists. By employing this novel approach, referred as DeepSnap-DL method, which uses images captured from 3-dimension (3D) chemical structure with multiple angles as input data into the DL classification, we constructed prediction models of the PR antagonists in this study. Here, the DeepSnap-DL method showed a high performance prediction of the PR antagonists by optimization of some parameters and image adjustment from 3D-structures. Furthermore, comparison of the prediction models from this approach with conventional machine learnings (MLs) indicated the DeepSnap-DL method outperformed these MLs. Therefore, the models predicted by DeepSnap-DL would be powerful tool for not only QSAR field in predicting physiological and agonist/antagonist activities, toxicity, and molecular bindings; but also for identifying biological or pathological phenomena.

Development of an Al-Based Web Diagnostic System for Phenotyping Psychiatric Disorders

Background: Artificial intelligence (AI)-based medical diagnostic applications are on the rise. Our recent study has suggested an explainable deep neural network (EDNN) framework for identifying key structural deficits related to the pathology of schizophrenia. Here, we presented an AI-based web diagnostic system for schizophrenia under the EDNN framework with three-dimensional (3D) visualization of subjects' neuroimaging dataset. Methods: This AI-based web diagnostic system consisted of a web server and a neuroimaging diagnostic database. The web server deployed the EDNN algorithm under the Node.js environment. Feature selection and network model building were performed on the dataset obtained from two hundred schizophrenic patients and healthy controls in the Taiwan Aging and Mental Illness (TAMI) cohort. We included an independent cohort with 88 schizophrenic patients and 44 healthy controls recruited at Tri-Service General Hospital Beitou Branch for validation purposes. Results: Our AI-based web diagnostic system achieved 84.00% accuracy (89.47% sensitivity, 80.62% specificity) for gray matter (GM) and 90.22% accuracy (89.21% sensitivity, 91.23% specificity) for white matter (WM) on the TAMI cohort. For the Beitou cohort as an unseen test set, the model achieved 77.27 and 70.45% accuracy for GM and WM. Furthermore, it achieved 85.50 and 88.20% accuracy after model retraining to mitigate the effects of drift on the predictive capability. Moreover, our system visualized the identified voxels in brain atrophy in a 3D manner with patients' structural image, optimizing the evaluation process of the diagnostic results. Discussion: Together, our approach under the EDNN framework demonstrated the potential future direction of making a schizophrenia diagnosis based on structural brain imaging data. Our deep learning model is explainable, arguing for the accuracy of the key information related to the pathology of schizophrenia when using the AI-based web assessment platform. The rationale of this approach is in accordance with the Research Domain Criteria suggested by the National Institute of Mental Health.

Machine learning algorithms, bull genetic information, and imbalanced datasets used in abortion incidence prediction models for Iranian Holstein dairy cattle

The ability to predict abortion incidence, especially in regions with high abortion rates (e.g., Iran), helps improve reproductive performance and, thereby, dairy farm profitability. The objective of this study was to predict pregnancy loss in Iranian dairy herds. For this purpose, the cow history records and bull genetic information available at 6 large commercial dairy farms with cows calved between 2005 and 2014 were extracted from an on-farm record-keeping software. Using WEKA, 12 commonly used machine learning (ML) algorithms were applied to the dataset. The algorithms belonged to 5 classifier groups which were Bayes, meta, functions, rules, and trees. The original dataset including herd-cow factors was randomly divided into 2 subsets: a training dataset and a test one (at a ratio of 60:40). The original dataset was combined with the bull genetic information to create a full dataset. The average abortion rate was 15.4 %, which represented an imbalanced dataset. Therefore, 2 down- and up-sampling techniques were additionally implemented on the original dataset (more specifically on the training one) to create 2 balanced datasets. This ultimately yielded 4 datasets; original, full, down-sampling, and up-sampling. Different algorithms and models were evaluated based on F-measure and area under the curve (AUC). Based on the results obtained, ML algorithms exhibited a high performance in predicting abortion when applied to the balanced dataset. However, their performance varied from 32.3 % (poor) to 69.2 % (medium upward) when applied to the imbalanced original dataset. In addition to the imbalance in the original dataset, the reason for these poor results were attributed to the high proportion of unknown risk factors underlying abortion incidence. Even when including the bull genetic information, it did not lead to any significant improvements in the prediction model. From among the datasets used, the Bayes algorithms outperformed the others in predicting pregnancy losses while rules had the worst performance. Furthermore, while the Bayes algorithms were not affected by the type of dataset (balanced or imbalanced), substantial increases in F-measure and AUC were observed for rules, trees, and functions with balanced datasets. Overall, the balanced models outperformed the others, with the down-sampling method exhibiting the highest performance. Despite the fact that the prediction models used in this study did not perform as expected, it was shown that they can be beneficially used to predict and reduce pregnancy losses, despite their moderate accuracy, especially when used for herds with high abortion rates and low reproductive performances.