Genetic Algorithm-Neural Network (GANN): a study of neural network activation functions and depth of genetic algorithm search applied to feature selection

A Multimorbidity Analysis of Hospitalized Patients With COVID-19 in Northwest Italy: Longitudinal Study Using Evolutionary Machine Learning and Health Administrative Data

BACKGROUND

Multimorbidity is a significant public health concern, characterized by the coexistence and interaction of multiple preexisting medical conditions. This complex condition has been associated with an increased risk of COVID-19. Individuals with multimorbidity who contract COVID-19 often face a significant reduction in life expectancy. The postpandemic period has also highlighted an increase in frailty, emphasizing the importance of integrating existing multimorbidity details into epidemiological risk assessments. Managing clinical data that include medical histories presents significant challenges, particularly due to the sparsity of data arising from the rarity of multimorbidity conditions. Also, the complex enumeration of combinatorial multimorbidity features introduces challenges associated with combinatorial explosions.

OBJECTIVE

This study aims to assess the severity of COVID-19 in individuals with multiple medical conditions, considering their demographic characteristics such as age and sex. We propose an evolutionary machine learning model designed to handle sparsity, analyzing preexisting multimorbidity profiles of patients hospitalized with COVID-19 based on their medical history. Our objective is to identify the optimal set of multimorbidity feature combinations strongly associated with COVID-19 severity. We also apply the Apriori algorithm to these evolutionarily derived predictive feature combinations to identify those with high support.

METHODS

We used data from 3 administrative sources in Piedmont, Italy, involving 12,793 individuals aged 45-74 years who tested positive for COVID-19 between February and May 2020. From their 5-year pre-COVID-19 medical histories, we extracted multimorbidity features, including drug prescriptions, disease diagnoses, sex, and age. Focusing on COVID-19 hospitalization, we segmented the data into 4 cohorts based on age and sex. Addressing data imbalance through random resampling, we compared various machine learning algorithms to identify the optimal classification model for our evolutionary approach. Using 5-fold cross-validation, we evaluated each model's performance. Our evolutionary algorithm, utilizing a deep learning classifier, generated prediction-based fitness scores to pinpoint multimorbidity combinations associated with COVID-19 hospitalization risk. Eventually, the Apriori algorithm was applied to identify frequent combinations with high support.

RESULTS

We identified multimorbidity predictors associated with COVID-19 hospitalization, indicating more severe COVID-19 outcomes. Frequently occurring morbidity features in the final evolved combinations were age>53, R03BA (glucocorticoid inhalants), and N03AX (other antiepileptics) in cohort 1; A10BA (biguanide or metformin) and N02BE (anilides) in cohort 2; N02AX (other opioids) and M04AA (preparations inhibiting uric acid production) in cohort 3; and G04CA (Alpha-adrenoreceptor antagonists) in cohort 4.

CONCLUSIONS

When combined with other multimorbidity features, even less prevalent medical conditions show associations with the outcome. This study provides insights beyond COVID-19, demonstrating how repurposed administrative data can be adapted and contribute to enhanced risk assessment for vulnerable populations.

Application of Meta-Heuristic Algorithms for Training Neural Networks and Deep Learning Architectures: A Comprehensive Review

The learning process and hyper-parameter optimization of artificial neural networks (ANNs) and deep learning (DL) architectures is considered one of the most challenging machine learning problems. Several past studies have used gradient-based back propagation methods to train DL architectures. However, gradient-based methods have major drawbacks such as stucking at local minimums in multi-objective cost functions, expensive execution time due to calculating gradient information with thousands of iterations and needing the cost functions to be continuous. Since training the ANNs and DLs is an NP-hard optimization problem, their structure and parameters optimization using the meta-heuristic (MH) algorithms has been considerably raised. MH algorithms can accurately formulate the optimal estimation of DL components (such as hyper-parameter, weights, number of layers, number of neurons, learning rate, etc.). This paper provides a comprehensive review of the optimization of ANNs and DLs using MH algorithms. In this paper, we have reviewed the latest developments in the use of MH algorithms in the DL and ANN methods, presented their disadvantages and advantages, and pointed out some research directions to fill the gaps between MHs and DL methods. Moreover, it has been explained that the evolutionary hybrid architecture still has limited applicability in the literature. Also, this paper classifies the latest MH algorithms in the literature to demonstrate their effectiveness in DL and ANN training for various applications. Most researchers tend to extend novel hybrid algorithms by combining MHs to optimize the hyper-parameters of DLs and ANNs. The development of hybrid MHs helps improving algorithms performance and capable of solving complex optimization problems. In general, the optimal performance of the MHs should be able to achieve a suitable trade-off between exploration and exploitation features. Hence, this paper tries to summarize various MH algorithms in terms of the convergence trend, exploration, exploitation, and the ability to avoid local minima. The integration of MH with DLs is expected to accelerate the training process in the coming few years. However, relevant publications in this way are still rare.

A Novel Genetic Neural Network Algorithm with Link Switches and Its Application in University Professional Course Evaluation

This study exploits a novel enhanced genetic neural network algorithm with link switches (EGA-NNLS) to model the professional university course evaluating system. Various indices should be employed to evaluate the learning effect of a professional course comprehensively and objectively, and the traditional artificial evaluation methods cannot achieve this goal. The presented data-driven modeling method, EGA-NNLS, combines a neural network with link switches (NN-LS) with an enhanced genetic algorithm (EGA) and the Levenberg-Marquardt (LM) algorithm. It employs an optimized network structure combined with EGA and NN-LS to learn the relationships between the system's input and output from historical data and uses the network's gradient information via the LM algorithm. Compared with the traditional backpropagation neural network (BPNN), EGA-NNLS achieves a faster convergence speed and higher evaluation precision. In order to verify the efficiency of EGA-NNLS, it is applied to a collection of experimental data for modeling the professional university course evaluating system.

Prediction of Concrete Compressive Strength Using a Back-Propagation Neural Network Optimized by a Genetic Algorithm and Response Surface Analysis Considering the Appearance of Aggregates and Curing Conditions

In the present research, the authors have attempted to examine the compressive strength of conventional concrete, which is made using different aggregate sizes and geometries considering various curing temperatures. To this end, different aggregate geometries (rounded and angular) were utilized in various aggregate sizes (10, 20, and 30 mm) to prepare 108 rectangular cubic specimens. Then, the curing process was carried out in the vicinity of wind at different temperatures (5 °C < T < 30 °C). Next, the static compression experiments were performed on 28-day concrete specimens. Additionally, each test was repeated three times to check the repeatability of the results. Finally, the mean results were reported as the strength of concrete specimens. Response Surface Analysis (RSA) was utilized to determine the interaction effects of different parameters including the appearance of aggregates (shape and size) and curing temperature on the concrete strength. Afterwards, the optimum values of parameters were reported based on the RSA results to achieve maximum compressive strength. Moreover, to estimate concrete strength, a back-propagation neural network (OBPNN) optimized by a genetic algorithm (GA) was used. The findings of this study indicated that the developed neural network approach is greatly consistent with the experimental ones. Additionally, the compressive strength of concrete can be significantly increased (about 30%) by controlling the curing temperature in the range of 5–15 °C.

Applying hybrid genetic and artificial bee colony algorithms to simulate a bio-treatment of synthetic dye-polluted wastewater using a rhamnolipid biosurfactant

The present work aims at optimization and advanced simulation of removal efficiency of dye material from a synthetic wastewater using a locally generated rhamnolipid (RL) biosurfactant. For this purpose, bio-treatment of dye polluted synthetic wastewater was experimentally, kinetically, and statistically investigated by the ion flotation process in the presence of the RL. The removal rate of methylene blue (MB) as the dye material was assessed by the ultraviolet (UV)-visible absorbance measurements. The impact of operating variables including RL concentration (as a dye collector, 5-50 ppm), methyl isobutyl carbinol (MIBC) dosage (as a frother, 10-70 ppm), solution pH (2-12) and aeration rate (1-5 l/min) were assessed through one-way analysis of variance (ANOVA) and Anderson-Darling as the normality analysis strategy. The process was simulated using two artificial neural network (ANN) optimization algorithms, i.e., genetic algorithm (GA) and artificial bee colony (ABC) as a novel approach. The statistical results indicated that the dye removal process was significantly influenced by all operating variables (p_value<0.05) while their relative intensity followed the order of aeration rate > solution pH > RL concentration > MIBC dosage. Anderson-Darling approach disclosed that the all factors were perfectly followed the normal trend with A² less than unity and p-value of greater than 0.05 at 95% confidence level. Main effect plots revealed that except MIBC dosage with nonlinear trend, the rest of factors had an ascending influence on the removal efficiency. The process was optimized by interpreting the interaction effect among various variables to reach the maximum dye bioflotation. The maximum removal of 97 ± 0.13% was achieved at pH 12, airflow rate of 5 l/min, MIBC and rhamnolipid concentrations of 30 and 40 ppm, respectively with a flotation kinetic rate of 0.015 sec^-1. Finally, the intelligent simulation results showed that the process could be modelled using an artificial bee colony algorithm of 4-7-1 structure with 99% and 98.8% accuracies in the training and testing steps, respectively. Further, we found that the artificial bee colony algorithm was superior to the genetic algorithm in terms of complexity analysis.

AN IMPROVED GREY WOLF OPTIMIZATION-BASED LEARNING OF ARTIFICIAL NEURAL NETWORK FOR MEDICAL DATA CLASSIFICATION

Grey wolf optimization (GWO) is a recent and popular swarm-based metaheuristic approach. It has been used in numerous fields such as numerical optimization, engineering problems, and machine learning. The different variants of GWO have been developed in the last 5 years for solving optimization problems in diverse fields. Like other metaheuristic algorithms, GWO also suffers from local optima and slow convergence problems, resulted in degraded performance. An adequate equilibrium among exploration and exploitation is a key factor to the success of meta-heuristic algorithms especially for optimization task. In this paper, a new variant of GWO, called inertia motivated GWO (IMGWO) is proposed. The aim of IMGWO is to establish better balance between exploration and exploitation. Traditionally, artificial neural network (ANN) with backpropagation (BP) depends on initial values and in turn, attains poor convergence. The metaheuristic approaches are better alternative instead of BP. The proposed IMGWO is used to train the ANN to prove its competency in terms of prediction. The proposed IMGWO-ANN is used for medical diagnosis task. Some benchmark medical datasets including heart disease, breast cancer, hepatitis, and parkinson's diseases are used for assessing the performance of IMGWO-ANN. The performance measures are described in terms of mean squared errors (MSEs), classification accuracies, sensitivities, specificities, the area under the curve (AUC), and receiver operating characteristic (ROC) curve. It is found that IMGWO outperforms than three popular metaheuristic approaches including GWO, genetic algorithm (GA), and particle swarm optimization (PSO). Results confirmed the potency of IMGWO as a viable learning technique for an ANN.

A New Hybrid Feature Subset Selection Framework Based on Binary Genetic Algorithm and Information Theory

The explosion of the high-dimensional dataset in the scientific repository has been encouraging interdisciplinary research on data mining, pattern recognition and bioinformatics. The fundamental problem of the individual Feature Selection (FS) method is extracting informative features for classification model and to seek for the malignant disease at low computational cost. In addition, existing FS approaches overlook the fact that for a given cardinality, there can be several subsets with similar information. This paper introduces a novel hybrid FS algorithm, called Filter-Wrapper Feature Selection (FWFS) for a classification problem and also addresses the limitations of existing methods. In the proposed model, the front-end filter ranking method as Conditional Mutual Information Maximization (CMIM) selects the high ranked feature subset while the succeeding method as Binary Genetic Algorithm (BGA) accelerates the search in identifying the significant feature subsets. One of the merits of the proposed method is that, unlike an exhaustive method, it speeds up the FS procedure without lancing of classification accuracy on reduced dataset when a learning model is applied to the selected subsets of features. The efficacy of the proposed (FWFS) method is examined by Naive Bayes (NB) classifier which works as a fitness function. The effectiveness of the selected feature subset is evaluated using numerous classifiers on five biological datasets and five UCI datasets of a varied dimensionality and number of instances. The experimental results emphasize that the proposed method provides additional support to the significant reduction of the features and outperforms the existing methods. For microarray data-sets, we found the lowest classification accuracy is 61.24% on SRBCT dataset and highest accuracy is 99.32% on Diffuse large B-cell lymphoma (DLBCL). In UCI datasets, the lowest classification accuracy is 40.04% on the Lymphography using k-nearest neighbor (k-NN) and highest classification accuracy is 99.05% on the ionosphere using support vector machine (SVM).

Visualisation and Modelling of High-Dimensional Cancerous Gene Expression Dataset

The increase in the number of dimensions of cancerous gene expression dataset causes an increase in complexity, misinterpretation and decrease in the visualisation of the particular dataset for further analysis. Therefore, dimensionality reduction, visualisation and modelling tasks of these dataset become challenging. In this paper, a framework is developed which helps to understand, visualise and model high-dimensional cancerous gene expression dataset into lower dimensions which may be helpful in revealing cancer mechanism and diagnosis. Initially, cancerous gene expression datasets are preprocessed to make them complete, precise and efficient; and principal component analysis is applied for dimensionality reduction and visualisation purpose. The regression is used to model the cancerous gene expression dataset so that type of association (linear or nonlinear) and directions between gene profiles may be estimated. To assess the performance of the developed framework, three different types of cancerous gene expression datasets are taken namely: breast (GEO Acc. No. GDS5076), lung (GEO Acc. No. GDS5040) and prostate (GEO Acc. No. GDS5072) which are publicly available. To validate the results of the regression the cross-validation method is used. The results revealed that a linear approach is to be used for prostate cancer dataset and nonlinear approach for breast and lung cancer datasets in finding an association between gene pairs.

Data-Driven Modeling for UGI Gasification Processes via an Enhanced Genetic BP Neural Network With Link Switches

In this brief, an enhanced genetic back-propagation neural network with link switches (EGA-BPNN-LS) is proposed to address a data-driven modeling problem for gasification processes inside United Gas Improvement (UGI) gasifiers. The online-measured temperature of crude gas produced during the gasification processes plays a dominant role in the syngas industry; however, it is difficult to model temperature dynamics via first principles due to the practical complexity of the gasification process, especially as reflected by severe changes in the gas temperature resulting from infrequent manipulations of the gasifier in practice. The proposed data-driven modeling approach, EGA-BPNN-LS, incorporates an NN-LS, an EGA, and the Levenberg-Marquardt (LM) algorithm. The approach cannot only learn the relationships between the control input and the system output from historical data using an optimized network structure through a combination of EGA and NN-LS but also makes use of the networks gradient information via the LM algorithm. EGA-BPNN-LS is applied to a set of data collected from the field to model the UGI gasification processes, and the effectiveness of EGA-BPNN-LS is verified.

Supervised, Unsupervised, and Semi-Supervised Feature Selection: A Review on Gene Selection

Recently, feature selection and dimensionality reduction have become fundamental tools for many data mining tasks, especially for processing high-dimensional data such as gene expression microarray data. Gene expression microarray data comprises up to hundreds of thousands of features with relatively small sample size. Because learning algorithms usually do not work well with this kind of data, a challenge to reduce the data dimensionality arises. A huge number of gene selection are applied to select a subset of relevant features for model construction and to seek for better cancer classification performance. This paper presents the basic taxonomy of feature selection, and also reviews the state-of-the-art gene selection methods by grouping the literatures into three categories: supervised, unsupervised, and semi-supervised. The comparison of experimental results on top 5 representative gene expression datasets indicates that the classification accuracy of unsupervised and semi-supervised feature selection is competitive with supervised feature selection.

A benchmark for evaluation of algorithms for identification of cellular correlates of clinical outcomes

The Flow Cytometry: Critical Assessment of Population Identification Methods (FlowCAP) challenges were established to compare the performance of computational methods for identifying cell populations in multidimensional flow cytometry data. Here we report the results of FlowCAP-IV where algorithms from seven different research groups predicted the time to progression to AIDS among a cohort of 384 HIV+ subjects, using antigen-stimulated peripheral blood mononuclear cell (PBMC) samples analyzed with a 14-color staining panel. Two approaches (FlowReMi.1 and flowDensity-flowType-RchyOptimyx) provided statistically significant predictive value in the blinded test set. Manual validation of submitted results indicated that unbiased analysis of single cell phenotypes could reveal unexpected cell types that correlated with outcomes of interest in high dimensional flow cytometry datasets.

A variable structure fuzzy neural network model of squamous dysplasia and esophageal squamous cell carcinoma based on a global chaotic optimization algorithm

Identification of squamous dysplasia and esophageal squamous cell carcinoma (ESCC) is of great importance in prevention of cancer incidence. Computer aided algorithms can be very useful for identification of people with higher risks of squamous dysplasia, and ESCC. Such method can limit the clinical screenings to people with higher risks. Different regression methods have been used to predict ESCC and dysplasia. In this paper, a Fuzzy Neural Network (FNN) model is selected for ESCC and dysplasia prediction. The inputs to the classifier are the risk factors. Since the relation between risk factors in the tumor system has a complex nonlinear behavior, in comparison to most of ordinary data, the cost function of its model can have more local optimums. Thus the need for global optimization methods is more highlighted. The proposed method in this paper is a Chaotic Optimization Algorithm (COA) proceeding by the common Error Back Propagation (EBP) local method. Since the model has many parameters, we use a strategy to reduce the dependency among parameters caused by the chaotic series generator. This dependency was not considered in the previous COA methods. The algorithm is compared with logistic regression model as the latest successful methods of ESCC and dysplasia prediction. The results represent a more precise prediction with less mean and variance of error.

A simpler method of preprocessing MALDI-TOF MS data for differential biomarker analysis: stem cell and melanoma cancer studies

INTRODUCTION

Raw spectral data from matrix-assisted laser desorption/ionisation time-of-flight (MALDI-TOF) with MS profiling techniques usually contains complex information not readily providing biological insight into disease. The association of identified features within raw data to a known peptide is extremely difficult. Data preprocessing to remove uncertainty characteristics in the data is normally required before performing any further analysis. This study proposes an alternative yet simple solution to preprocess raw MALDI-TOF-MS data for identification of candidate marker ions. Two in-house MALDI-TOF-MS data sets from two different sample sources (melanoma serum and cord blood plasma) are used in our study.

METHOD

Raw MS spectral profiles were preprocessed using the proposed approach to identify peak regions in the spectra. The preprocessed data was then analysed using bespoke machine learning algorithms for data reduction and ion selection. Using the selected ions, an ANN-based predictive model was constructed to examine the predictive power of these ions for classification.

RESULTS

Our model identified 10 candidate marker ions for both data sets. These ion panels achieved over 90% classification accuracy on blind validation data. Receiver operating characteristics analysis was performed and the area under the curve for melanoma and cord blood classifiers was 0.991 and 0.986, respectively.

CONCLUSION

The results suggest that our data preprocessing technique removes unwanted characteristics of the raw data, while preserving the predictive components of the data. Ion identification analysis can be carried out using MALDI-TOF-MS data with the proposed data preprocessing technique coupled with bespoke algorithms for data reduction and ion selection.