A memetic dynamic coral reef optimisation algorithm for simultaneous training, design, and optimisation of artificial neural networks

Artificial Neural Networks (ANNs) have been used in a multitude of real-world applications given their predictive capabilities, and algorithms based on gradient descent, such as Backpropagation (BP) and variants, are usually considered for their optimisation. However, these algorithms have been shown to get stuck at local optima, and they require a cautious design of the architecture of the model. This paper proposes a novel memetic training method for simultaneously learning the ANNs structure and weights based on the Coral Reef Optimisation algorithms (CROs), a global-search metaheuristic based on corals' biology and coral reef formation. Three versions based on the original CRO combined with a Local Search procedure are developed: (1) the basic one, called Memetic CRO; (2) a statistically guided version called Memetic SCRO (M-SCRO) that adjusts the algorithm parameters based on the population fitness; (3) and, finally, an improved Dynamic Statistically-driven version called Memetic Dynamic SCRO (M-DSCRO). M-DSCRO is designed with the idea of improving the M-SCRO version in the evolutionary process, evaluating whether the fitness distribution of the population of ANNs is normal to automatically decide the statistic to be used for assigning the algorithm parameters. Furthermore, all algorithms are adapted to the design of ANNs by means of the most suitable operators. The performance of the different algorithms is evaluated with 40 classification datasets, showing that the proposed M-DSCRO algorithm outperforms the other two versions on most of the datasets. In the final analysis, M-DSCRO is compared against four state-of-the-art methods, demonstrating its superior efficacy in terms of overall accuracy and minority class performance.

Accuracy Versus Simplification in an Approximate Logic Neural Model

An approximate logic neural model (ALNM) is a novel single-neuron model with plastic dendritic morphology. During the training process, the model can eliminate unnecessary synapses and useless branches of dendrites. It will produce a specific dendritic structure for a particular task. The simplified structure of ALNM can be substituted by a logic circuit classifier (LCC) without losing any essential information. The LCC merely consists of the comparator and logic NOT, AND, and OR gates. Thus, it can be easily implemented in hardware. However, the architecture of ALNM affects the learning capacity, generalization capability, computing time and approximation of LCC. Thus, a Pareto-based multiobjective differential evolution (MODE) algorithm is proposed to simultaneously optimize ALNM's topology and weights. MODE can generate a concise and accurate LCC for every specific task from ALNM. To verify the effectiveness of MODE, extensive experiments are performed on eight benchmark classification problems. The statistical results demonstrate that MODE is superior to conventional learning methods, such as the backpropagation algorithm and single-objective evolutionary algorithms. In addition, compared against several commonly used classifiers, both ALNM and LCC are capable of obtaining promising and competitive classification performances on the benchmark problems. Besides, the experimental results also verify that the LCC obtains the faster classification speed than the other classifiers.

A Learning-Based Hybrid Framework for Dynamic Balancing of Exploration-Exploitation: Combining Regression Analysis and Metaheuristics

The idea of hybrid approaches have become a powerful strategy for tackling several complex optimisation problems. In this regard, the present work is concerned with contributing with a novel optimisation framework, named learning-based linear balancer (LB2). A regression model is designed, with the objective to predict better movements for the approach and improve the performance. The main idea is to balance the intensification and diversification performed by the hybrid model in an online-fashion. In this paper, we employ movement operators of a spotted hyena optimiser, a modern algorithm which has proved to yield good results in the literature. In order to test the performance of our hybrid approach, we solve 15 benchmark functions, composed of unimodal, multimodal, and mutimodal functions with fixed dimension. Additionally, regarding the competitiveness, we carry out a comparison against state-of-the-art algorithms, and the sequential parameter optimisation procedure, which is part of multiple successful tuning methods proposed in the literature. Finally, we compare against the traditional implementation of a spotted hyena optimiser and a neural network approach, the respective statistical analysis is carried out. We illustrate experimental results, where we obtain interesting performance and robustness, which allows us to conclude that our hybrid approach is a competitive alternative in the optimisation field.

Ordinal classification of the affectation level of 3D-images in Parkinson diseases

Parkinson’s disease is characterised by a decrease in the density of presynaptic dopamine transporters in the striatum. Frequently, the corresponding diagnosis is performed using a qualitative analysis of the 3D-images obtained after the administration of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{123}$$\end{document}123I-ioflupane, considering a binary classification problem (absence or existence of Parkinson’s disease). In this work, we propose a new methodology for classifying this kind of images in three classes depending on the level of severity of the disease in the image. To tackle this problem, we use an ordinal classifier given the natural order of the class labels. A novel strategy to perform feature selection is developed because of the large number of voxels in the image, and a method for generating synthetic images is proposed to improve the quality of the classifier. The methodology is tested on 434 studies conducted between September 2015 and January 2019, divided into three groups: 271 without alteration of the presynaptic nigrostriatal pathway, 73 with a slight alteration and 90 with severe alteration. Results confirm that the methodology improves the state-of-the-art algorithms, and that it is able to find informative voxels outside the standard regions of interest used for this problem. The differences are assessed by statistical tests which show that the proposed image ordinal classification could be considered as a decision support system in medicine.

Using machine learning methods to determine a typology of patients with HIV-HCV infection to be treated with antivirals

Several European countries have established criteria for prioritising initiation of treatment in patients infected with the hepatitis C virus (HCV) by grouping patients according to clinical characteristics. Based on neural network techniques, our objective was to identify those factors for HIV/HCV co-infected patients (to which clinicians have given careful consideration before treatment uptake) that have not being included among the prioritisation criteria. This study was based on the Spanish HERACLES cohort (NCT02511496) (April-September 2015, 2940 patients) and involved application of different neural network models with different basis functions (product-unit, sigmoid unit and radial basis function neural networks) for automatic classification of patients for treatment. An evolutionary algorithm was used to determine the architecture and estimate the coefficients of the model. This machine learning methodology found that radial basis neural networks provided a very simple model in terms of the number of patient characteristics to be considered by the classifier (in this case, six), returning a good overall classification accuracy of 0.767 and a minimum sensitivity (for the classification of the minority class, untreated patients) of 0.550. Finally, the area under the ROC curve was 0.802, which proved to be exceptional. The parsimony of the model makes it especially attractive, using just eight connections. The independent variable “recent PWID” is compulsory due to its importance. The simplicity of the model means that it is possible to analyse the relationship between patient characteristics and the probability of belonging to the treated group.

Co-Learning Feature Fusion Maps from PET-CT Images of Lung Cancer

The analysis of multi-modality positron emission tomography and computed tomography (PET-CT) images for computer aided diagnosis applications (e.g., detection and segmentation) requires combining the sensitivity of PET to detect abnormal regions with anatomical localization from CT. Current methods for PET-CT image analysis either process the modalities separately or fuse information from each modality based on knowledge about the image analysis task. These methods generally do not consider the spatially varying visual characteristics that encode different information across the different modalities, which have different priorities at different locations. For example, a high abnormal PET uptake in the lungs is more meaningful for tumor detection than physiological PET uptake in the heart. Our aim is to improve fusion of the complementary information in multi-modality PET-CT with a new supervised convolutional neural network (CNN) that learns to fuse complementary information for multi-modality medical image analysis. Our CNN first encodes modality-specific features and then uses them to derive a spatially varying fusion map that quantifies the relative importance of each modality's features across different spatial locations. These fusion maps are then multiplied with the modality-specific feature maps to obtain a representation of the complementary multi-modality information at different locations, which can then be used for image analysis. We evaluated the ability of our CNN to detect and segment multiple regions (lungs, mediastinum, tumors) with different fusion requirements using a dataset of PET-CT images of lung cancer. We compared our method to baseline techniques for multi-modality image fusion (fused inputs (FS), multi-branch (MB) techniques, and multichannel (MC) techniques) and segmentation. Our findings show that our CNN had a significantly higher foreground detection accuracy (99.29%, p < 0:05) than the fusion baselines (FS: 99.00%, MB: 99.08%, TC: 98.92%) and a significantly higher Dice score (63.85%) than recent PET-CT tumor segmentation methods.

Convolutional Neural Networks for Recognition of Lymphoblast Cell Images

This paper presents the recognition for WHO classification of acute lymphoblastic leukaemia (ALL) subtypes. The two ALL subtypes considered are T-lymphoblastic leukaemia (pre-T) and B-lymphoblastic leukaemia (pre-B). They exhibit various characteristics which make it difficult to distinguish between subtypes from their mature cells, lymphocytes. In a common approach, handcrafted features must be well designed for this complex domain-specific problem. With deep learning approach, handcrafted feature engineering can be eliminated because a deep learning method can automate this task through the multilayer architecture of a convolutional neural network (CNN). In this work, we implement a CNN classifier to explore the feasibility of deep learning approach to identify lymphocytes and ALL subtypes, and this approach is benchmarked against a dominant approach of support vector machines (SVMs) applying handcrafted feature engineering. Additionally, two traditional machine learning classifiers, multilayer perceptron (MLP), and random forest are also applied for the comparison. The experiments show that our CNN classifier delivers better performance to identify normal lymphocytes and pre-B cells. This shows a great potential for image classification with no requirement of multiple preprocessing steps from feature engineering.

Convolutional Neural Networks for Recognition of Lymphoblast Cell Images

This paper presents the recognition for WHO classification of acute lymphoblastic leukaemia (ALL) subtypes. The two ALL subtypes considered are T-lymphoblastic leukaemia (pre-T) and B-lymphoblastic leukaemia (pre-B). They exhibit various characteristics which make it difficult to distinguish between subtypes from their mature cells, lymphocytes. In a common approach, handcrafted features must be well designed for this complex domain-specific problem. With deep learning approach, handcrafted feature engineering can be eliminated because a deep learning method can automate this task through the multilayer architecture of a convolutional neural network (CNN). In this work, we implement a CNN classifier to explore the feasibility of deep learning approach to identify lymphocytes and ALL subtypes, and this approach is benchmarked against a dominant approach of support vector machines (SVMs) applying handcrafted feature engineering. Additionally, two traditional machine learning classifiers, multilayer perceptron (MLP), and random forest are also applied for the comparison. The experiments show that our CNN classifier delivers better performance to identify normal lymphocytes and pre-B cells. This shows a great potential for image classification with no requirement of multiple preprocessing steps from feature engineering.

Liver Cancer Classification Model Using Hybrid Feature Selection Based on Class-Dependent Technique for the Central Region of Thailand

Liver cancer data always consist of a large number of multidimensional datasets. A dataset that has huge features and multiple classes may be irrelevant to the pattern classification in machine learning. Hence, feature selection improves the performance of the classification model to achieve maximum classification accuracy. The aims of the present study were to find the best feature subset and to evaluate the classification performance of the predictive model. This paper proposed a hybrid feature selection approach by combining information gain and sequential forward selection based on the class-dependent technique (IGSFS-CD) for the liver cancer classification model. Two different classifiers (decision tree and naïve Bayes) were used to evaluate feature subsets. The liver cancer datasets were obtained from the Cancer Hospital Thailand database. Three ensemble methods (ensemble classifiers, bagging, and AdaBoost) were applied to improve the performance of classification. The IGSFS-CD method provided good accuracy of 78.36% (sensitivity 0.7841 and specificity 0.9159) on LC_dataset-1. In addition, LC_dataset II delivered the best performance with an accuracy of 84.82% (sensitivity 0.8481 and specificity 0.9437). The IGSFS-CD method achieved better classification performance compared to the class-independent method. Furthermore, the best feature subset selection could help reduce the complexity of the predictive model.

Determining the macroinvertebrate community indicators and relevant environmental predictors of the Hun-Tai River Basin (Northeast China): A study based on community patterning

It is essential to understand the patterning of biota and environmental influencing factors for proper rehabilitation and management at the river basin scale. The Hun-Tai River Basin was extensively sampled four times for macroinvertebrate community and environmental variables during one year. Self-Organizing Maps (SOMs) were used to reveal the aggregation patterns of the 355 samples. Three community types (i.e., clusters) were found (at the family level) based on the community composition, which showed a clearly gradient by combining them with the representative environmental variables: minimally impacted source area, intermediately anthropogenic impacted sites, and highly anthropogenic impacted downstream area, respectively. This gradient was corroborated by the decreasing trends in density and diversity of macroinvertebrates. Distance from source, total phosphorus and water temperature were identified as the most important variables that distinguished the delineated communities. In addition, the sampling season, substrate type, pH and the percentage of grassland were also identified as relevant variables. These results demonstrated that macroinvertebrates communities are structured in a hierarchical manner where geographic and water quality prevail over temporal (season) and habitat (substrate type) features at the basin scale. In addition, it implied that the local-scale environment variables affected macroinvertebrates under the longitudinal gradient of the geographical and anthropogenic pressure. More than one family was identified as the indicator for each type of community. Abundance contributed significantly for distinguishing the indicators, while Baetidae with higher density indicated minimally and intermediately impacted area and lower density indicated highly impacted area. Therefore, we suggested the use of abundance data in community patterning and classification, especially in the identification of the indicator taxa.

Validation of artificial neural networks as a methodology for donor-recipient matching for liver transplantation

In 2014, we reported a model for donor-recipient (D-R) matching in liver transplantation (LT) based on artificial neural networks (ANNs) from a Spanish multicenter study (Model for Allocation of Donor and Recipient in España [MADR-E]). The aim is to test the ANN-based methodology in a different European health care system in order to validate it. An ANN model was designed using a cohort of patients from King's College Hospital (KCH; n = 822). The ANN was trained and tested using KCH pairs for both 3- and 12-month survival models. End points were probability of graft survival (correct classification rate [CCR]) and nonsurvival (minimum sensitivity [MS]). The final model is a rule-based system for facilitating the decision about the most appropriate D-R matching. Models designed for KCH had excellent prediction capabilities for both 3 months (CCR-area under the curve [AUC] = 0.94; MS-AUC = 0.94) and 12 months (CCR-AUC = 0.78; MS-AUC = 0.82), almost 15% higher than the best obtained by other known scores such as Model for End-Stage Liver Disease and balance of risk. Moreover, these results improve the previously reported ones in the multicentric MADR-E database. In conclusion, the use of ANN for D-R matching in LT in other health care systems achieved excellent prediction capabilities supporting the validation of these tools. It should be considered as the most advanced, objective, and useful tool to date for the management of waiting lists. Liver Transplantation 24 192-203 2018 AASLD.

iDEAS: A web-based system for dry eye assessment

BACKGROUND AND OBJECTIVES

Dry eye disease is a public health problem, whose multifactorial etiology challenges clinicians and researchers making necessary the collaboration between different experts and centers. The evaluation of the interference patterns observed in the tear film lipid layer is a common clinical test used for dry eye diagnosis. However, it is a time-consuming task with a high degree of intra- as well as inter-observer variability, which makes the use of a computer-based analysis system highly desirable. This work introduces iDEAS (Dry Eye Assessment System), a web-based application to support dry eye diagnosis.

METHODS

iDEAS provides a framework for eye care experts to collaboratively work using image-based services in a distributed environment. It is composed of three main components: the web client for user interaction, the web application server for request processing, and the service module for image analysis. Specifically, this manuscript presents two automatic services: tear film classification, which classifies an image into one interference pattern; and tear film map, which illustrates the distribution of the patterns over the entire tear film.

RESULTS

iDEAS has been evaluated by specialists from different institutions to test its performance. Both services have been evaluated in terms of a set of performance metrics using the annotations of different experts. Note that the processing time of both services has been also measured for efficiency purposes.

CONCLUSIONS

iDEAS is a web-based application which provides a fast, reliable environment for dry eye assessment. The system allows practitioners to share images, clinical information and automatic assessments between remote computers. Additionally, it save time for experts, diminish the inter-expert variability and can be used in both clinical and research settings.

Evaluation of an automatic dry eye test using MCDM methods and rank correlation

Dry eye is an increasingly common disease in modern society which affects a wide range of population and has a negative impact on their daily activities, such as working with computers or driving. It can be diagnosed through an automatic clinical test for tear film lipid layer classification based on color and texture analysis. Up to now, researchers have mainly focused on the improvement of the image analysis step. However, there is still large room for improvement on the machine learning side. This paper presents a methodology to optimize this problem by means of class binarization, feature selection, and classification. The methodology can be used as a baseline in other classification problems to provide several solutions and evaluate their performance using a set of representative metrics and decision-making methods. When several decision-making methods are used, they may offer disagreeing rankings that will be solved by conflict handling in which rankings are merged into a single one. The experimental results prove the effectiveness of the proposed methodology in this domain. Also, its general purpose allows to adapt it to other classification problems in different fields such as medicine and biology.

MLPNN Training via a Multiobjective Optimization of Training Error and Stochastic Sensitivity

The training of a multilayer perceptron neural network (MLPNN) concerns the selection of its architecture and the connection weights via the minimization of both the training error and a penalty term. Different penalty terms have been proposed to control the smoothness of the MLPNN for better generalization capability. However, controlling its smoothness using, for instance, the norm of weights or the Vapnik-Chervonenkis dimension cannot distinguish individual MLPNNs with the same number of free parameters or the same norm. In this paper, to enhance generalization capabilities, we propose a stochastic sensitivity measure (ST-SM) to realize a new penalty term for MLPNN training. The ST-SM determines the expectation of the squared output differences between the training samples and the unseen samples located within their Q -neighborhoods for a given MLPNN. It provides a direct measurement of the MLPNNs output fluctuations, i.e., smoothness. We adopt a two-phase Pareto-based multiobjective training algorithm for minimizing both the training error and the ST-SM as biobjective functions. Experiments on 20 UCI data sets show that the MLPNNs trained by the proposed algorithm yield better accuracies on testing data than several recent and classical MLPNN training methods.

Machine Learning Methods for Binary and Multiclass Classification of Melanoma Thickness From Dermoscopic Images

Thickness of the melanoma is the most important factor associated with survival in patients with melanoma. It is most commonly reported as a measurement of depth given in millimeters (mm) and computed by means of pathological examination after a biopsy of the suspected lesion. In order to avoid the use of an invasive method in the estimation of the thickness of melanoma before surgery, we propose a computational image analysis system from dermoscopic images. The proposed feature extraction is based on the clinical findings that correlate certain characteristics present in dermoscopic images and tumor depth. Two supervised classification schemes are proposed: a binary classification in which melanomas are classified into thin or thick, and a three-class scheme (thin, intermediate, and thick). The performance of several nominal classification methods, including a recent interpretable method combining logistic regression with artificial neural networks (Logistic regression using Initial variables and Product Units, LIPU), is compared. For the three-class problem, a set of ordinal classification methods (considering ordering relation between the three classes) is included. For the binary case, LIPU outperforms all the other methods with an accuracy of 77.6%, while, for the second scheme, although LIPU reports the highest overall accuracy, the ordinal classification methods achieve a better balance between the performances of all classes.

Diversified Sensitivity-Based Undersampling for Imbalance Classification Problems

Undersampling is a widely adopted method to deal with imbalance pattern classification problems. Current methods mainly depend on either random resampling on the majority class or resampling at the decision boundary. Random-based undersampling fails to take into consideration informative samples in the data while resampling at the decision boundary is sensitive to class overlapping. Both techniques ignore the distribution information of the training dataset. In this paper, we propose a diversified sensitivity-based undersampling method. Samples of the majority class are clustered to capture the distribution information and enhance the diversity of the resampling. A stochastic sensitivity measure is applied to select samples from both clusters of the majority class and the minority class. By iteratively clustering and sampling, a balanced set of samples yielding high classifier sensitivity is selected. The proposed method yields a good generalization capability for 14 UCI datasets.

Use of artificial intelligence as an innovative donor-recipient matching model for liver transplantation: results from a multicenter Spanish study

BACKGROUND & AIMS

There is an increasing discrepancy between the number of potential liver graft recipients and the number of organs available. Organ allocation should follow the concept of benefit of survival, avoiding human-innate subjectivity. The aim of this study is to use artificial-neural-networks (ANNs) for donor-recipient (D-R) matching in liver transplantation (LT) and to compare its accuracy with validated scores (MELD, D-MELD, DRI, P-SOFT, SOFT, and BAR) of graft survival.

METHODS

64 donor and recipient variables from a set of 1003 LTs from a multicenter study including 11 Spanish centres were included. For each D-R pair, common statistics (simple and multiple regression models) and ANN formulae for two non-complementary probability-models of 3-month graft-survival and -loss were calculated: a positive-survival (NN-CCR) and a negative-loss (NN-MS) model. The NN models were obtained by using the Neural Net Evolutionary Programming (NNEP) algorithm. Additionally, receiver-operating-curves (ROC) were performed to validate ANNs against other scores.

RESULTS

Optimal results for NN-CCR and NN-MS models were obtained, with the best performance in predicting the probability of graft-survival (90.79%) and -loss (71.42%) for each D-R pair, significantly improving results from multiple regressions. ROC curves for 3-months graft-survival and -loss predictions were significantly more accurate for ANN than for other scores in both NN-CCR (AUROC-ANN=0.80 vs. -MELD=0.50; -D-MELD=0.54; -P-SOFT=0.54; -SOFT=0.55; -BAR=0.67 and -DRI=0.42) and NN-MS (AUROC-ANN=0.82 vs. -MELD=0.41; -D-MELD=0.47; -P-SOFT=0.43; -SOFT=0.57, -BAR=0.61 and -DRI=0.48).

CONCLUSIONS

ANNs may be considered a powerful decision-making technology for this dataset, optimizing the principles of justice, efficiency and equity. This may be a useful tool for predicting the 3-month outcome and a potential research area for future D-R matching models.

Feature selection and classification of leukocytes using random forest

In automatic segmentation of leukocytes from the complex morphological background of tissue section images, a vast number of artifacts/noise are also extracted causing large amount of multivariate data generation. This multivariate data degrades the performance of a classifier to discriminate between leukocytes and artifacts/noise. However, the selection of prominent features plays an important role in reducing the computational complexity and increasing the performance of the classifier as compared to a high-dimensional features space. Therefore, this paper introduces a novel Gini importance-based binary random forest feature selection method. Moreover, the random forest classifier is used to classify the extracted objects into artifacts, mononuclear cells, and polymorphonuclear cells. The experimental results establish that the proposed method effectively eliminates the irrelevant features, maintaining the high classification accuracy as compared to other feature reduction methods.

A STRUCTURAL DISTANCE-BASED CROSSOVER FOR NEURAL NETWORK CLASSIFIERS

This paper presents a structural distance-based crossover for neural network classifiers, which is applied as part of a Memetic Algorithm (MA) for evolving simultaneously the structure and weights of neural network models applied to multiclass problems. Previous researchers have shown that this simultaneous evolution is a way to avoid the noisy fitness evaluation. The MA incorporates a crossover operator that shows to be useful for ameliorating the permutation problem of the network representation (i.e. different genotypes can be used to represent the same neural network phenotype), increasing the structural diversity of the individuals and improving the accuracy of the results. Instead of a recombination probability, the crossover operator considers a similarity parameter (the minimum structural distance), which allows to maintain a trade-off between global and local search. The neural network models selected in this work are the product-unit neural networks (PUNNs), due to their increasing relevance in those classification problems which show a high order relationship between the input variables. The proposed MA is intended to reduce the possible overtraining problems which can raise in some datasets for this kind of models. The evolutionary system is applied to eight classification benchmarks and the results of an analysis of variance contrast (ANOVA) show the effectiveness of the structural-based crossover operator and the capacity of our algorithm to obtain evolved PUNNs with a higher classification accuracy than those obtained using other evolutionary techniques. On the other hand, the results obtained are compared with popular effective machine learning classification methods, resulting in a competitive performance.