Deep neural networks with Elastic Rectified Linear Units for object recognition

Activation Functions for Convolutional Neural Networks: Proposals and Experimental Study

Activation functions lie at the core of every neural network model from shallow to deep convolutional neural networks. Their properties and characteristics shape the output range of each layer and, thus, their capabilities. Modern approaches rely mostly on a single function choice for the whole network, usually ReLU or other similar alternatives. In this work, we propose two new activation functions and analyze their properties and compare them with 17 different function proposals from recent literature on six distinct problems with different characteristics. The objective is to shed some light on their comparative performance. The results show that the proposed functions achieved better performance than the most commonly used ones.

ResNet-SVM: Fusion based glioblastoma tumor segmentation and classification

Computerized segmentation of brain tumor based on magnetic resonance imaging (MRI) data presents an important challenging act in computer vision. In image segmentation, numerous studies have explored the feasibility and advantages of employing deep neural network methods to automatically detect and segment brain tumors depicting on MRI. For training the deeper neural network, the procedure usually requires extensive computational power and it is also very time-consuming due to the complexity and the gradient diffusion difficulty. In order to address and help solve this challenge, we in this study present an automatic approach for Glioblastoma brain tumor segmentation based on deep Residual Learning Network (ResNet) to get over the gradient problem of deep Convolutional Neural Networks (CNNs). Using the extra layers added to a deep neural network, ResNet algorithm can effectively improve the accuracy and the performance, which is useful in solving complex problems with a much rapid training process. An additional method is then proposed to fully automatically classify different brain tumor categories (necrosis, edema, and enhancing regions). Results confirm that the proposed fusion method (ResNet-SVM) has an increased classification results of accuracy (AC = 89.36%), specificity (SP = 92.52%) and precision (PR = 90.12%) using 260 MRI data for the training and 112 data used for testing and validation of Glioblastoma tumor cases. Compared to the state-of-the art methods, the proposed scheme provides a higher performance by identifying Glioblastoma tumor type.

Learning and visualizing chronic latent representations using electronic health records

Background

Nowadays, patients with chronic diseases such as diabetes and hypertension have reached alarming numbers worldwide. These diseases increase the risk of developing acute complications and involve a substantial economic burden and demand for health resources. The widespread adoption of Electronic Health Records (EHRs) is opening great opportunities for supporting decision-making. Nevertheless, data extracted from EHRs are complex (heterogeneous, high-dimensional and usually noisy), hampering the knowledge extraction with conventional approaches.

Methods

We propose the use of the Denoising Autoencoder (DAE), a Machine Learning (ML) technique allowing to transform high-dimensional data into latent representations (LRs), thus addressing the main challenges with clinical data. We explore in this work how the combination of LRs with a visualization method can be used to map the patient data in a two-dimensional space, gaining knowledge about the distribution of patients with different chronic conditions. Furthermore, this representation can be also used to characterize the patient’s health status evolution, which is of paramount importance in the clinical setting.

Results

To obtain clinical LRs, we considered real-world data extracted from EHRs linked to the University Hospital of Fuenlabrada in Spain. Experimental results showed the great potential of DAEs to identify patients with clinical patterns linked to hypertension, diabetes and multimorbidity. The procedure allowed us to find patients with the same main chronic disease but different clinical characteristics. Thus, we identified two kinds of diabetic patients with differences in their drug therapy (insulin and non-insulin dependant), and also a group of women affected by hypertension and gestational diabetes. We also present a proof of concept for mapping the health status evolution of synthetic patients when considering the most significant diagnoses and drugs associated with chronic patients.

Conclusion

Our results highlighted the value of ML techniques to extract clinical knowledge, supporting the identification of patients with certain chronic conditions. Furthermore, the patient’s health status progression on the two-dimensional space might be used as a tool for clinicians aiming to characterize health conditions and identify their more relevant clinical codes.

Supplementary Information

The online version contains supplementary material available at (10.1186/s13040-022-00303-z).

3D Deep Attentive U-Net with Transformer for Breast Tumor Segmentation from Automated Breast Volume Scanner

Breast cancer has become the primary factor threatening women's health. Automated breast volume scanner (ABVS) is applied for automatic scanning which is meaningful for the rapid and accurate detection of breast tumor. However, accurate segmentation of tumor regions is a huge challenge for clinicians from the ABVS images since it has the large image size and low data quality. Therefore, we propose a novel 3D deep convolutional neural network for automatic breast tumor segmentation from ABVS data. The structure based on 3D U-Net is designed with attention mechanism and transformer layers to optimize the extracted image features. In addition, we integrate the atrous spatial pyramid pooling block and the deep supervision for further performance improvement. The experimental results demonstrate that our model has achieved dice coefficient of 76.36% for 3D segmentation of breast tumor via our self-collected data.

Deep learning in electron microscopy

Deep learning is transforming most areas of science and technology, including electron microscopy. This review paper offers a practical perspective aimed at developers with limited familiarity. For context, we review popular applications of deep learning in electron microscopy. Following, we discuss hardware and software needed to get started with deep learning and interface with electron microscopes. We then review neural network components, popular architectures, and their optimization. Finally, we discuss future directions of deep learning in electron microscopy.

Ordinal classification of the affectation level of 3D-images in Parkinson diseases

Parkinson’s disease is characterised by a decrease in the density of presynaptic dopamine transporters in the striatum. Frequently, the corresponding diagnosis is performed using a qualitative analysis of the 3D-images obtained after the administration of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{123}$$\end{document}123I-ioflupane, considering a binary classification problem (absence or existence of Parkinson’s disease). In this work, we propose a new methodology for classifying this kind of images in three classes depending on the level of severity of the disease in the image. To tackle this problem, we use an ordinal classifier given the natural order of the class labels. A novel strategy to perform feature selection is developed because of the large number of voxels in the image, and a method for generating synthetic images is proposed to improve the quality of the classifier. The methodology is tested on 434 studies conducted between September 2015 and January 2019, divided into three groups: 271 without alteration of the presynaptic nigrostriatal pathway, 73 with a slight alteration and 90 with severe alteration. Results confirm that the methodology improves the state-of-the-art algorithms, and that it is able to find informative voxels outside the standard regions of interest used for this problem. The differences are assessed by statistical tests which show that the proposed image ordinal classification could be considered as a decision support system in medicine.

Convolutional Neural Network for Dust and Hotspot Classification in PV Modules

This paper proposes an innovative approach to classify the losses related to photovoltaic (PV) systems, through the use of thermographic non-destructive tests (TNDTs) supported by artificial intelligence techniques. Low electricity production in PV systems can be caused by an efficiency decrease in PV modules due to abnormal operating conditions such as failures or malfunctions. The most common performance decreases are due to the presence of dirt on the surface of the module, the impact of which depends on many parameters and conditions, and can be identified through the use of the TNDTs. The proposed approach allows one to automatically classify the thermographic images from the convolutional neural network (CNN) of the system, achieving an accuracy of 98% in tests that last a couple of minutes. This approach, compared to approaches in literature, offers numerous advantages, including speed of execution, speed of diagnosis, reduced costs, reduction in electricity production losses.

Monostability and Multistability for Almost-Periodic Solutions of Fractional-Order Neural Networks With Unsaturating Piecewise Linear Activation Functions

Since the unsaturating activation function is unbounded, more complex dynamics may exist in neural networks with this kind of activation function. In this article, monostability and multistability results of almost-periodic solutions are developed for fractional-order neural networks with unsaturating piecewise linear activation functions. Some globally Mittag-Leffler attractive sets are given, and the existence of globally Mittag-Leffler stable almost-periodic solution is demonstrated by using Ascoli-Arzela theorem. In particular, some sufficient conditions are provided to ascertain the multistability of almost-periodic solutions based on locally positively invariant set. It shows that there exists an almost-periodic solution in each positively invariant set, and all trajectories converge to this periodic trajectory in that rectangular area. Two illustrative examples are provided to demonstrate the effectiveness of the proposed sufficient criteria.

Particle swarm-based and neuro-based FOPID controllers for a Twin Rotor System with improved tracking performance and energy reduction

This paper examines two approaches in tuning fractional order proportional-integral-differential (FOPID) control named as neuro-based FOPID (NNFOPID) and particle swarm-based FOPID (PSOFOPID) for pitch control of a Twin Rotor Aerodynamic System (TRAS). For the neuro-based FOPID control, the innovations are the modification of output equation in the artificial neural network and the implementation of the Rectified Linear Unit (ReLU) activation function. The advantages of the proposed approach are a lighter network and the ability to tune more practical controller parameters without a deep knowledge of the system to achieve a satisfying pitch tracking response. As for the particle swarm-based FOPID control, the application of PSO with spreading factor algorithm is extended for tuning the FOPID controller gains and the innovation here is a new procedure in setting the initial search range. The important advantages of this proposed swarm-based algorithm are the avoidance of being trapped in local optima and reduction of the search area respectively. The performances of the proposed controllers are proven by extensive simulations and experimental verifications based on five standard criteria: square-wave characteristics, reference to disturbance ratio, evaluation time, energy consumption of the control signal and tracking performance. The performances of the proposed controllers are compared against an optimised PID control in three system conditions, namely Case I) without coupling effect and wind disturbance, Case II) with coupling effect only and Case III) with wind disturbance only. Together, this study finds that NNFOPID control offers an accurate system positioning by a 34% reduction in steady-state error with the lowest energy consumption and minimum evaluation time in Case II. In terms of the tracking performance and robustness for Case II, the superiority of PSOFOPID control is confirmed by a 27% reduction in the tracking error and the lowest oscillation value. The experimental results also validate the robustness and energy consumption of both controllers in Case III. It is envisaged that the proposed control designs can be very useful in tuning FOPID controller gains for high performance, low energy, and robust aerodynamics systems.