Autonomous deep learning: A genetic DCNN designer for image classification

Efficient adaptive learning rate for convolutional neural network based on quadratic interpolation egret swarm optimization algorithm

Convolutional neural network (CNN) has recently become popular for addressing multi-domain image classification. However, most existing methods frequently suffer from poor performance, especially in performance and convergence for various datasets. Herein, we have proposed an algorithm for multi-domain image classification by introducing a novel adaptive learning rate rule to the conventional CNN. Specifically, we adopt the CNN to extract rich feature representations. Given that the hyperparameters of the learning rate have a positive effect on the prediction error, the Egret Swarm Optimization Algorithm (ESOA) is introduced to update the learning rate, which can jump out of local extrema during exploration. Therefore, combined with quadratic interpolation, the objective function can be approximated by a polynomial, thereby improving its prediction accuracy. To verify the robustness of the proposed algorithm, we conducted comprehensive experiments in five domain public datasets to fulfil the task of image classification. Meanwhile, the highest accuracy rate of 97.15 % was obtained on the test set. The performances of our method on 24 benchmark functions (CEC2017 and CEC2022) are compared with Particle Swarm Optimization (PSO), Genetic Algorithm(GA), Whale Optimization Algorithm(WOA), Catch Fish Optimization Algorithm(CFOA), GOOSE Algorithm(GO) and ESOA. In two benchmark sets, the performance metric values of our algorithm rank no. 1, especially in all unimodal functions in contrast with other baseline algorithms.

Towards secure IoT networks: A comprehensive study of metaheuristic algorithms in conjunction with CNN using a self-generated dataset

The Internet of Things (IoT) has radically reformed various sectors and industries, enabling unprecedented levels of connectivity and automation. However, the surge in the number of IoT devices has also widened the attack surface, rendering IoT networks potentially susceptible to a plethora of security risks. Addressing the critical challenge of enhancing security in IoT networks is of utmost importance. Moreover, there is a considerable lack of datasets designed exclusively for IoT applications. To bridge this gap, a customized dataset that accurately mimics real-world IoT scenarios impacted by four different types of attacks-blackhole, sinkhole, flooding, and version number attacks was generated using the Contiki-OS Cooja Simulator in this study. The resulting dataset is then consequently employed to evaluate the efficacy of several metaheuristic algorithms, in conjunction with Convolutional Neural Network (CNN) for IoT networks. •The proposed study's goal is to identify optimal hyperparameters for CNNs, ensuring their peak performance in intrusion detection tasks.•This study not only intensifies our comprehension of IoT network security but also provides practical guidance for implementation of the robust security measures in real-world IoT applications.

A CNN-CBAM-BIGRU model for protein function prediction

Understanding a protein's function based solely on its amino acid sequence is a crucial but intricate task in bioinformatics. Traditionally, this challenge has proven difficult. However, recent years have witnessed the rise of deep learning as a powerful tool, achieving significant success in protein function prediction. Their strength lies in their ability to automatically learn informative features from protein sequences, which can then be used to predict the protein's function. This study builds upon these advancements by proposing a novel model: CNN-CBAM+BiGRU. It incorporates a Convolutional Block Attention Module (CBAM) alongside BiGRUs. CBAM acts as a spotlight, guiding the CNN to focus on the most informative parts of the protein data, leading to more accurate feature extraction. BiGRUs, a type of Recurrent Neural Network (RNN), excel at capturing long-range dependencies within the protein sequence, which are essential for accurate function prediction. The proposed model integrates the strengths of both CNN-CBAM and BiGRU. This study's findings, validated through experimentation, showcase the effectiveness of this combined approach. For the human dataset, the suggested method outperforms the CNN-BIGRU+ATT model by +1.0 % for cellular components, +1.1 % for molecular functions, and +0.5 % for biological processes. For the yeast dataset, the suggested method outperforms the CNN-BIGRU+ATT model by +2.4 % for the cellular component, +1.2 % for molecular functions, and +0.6 % for biological processes.

Vehicle Detection and Tracking with Roadside LiDAR Using Improved ResNet18 and the Hungarian Algorithm

By the end of the 2020s, full autonomy in autonomous driving may become commercially viable in certain regions. However, achieving Level 5 autonomy requires crucial collaborations between vehicles and infrastructure, necessitating high-speed data processing and low-latency capabilities. This paper introduces a vehicle tracking algorithm based on roadside LiDAR (light detection and ranging) infrastructure to reduce the latency to 100 ms without compromising the detection accuracy. We first develop a vehicle detection architecture based on ResNet18 that can more effectively detect vehicles at a full frame rate by improving the BEV mapping and the loss function of the optimizer. Then, we propose a new three-stage vehicle tracking algorithm. This algorithm enhances the Hungarian algorithm to better match objects detected in consecutive frames, while time-space logicality and trajectory similarity are proposed to address the short-term occlusion problem. Finally, the system is tested on static scenes in the KITTI dataset and the MATLAB/Simulink simulation dataset. The results show that the proposed framework outperforms other methods, with F1-scores of 96.97% and 98.58% for vehicle detection for the KITTI and MATLAB/Simulink datasets, respectively. For vehicle tracking, the MOTA are 88.12% and 90.56%, and the ID-F1 are 95.16% and 96.43%, which are better optimized than the traditional Hungarian algorithm. In particular, it has a significant improvement in calculation speed, which is important for real-time transportation applications.

A genetic programming-based convolutional deep learning algorithm for identifying COVID-19 cases via X-ray images

Evolutionary algorithms have been successfully employed to find the best structure for many learning algorithms including neural networks. Due to their flexibility and promising results, Convolutional Neural Networks (CNNs) have found their application in many image processing applications. The structure of CNNs greatly affects the performance of these algorithms both in terms of accuracy and computational cost, thus, finding the best architecture for these networks is a crucial task before they are employed. In this paper, we develop a genetic programming approach for the optimization of CNN structure in diagnosing COVID-19 cases via X-ray images. A graph representation for CNN architecture is proposed and evolutionary operators including crossover and mutation are specifically designed for the proposed representation. The proposed architecture of CNNs is defined by two sets of parameters, one is the skeleton which determines the arrangement of the convolutional and pooling operators and their connections and one is the numerical parameters of the operators which determine the properties of these operators like filter size and kernel size. The proposed algorithm in this paper optimizes the skeleton and the numerical parameters of the CNN architectures in a co-evolutionary scheme. The proposed algorithm is used to identify covid-19 cases via X-ray images.

A novel approach for optimization of convolution neural network with hybrid particle swarm and grey wolf algorithm for classification of Indian classical dances

Deep learning is the most dominant area to perform the complex challenging tasks such as image classification and recognition. Earlier researchers have been proposed various convolution neural network (CNN) with different architectures to improve the performance accuracy for the classification and recognition of images. However, the fine-tuning of hyper parameters, resulting the optimal network, regularization of parameters is the difficult task. The metaheuristic optimization algorithms are used for solving such kind of problems. In this paper we proffer a fine tune automate CNN with Hybrid Particle Swarm Grey Wolf (HPSGW). This novel algorithm used to discover the optimal parameters of the CNN like batch size, number of hidden layers, number of epochs and size of filters. The proffered optimized architecture is implemented on MNIST, CIFAR are two bench mark datasets and Indian Classical Dance (ICD) for the classification of 8 Indian Classical Dances. The Proffered method improves the model performance accuracy of 97.3% on ICD Dataset, and other benchmark datasets MNIST, CIFAR with an improved accuracy of 99.4% and 91.1%. This auto-tuned network improved the performance by 5.6% for Indian Classical Dance Forms Classification compared to earlier methods and also reduces the computational cost.

Learning Hierarchical Representations with Spike-and-Slab Inception Network

Recently, deep convolutional neural networks (CNN) with inception modules have attracted much attention due to their excellent performances on diverse domains. Nevertheless, the basic CNN can only capture a univariate feature, which is essentially linear. It leads to a weak ability in feature expression, further resulting in insufficient feature mining. In view of this issue, researchers incessantly deepened the network, bringing parameter redundancy and model over-fitting. Hence, whether we can employ this efficient deep neural network architecture to improve CNN and enhance the capacity of image recognition task still remains unknown. In this paper, we introduce spike-and-slab units to the modified inception module, enabling our model to capture dual latent variables and the average and covariance information. This operation further enhances the robustness of our model to variations of image intensity without increasing the model parameters. The results of several tasks demonstrated that dual variable operations can be well-integrated into inception modules, and excellent results have been achieved.

Optimization of Convolutional Neural Networks Architectures Using PSO for Sign Language Recognition

This paper presents an approach to design convolutional neural network architectures, using the particle swarm optimization algorithm. The adjustment of the hyper-parameters and finding the optimal network architecture of convolutional neural networks represents an important challenge. Network performance and achieving efficient learning models for a particular problem depends on setting hyper-parameter values and this implies exploring a huge and complex search space. The use of heuristic-based searches supports these types of problems; therefore, the main contribution of this research work is to apply the PSO algorithm to find the optimal parameters of the convolutional neural networks which include the number of convolutional layers, the filter size used in the convolutional process, the number of convolutional filters, and the batch size. This work describes two optimization approaches; the first, the parameters obtained by PSO are kept under the same conditions in each convolutional layer, and the objective function evaluated by PSO is given by the classification rate; in the second, the PSO generates different parameters per layer, and the objective function is composed of the recognition rate in conjunction with the Akaike information criterion, the latter helps to find the best network performance but with the minimum parameters. The optimized architectures are implemented in three study cases of sign language databases, in which are included the Mexican Sign Language alphabet, the American Sign Language MNIST, and the American Sign Language alphabet. According to the results, the proposed methodologies achieved favorable results with a recognition rate higher than 99%, showing competitive results compared to other state-of-the-art approaches.

Biomedical Image Classification in a Big Data Architecture Using Machine Learning Algorithms

In modern-day medicine, medical imaging has undergone immense advancements and can capture several biomedical images from patients. In the wake of this, to assist medical specialists, these images can be used and trained in an intelligent system in order to aid the determination of the different diseases that can be identified from analyzing these images. Classification plays an important role in this regard; it enhances the grouping of these images into categories of diseases and optimizes the next step of a computer-aided diagnosis system. The concept of classification in machine learning deals with the problem of identifying to which set of categories a new population belongs. When category membership is known, the classification is done on the basis of a training set of data containing observations. The goal of this paper is to perform a survey of classification algorithms for biomedical images. The paper then describes how these algorithms can be applied to a big data architecture by using the Spark framework. This paper further proposes the classification workflow based on the observed optimal algorithms, Support Vector Machine and Deep Learning as drawn from the literature. The algorithm for the feature extraction step during the classification process is presented and can be customized in all other steps of the proposed classification workflow.

Review of deep learning: concepts, CNN architectures, challenges, applications, future directions

In the last few years, the deep learning (DL) computing paradigm has been deemed the Gold Standard in the machine learning (ML) community. Moreover, it has gradually become the most widely used computational approach in the field of ML, thus achieving outstanding results on several complex cognitive tasks, matching or even beating those provided by human performance. One of the benefits of DL is the ability to learn massive amounts of data. The DL field has grown fast in the last few years and it has been extensively used to successfully address a wide range of traditional applications. More importantly, DL has outperformed well-known ML techniques in many domains, e.g., cybersecurity, natural language processing, bioinformatics, robotics and control, and medical information processing, among many others. Despite it has been contributed several works reviewing the State-of-the-Art on DL, all of them only tackled one aspect of the DL, which leads to an overall lack of knowledge about it. Therefore, in this contribution, we propose using a more holistic approach in order to provide a more suitable starting point from which to develop a full understanding of DL. Specifically, this review attempts to provide a more comprehensive survey of the most important aspects of DL and including those enhancements recently added to the field. In particular, this paper outlines the importance of DL, presents the types of DL techniques and networks. It then presents convolutional neural networks (CNNs) which the most utilized DL network type and describes the development of CNNs architectures together with their main features, e.g., starting with the AlexNet network and closing with the High-Resolution network (HR.Net). Finally, we further present the challenges and suggested solutions to help researchers understand the existing research gaps. It is followed by a list of the major DL applications. Computational tools including FPGA, GPU, and CPU are summarized along with a description of their influence on DL. The paper ends with the evolution matrix, benchmark datasets, and summary and conclusion.

Deep Learning for Time Series Forecasting: A Survey

Time series forecasting has become a very intensive field of research, which is even increasing in recent years. Deep neural networks have proved to be powerful and are achieving high accuracy in many application fields. For these reasons, they are one of the most widely used methods of machine learning to solve problems dealing with big data nowadays. In this work, the time series forecasting problem is initially formulated along with its mathematical fundamentals. Then, the most common deep learning architectures that are currently being successfully applied to predict time series are described, highlighting their advantages and limitations. Particular attention is given to feed forward networks, recurrent neural networks (including Elman, long-short term memory, gated recurrent units, and bidirectional networks), and convolutional neural networks. Practical aspects, such as the setting of values for hyper-parameters and the choice of the most suitable frameworks, for the successful application of deep learning to time series are also provided and discussed. Several fruitful research fields in which the architectures analyzed have obtained a good performance are reviewed. As a result, research gaps have been identified in the literature for several domains of application, thus expecting to inspire new and better forms of knowledge.

Multi-objective simulated annealing for hyper-parameter optimization in convolutional neural networks

In this study, we model a CNN hyper-parameter optimization problem as a bi-criteria optimization problem, where the first objective being the classification accuracy and the second objective being the computational complexity which is measured in terms of the number of floating point operations. For this bi-criteria optimization problem, we develop a Multi-Objective Simulated Annealing (MOSA) algorithm for obtaining high-quality solutions in terms of both objectives. CIFAR-10 is selected as the benchmark dataset, and the MOSA trade-off fronts obtained for this dataset are compared to the fronts generated by a single-objective Simulated Annealing (SA) algorithm with respect to several front evaluation metrics such as generational distance, spacing and spread. The comparison results suggest that the MOSA algorithm is able to search the objective space more effectively than the SA method. For each of these methods, some front solutions are selected for longer training in order to see their actual performance on the original test set. Again, the results state that the MOSA performs better than the SA under multi-objective setting. The performance of the MOSA configurations are also compared to other search generated and human designed state-of-the-art architectures. It is shown that the network configurations generated by the MOSA are not dominated by those architectures, and the proposed method can be of great use when the computational complexity is as important as the test accuracy.

Detecting Vulnerabilities in Critical Infrastructures by Classifying Exposed Industrial Control Systems Using Deep Learning

Industrial control systems depend heavily on security and monitoring protocols. Several tools are available for this purpose, which scout vulnerabilities and take screenshots of various control panels for later analysis. However, they do not adequately classify images into specific control groups, which is crucial for security-based tasks performed by manual operators. To solve this problem, we propose a pipeline based on deep learning to classify snapshots of industrial control panels into three categories: internet technologies, operation technologies, and others. More specifically, we compare the use of transfer learning and fine-tuning in convolutional neural networks (CNNs) pre-trained on ImageNet to select the best CNN architecture for classifying the screenshots of industrial control systems. We propose the critical infrastructure dataset (CRINF-300), which is the first publicly available information technology (IT)/operational technology (OT) snapshot dataset, with 337 manually labeled images. We used the CRINF-300 to train and evaluate eighteen different pipelines, registering their performance under CPU and GPU environments. We found out that the Inception-ResNet-V2 and VGG16 architectures obtained the best results on transfer learning and fine-tuning, with F1-scores of 0.9832 and 0.9373, respectively. In systems where time is critical and the GPU is available, we recommend using the MobileNet-V1 architecture, with an average time of 0.03 s to process an image and with an F1-score of 0.9758.

Using Convolutional Neural Networks Based on a Taguchi Method for Face Gender Recognition

In general, a convolutional neural network (CNN) consists of one or more convolutional layers, pooling layers, and fully connected layers. Most designers adopt a trial-and-error method to select CNN parameters. In this study, an AlexNet network with optimized parameters is proposed for face image recognition. A Taguchi method is used for selecting preliminary factors and experiments are performed through orthogonal table design. The proposed method filters out factors that are significantly affected. Finally, experimental results show that the proposed Taguchi-based AlexNet network obtains 87.056% and 98.72% average accuracy of image gender recognition in the CIA and MORPH databases, respectively. In addition, the average accuracy of the proposed Taguchi-based AlexNet network is 1.576% and 3.47% higher than that of the original AlexNet network in CIA and MORPH databases, respectively.

Analysis of the pattern recognition algorithm of broadband satellite modulation signal under deformable convolutional neural networks

This research aims to analyze the effects of different parameter estimation on the recognition performance of satellite modulation signals based on deep learning (DL) under low signal to noise ratio (SNR) or channel non-ideal conditions. In this study, first, the common characteristics of broadband satellite modulation signal and the commonly used signal feature extraction algorithm are introduced. Then, the broadband satellite modulation signal pattern recognition model based on deformable convolutional neural networks (DCNN) is built, and the broadband satellite signal simulation is conducted based on Matlab software. Next, the signal characteristics of binary phase shift keying (BPSK), quadrature phase shift keying (QPSK), 8 phase shift keying (PSK), 16 quadratic amplitude modulation (QAM), 64QAM, and 32 absolute phase shift keying (APSK) are extracted by the constellation map, and the ratio changes of T1 and T2 with SNR are compared. When SNR is given, it is compared with VGG model, AlexNet model, and ResNe model. The results show that the constellation points of satellite signals with different modulations are evenly distributed. T1 of PSK modulation signals increases significantly with the increase of SNR. When SNR is greater than 10, PSK modulation signals can be identified. When T2 is set and SNR is greater than 15dB, 16QAM and 32APSK signals can be distinguished. In the model, the Relu activation function, mini-batch gradient descent (MBGD) algorithm, and Softmax classifier have the best recognition accuracy. PSK modulation signals have the best recognition rate when the SNR is 0dB, and the recognition accuracy of different modulation signals at 20dB is over 98%. When the data length reaches 4000, the recognition accuracy of different modulation signals is higher than 97%. Compared with other algorithms, this algorithm has the highest recognition accuracy (99.83%) and shorter training time (3960s). In conclusion, the broadband satellite modulation signal pattern recognition algorithm of DCNN constructed in this study can effectively identify the patterns of different modulation signals.

Lung Nodule Classification Using Taguchi-Based Convolutional Neural Networks for Computer Tomography Images

Lung cancer occurs in the lungs, trachea, or bronchi. This cancer is often caused by malignant nodules. These cancer cells spread uncontrollably to other organs of the body and pose a threat to life. An accurate assessment of disease severity is critical to determining the optimal treatment approach. In this study, a Taguchi-based convolutional neural network (CNN) was proposed for classifying nodules into malignant or benign. For setting parameters in a CNN, most users adopt trial and error to determine structural parameters. This study used the Taguchi method for selecting preliminary factors. The orthogonal table design is used in the Taguchi method. The final optimal parameter combination was determined, as were the most significant parameters. To verify the proposed method, the lung image database consortium data set from the National Cancer Institute was used for analysis. The database contains a total of 16,471 images, including 11,139 malignant nodule images. The experimental results demonstrated that the proposed method with the optimal parameter combination obtained an accuracy of 99.6%.

A Sustainable Deep Learning Framework for Object Recognition Using Multi-Layers Deep Features Fusion and Selection

With an overwhelming increase in the demand of autonomous systems, especially in the applications related to intelligent robotics and visual surveillance, come stringent accuracy requirements for complex object recognition. A system that maintains its performance against a change in the object’s nature is said to be sustainable and it has become a major area of research for the computer vision research community in the past few years. In this work, we present a sustainable deep learning architecture, which utilizes multi-layer deep features fusion and selection, for accurate object classification. The proposed approach comprises three steps: (1) By utilizing two deep learning architectures, Very Deep Convolutional Networks for Large-Scale Image Recognition and Inception V3, it extracts features based on transfer learning, (2) Fusion of all the extracted feature vectors is performed by means of a parallel maximum covariance approach, and (3) The best features are selected using Multi Logistic Regression controlled Entropy-Variances method. For verification of the robust selected features, the Ensemble Learning method named Subspace Discriminant Analysis is utilized as a fitness function. The experimental process is conducted using four publicly available datasets, including Caltech-101, Birds database, Butterflies database and CIFAR-100, and a ten-fold validation process which yields the best accuracies of 95.5%, 100%, 98%, and 68.80% for the datasets respectively. Based on the detailed statistical analysis and comparison with the existing methods, the proposed selection method gives significantly more accuracy. Moreover, the computational time of the proposed selection method is better for real-time implementation.