HBO-LSTM: Optimized long short term memory with heap-based optimizer for wind power forecasting

Level-set based adaptive-active contour segmentation technique with long short-term memory for diabetic retinopathy classification

Diabetic Retinopathy (DR) is a major type of eye defect that is caused by abnormalities in the blood vessels within the retinal tissue. Early detection by automatic approach using modern methodologies helps prevent consequences like vision loss. So, this research has developed an effective segmentation approach known as Level-set Based Adaptive-active Contour Segmentation (LBACS) to segment the images by improving the boundary conditions and detecting the edges using Level Set Method with Improved Boundary Indicator Function (LSMIBIF) and Adaptive-Active Counter Model (AACM). For evaluating the DR system, the information is collected from the publically available datasets named as Indian Diabetic Retinopathy Image Dataset (IDRiD) and Diabetic Retinopathy Database 1 (DIARETDB 1). Then the collected images are pre-processed using a Gaussian filter, edge detection sharpening, Contrast enhancement, and Luminosity enhancement to eliminate the noises/interferences, and data imbalance that exists in the available dataset. After that, the noise-free data are processed for segmentation by using the Level set-based active contour segmentation technique. Then, the segmented images are given to the feature extraction stage where Gray Level Co-occurrence Matrix (GLCM), Local ternary, and binary patterns are employed to extract the features from the segmented image. Finally, extracted features are given as input to the classification stage where Long Short-Term Memory (LSTM) is utilized to categorize various classes of DR. The result analysis evidently shows that the proposed LBACS-LSTM achieved better results in overall metrics. The accuracy of the proposed LBACS-LSTM for IDRiD and DIARETDB 1 datasets is 99.43% and 97.39%, respectively which is comparably higher than the existing approaches such as Three-dimensional semantic model, Delimiting Segmentation Approach Using Knowledge Learning (DSA-KL), K-Nearest Neighbor (KNN), Computer aided method and Chronological Tunicate Swarm Algorithm with Stacked Auto Encoder (CTSA-SAE).

Deep belief improved bidirectional LSTM for multivariate time series forecasting

Multivariate time series (MTS) play essential roles in daily life because most real-world time series datasets are multivariate and rich in time-dependent information. Traditional forecasting methods for MTS are time-consuming and filled with complicated limitations. One efficient method being explored within the dynamical systems is the extended short-term memory networks (LSTMs). However, existing MTS models only partially use the hidden spatial relationship as effectively as LSTMs. Shallow LSTMs are inadequate in extracting features from high-dimensional MTS; however, the multilayer bidirectional LSTM (BiLSTM) can learn more MTS features in both directions. This study tries to generate a novel and improved BiLSTM network (DBI-BiLSTM) based on a deep belief network (DBN), bidirectional propagation technique, and a chained structure. The deep structures are constructed by a DBN layer and multiple stacked BiLSTM layers, which increase the feature representation of DBI-BiLSTM and allow for the model to further learn the extended features in two directions. First, the input is processed by DBN to obtain comprehensive features. Then, the known features, divided into clusters based on a global sensitivity analysis method, are used as the inputs of every BiLSTM layer. Meanwhile, the previous outputs of the shallow layer are combined with the clustered features to reconstitute new input signals for the next deep layer. Four experimental real-world time series datasets illustrate our one-step-ahead prediction performance. The simulating results confirm that the DBI-BiLSTM not only outperforms the traditional shallow artificial neural networks (ANNs), deep LSTMs, and some recently improved LSTMs, but also learns more features of the MTS data. As compared with conventional LSTM, the percentage improvement of DBI-BiLSTM on the four MTS datasets is 85.41, 75.47, 61.66 and 30.72%, respectively.

Deep Belief Network with Swarm Spider Optimization Method for Renewable Energy Power Forecasting

Renewable energy power prediction plays a crucial role in the development of renewable energy generation, and it also faces a challenging issue because of the uncertainty and complex fluctuation caused by environmental and climatic factors. In recent years, deep learning has been increasingly applied in the time series prediction of new energy, where Deep Belief Networks (DBN) can perform outstandingly for learning of nonlinear features. In this paper, we employed the DBN as the prediction model to forecast wind power and PV power. A novel metaheuristic optimization algorithm, called swarm spider optimization (SSO), was utilized to optimize the parameters of the DBN so as to improve its performance. The SSO is a novel swarm spider behavior based optimization algorithm, and it can be employed for addressing complex optimization and engineering problems. Considering that the prediction performance of the DBN is affected by the number of the nodes in the hidden layer, the SSO is used to optimize this parameter during the training stage of DBN (called SSO-DBN), which can significantly enhance the DBN prediction performance. Two datasets, including wind power and PV power with their influencing factors, were used to evaluate the forecasting performance of the proposed SSO-DBN. We also compared the proposed model with several well-known methods, and the experiment results demonstrate that the proposed prediction model has better stability and higher prediction accuracy in comparison to other methods.

Prediction Model for the Chemical Futures Price Using Improved Genetic Algorithm Based Long Short-Term Memory

In this paper, a new prediction model for accurately recognizing and appropriately evaluating the trends of domestic chemical products and for improving the forecasting accuracy of the chemical products’ prices is proposed. The proposed model uses the minimum forecasting error as the evaluation objective to forecast the settlement price. Active contracts for polyethylene and polypropylene futures on the Dalian Commodity Futures Exchange for the next five days were used, the data were divided into a training set and test set through normalization, and the time window, batch processing size, number of hidden layers, and rejection rate of a long short-term memory (LSTM) network were optimized by an improved genetic algorithm (IGA). In the experiments, with respect to the shortcomings of the genetic algorithm, the crossover location determination and some gene exchange methods in the crossover strategy were improved, and the predicted results of the IGA–LSTM model were compared with those of other models. The results showed that the IGA–LSTM model could effectively capture the characteristics and trends of time-series changes. The results showed that the proposed model obtained the minimum values (MSE = 0.00107, RMSE = 0.03268, and MAPE = 0.0691) in the forecasting of futures prices for two types of chemical products, showing excellent forecasting performance.

Application of Meta-Heuristic Algorithms for Training Neural Networks and Deep Learning Architectures: A Comprehensive Review

The learning process and hyper-parameter optimization of artificial neural networks (ANNs) and deep learning (DL) architectures is considered one of the most challenging machine learning problems. Several past studies have used gradient-based back propagation methods to train DL architectures. However, gradient-based methods have major drawbacks such as stucking at local minimums in multi-objective cost functions, expensive execution time due to calculating gradient information with thousands of iterations and needing the cost functions to be continuous. Since training the ANNs and DLs is an NP-hard optimization problem, their structure and parameters optimization using the meta-heuristic (MH) algorithms has been considerably raised. MH algorithms can accurately formulate the optimal estimation of DL components (such as hyper-parameter, weights, number of layers, number of neurons, learning rate, etc.). This paper provides a comprehensive review of the optimization of ANNs and DLs using MH algorithms. In this paper, we have reviewed the latest developments in the use of MH algorithms in the DL and ANN methods, presented their disadvantages and advantages, and pointed out some research directions to fill the gaps between MHs and DL methods. Moreover, it has been explained that the evolutionary hybrid architecture still has limited applicability in the literature. Also, this paper classifies the latest MH algorithms in the literature to demonstrate their effectiveness in DL and ANN training for various applications. Most researchers tend to extend novel hybrid algorithms by combining MHs to optimize the hyper-parameters of DLs and ANNs. The development of hybrid MHs helps improving algorithms performance and capable of solving complex optimization problems. In general, the optimal performance of the MHs should be able to achieve a suitable trade-off between exploration and exploitation features. Hence, this paper tries to summarize various MH algorithms in terms of the convergence trend, exploration, exploitation, and the ability to avoid local minima. The integration of MH with DLs is expected to accelerate the training process in the coming few years. However, relevant publications in this way are still rare.