Q-Learning-based parameter control in differential evolution for structural optimization

SaMDE: A Self Adaptive Choice of DNDE and SPIDE Algorithms with MRLDE

Differential evolution (DE) is a proficient optimizer and has been broadly implemented in real life applications of various fields. Several mutation based adaptive approaches have been suggested to improve the algorithm efficiency in recent years. In this paper, a novel self-adaptive method called SaMDE has been designed and implemented on the mutation-based modified DE variants such as modified randomized localization-based DE (MRLDE), donor mutation based DE (DNDE), and sequential parabolic interpolation based DE (SPIDE), which were proposed by the authors in previous research. Using the proposed adaptive technique, an appropriate mutation strategy from DNDE and SPIDE can be selected automatically for the MRLDE algorithm. The experimental results on 50 benchmark problems taken of various test suits and a real-world application of minimization of the potential molecular energy problem validate the superiority of SaMDE over other DE variations.

QL-ADIFA: Hybrid optimization using Q-learning and an adaptive logarithmic spiral-levy firefly algorithm

Optimization problems are ubiquitous in engineering and scientific research, with a large number of such problems requiring resolution. Meta-heuristics offer a promising approach to solving optimization problems. The firefly algorithm (FA) is a swarm intelligence meta-heuristic that emulates the flickering patterns and behaviour of fireflies. Although FA has been significantly enhanced to improve its performance, it still exhibits certain deficiencies. To overcome these limitations, this study presents the Q-learning based on the adaptive logarithmic spiral-Levy flight firefly algorithm (QL-ADIFA). The Q-learning technique empowers the improved firefly algorithm to leverage the firefly's environmental awareness and memory while in flight, allowing further refinement of the enhanced firefly. Numerical experiments demonstrate that QL-ADIFA outperforms existing methods on 15 benchmark optimization functions and twelve engineering problems: cantilever arm design, pressure vessel design, three-bar truss design problem, and 9 constrained optimization problems in CEC2020.

Multi-strategy self-learning particle swarm optimization algorithm based on reinforcement learning

The trade-off between exploitation and exploration is a dilemma inherent to particle swarm optimization (PSO) algorithms. Therefore, a growing body of PSO variants is devoted to solving the balance between the two. Among them, the method of self-adaptive multi-strategy selection plays a crucial role in improving the performance of PSO algorithms but has yet to be well exploited. In this research, with the aid of the reinforcement learning technique to guide the generation of offspring, a novel self-adaptive multi-strategy selection mechanism is designed, and then a multi-strategy self-learning PSO algorithm based on reinforcement learning (MPSORL) is proposed. First, the fitness value of particles is regarded as a set of states that are divided into several state subsets non-uniformly. Second, the ε-greedy strategy is employed to select the optimal strategy for each particle. The personal best particle and the global best particle are then updated after executing the strategy. Subsequently, the next state is determined. Thus, the value of the Q-table, as a scheme adopted in self-learning, is reshaped by the reward value, the action and the state in a non-stationary environment. Finally, the proposed algorithm is compared with other state-of-the-art algorithms on two well-known benchmark suites and a real-world problem. Extensive experiments indicate that MPSORL has better performance in terms of accuracy, convergence speed and non-parametric tests in most cases. The multi-strategy selection mechanism presented in the manuscript is effective.

Dynamic sub-route-based self-adaptive beam search Q-learning algorithm for traveling salesman problem

In this paper, a dynamic sub-route-based self-adaptive beam search Q-learning (DSRABSQL) algorithm is proposed that provides a reinforcement learning (RL) framework combined with local search to solve the traveling salesman problem (TSP). DSRABSQL builds upon the Q-learning (QL) algorithm. Considering its problems of slow convergence and low accuracy, four strategies within the QL framework are designed first: the weighting function-based reward matrix, the power function-based initial Q-table, a self-adaptive ε-beam search strategy, and a new Q-value update formula. Then, a self-adaptive beam search Q-learning (ABSQL) algorithm is designed. To solve the problem that the sub-route is not fully optimized in the ABSQL algorithm, a dynamic sub-route optimization strategy is introduced outside the QL framework, and then the DSRABSQL algorithm is designed. Experiments are conducted to compare QL, ABSQL, DSRABSQL, our previously proposed variable neighborhood discrete whale optimization algorithm, and two advanced reinforcement learning algorithms. The experimental results show that DSRABSQL significantly outperforms the other algorithms. In addition, two groups of algorithms are designed based on the QL and DSRABSQL algorithms to test the effectiveness of the five strategies. From the experimental results, it can be found that the dynamic sub-route optimization strategy and self-adaptive ε-beam search strategy contribute the most for small-, medium-, and large-scale instances. At the same time, collaboration exists between the four strategies within the QL framework, which increases with the expansion of the instance scale.

Learning Path Optimization Based on Multi-Attribute Matching and Variable Length Continuous Representation

Personalized learning path considers matching symmetrical attributes from both learner and learning material. The evolutionary algorithm approach usually forms the learning path generation problem into a problem that optimizes the matching degree of the learner and the generated learning path. The proposed work considers the matching of the following symmetrical attributes of learner/material: ability level/difficulty level, learning objective/covered concept, learning style/supported learning styles, and expected learning time/required learning time. The prerequisites of material are considered constraints. A variable-length representation of the learning path is adopted based on floating numbers, which significantly reduces the encoding length and simplifies the learning path generating process. An improved differential evolution algorithm is applied to optimize the matching degree of learning path and learner. The quantitative experiments on different problem scales show that the proposed system outperforms the binary-based representation approaches in scaling ability and outperforms the comparative algorithms in efficiency.

Selective Strategy Differential Evolution for Stochastic Internal Task Scheduling Problem in Cross-Docking Terminals

This study proposed an algorithm called selective strategy differential evolution (SSDE) to handle the complexity of the stochastic internal task scheduling problem in cross-docking terminals. The aims of this study are to assign workers and transfer equipment to internal operations and sequence those operations under randomness and uncertainty with the purpose to minimise total tardiness. The main feature of SSDE is its ability to adapt itself in order to execute the best search strategy. The proposed algorithm was tested on 16 instances using generated data based on real-case scenarios of a pharmaceutical distribution centre. The results showed the significant performance of SSDE to other existing algorithms in terms of solution quality and computational time. The key success factors of SSDE are the use of various search strategies in a single run and the application of suitable termination conditions.

An Enhanced Differential Evolution Algorithm with Bernstein Operator and Refracted Oppositional-Mutual Learning Strategy

Numerical optimization has been a popular research topic within various engineering applications, where differential evolution (DE) is one of the most extensively applied methods. However, it is difficult to choose appropriate control parameters and to avoid falling into local optimum and poor convergence when handling complex numerical optimization problems. To handle these problems, an improved DE (BROMLDE) with the Bernstein operator and refracted oppositional-mutual learning (ROML) is proposed, which can reduce parameter selection, converge faster, and avoid trapping in local optimum. Firstly, a new ROML strategy integrates mutual learning (ML) and refractive oppositional learning (ROL), achieving stochastic switching between ROL and ML during the population initialization and generation jumping period to balance exploration and exploitation. Meanwhile, a dynamic adjustment factor is constructed to improve the ability of the algorithm to jump out of the local optimum. Secondly, a Bernstein operator, which has no parameters setting and intrinsic parameters tuning phase, is introduced to improve convergence performance. Finally, the performance of BROMLDE is evaluated by 10 bound-constrained benchmark functions from CEC 2019 and CEC 2020, respectively. Two engineering optimization problems are utilized simultaneously. The comparative experimental results show that BROMLDE has higher global optimization capability and convergence speed on most functions and engineering problems.

A Novel Multi-Factor Three-Step Feature Selection and Deep Learning Framework for Regional GDP Prediction: Evidence from China

Gross domestic product (GDP) is an important index reflecting the economic development of a region. Accurate GDP prediction of developing regions can provide technical support for sustainable urban development and economic policy formulation. In this paper, a novel multi-factor three-step feature selection and deep learning framework are proposed for regional GDP prediction. The core modeling process is mainly composed of the following three steps: In Step I, the feature crossing algorithm is used to deeply excavate hidden feature information of original datasets and fully extract key information. In Step II, BorutaRF and Q-learning algorithms analyze the deep correlation between extracted features and targets from two different perspectives and determine the features with the highest quality. In Step III, selected features are used as the input of TCN (Temporal convolutional network) to build a GDP prediction model and obtain final prediction results. Based on the experimental analysis of three datasets, the following conclusions can be drawn: (1) The proposed three-stage feature selection method effectively improves the prediction accuracy of TCN by more than 10%. (2) The proposed GDP prediction framework proposed in the paper has achieved better forecasting performance than 14 benchmark models. In addition, the MAPE values of the models are lower than 5% in all cases.