A Self-Adaptive Reinforcement-Exploration Q-Learning Algorithm

Enhancing the Efficiency of a Cybersecurity Operations Center Using Biomimetic Algorithms Empowered by Deep Q-Learning

In the complex and dynamic landscape of cyber threats, organizations require sophisticated strategies for managing Cybersecurity Operations Centers and deploying Security Information and Event Management systems. Our study enhances these strategies by integrating the precision of well-known biomimetic optimization algorithms-namely Particle Swarm Optimization, the Bat Algorithm, the Gray Wolf Optimizer, and the Orca Predator Algorithm-with the adaptability of Deep Q-Learning, a reinforcement learning technique that leverages deep neural networks to teach algorithms optimal actions through trial and error in complex environments. This hybrid methodology targets the efficient allocation and deployment of network intrusion detection sensors while balancing cost-effectiveness with essential network security imperatives. Comprehensive computational tests show that versions enhanced with Deep Q-Learning significantly outperform their native counterparts, especially in complex infrastructures. These results highlight the efficacy of integrating metaheuristics with reinforcement learning to tackle complex optimization challenges, underscoring Deep Q-Learning's potential to boost cybersecurity measures in rapidly evolving threat environments.

Revolutionizing Medicinal Chemistry: The Application of Artificial Intelligence (AI) in Early Drug Discovery

Artificial intelligence (AI) has permeated various sectors, including the pharmaceutical industry and research, where it has been utilized to efficiently identify new chemical entities with desirable properties. The application of AI algorithms to drug discovery presents both remarkable opportunities and challenges. This review article focuses on the transformative role of AI in medicinal chemistry. We delve into the applications of machine learning and deep learning techniques in drug screening and design, discussing their potential to expedite the early drug discovery process. In particular, we provide a comprehensive overview of the use of AI algorithms in predicting protein structures, drug-target interactions, and molecular properties such as drug toxicity. While AI has accelerated the drug discovery process, data quality issues and technological constraints remain challenges. Nonetheless, new relationships and methods have been unveiled, demonstrating AI's expanding potential in predicting and understanding drug interactions and properties. For its full potential to be realized, interdisciplinary collaboration is essential. This review underscores AI's growing influence on the future trajectory of medicinal chemistry and stresses the importance of ongoing synergies between computational and domain experts.

Energy-Saving Optimization Method of Urban Rail Transit Based on Improved Differential Evolution Algorithm

The transformation of railway infrastructure and traction equipment is an ideal way to realize energy savings of urban rail transit trains. However, upgrading railway infrastructure and traction equipment is a high investment and difficult process. To produce energy-savings in the urban rail transit system without changing the existing infrastructure, we propose an energy-saving optimization method by optimizing the traction curve of the train. Firstly, after analyzing the relationship between the idle distance and running energy-savings, an optimization method of traction energy-savings based on the combination of the inertia motion and energy optimization is established by taking the maximum idle distance as the objective; and the maximum allowable running speed, passenger comfort, train timetable, maximum allowable acceleration and kinematics equation as constraints. Secondly, a solution method based on the combination of the adaptive dynamic multimodal differential evolution algorithm and the Q learning algorithm is applied to solve the optimization model of energy-savings. Finally, numeric experiments are conducted to verify the proposed method. Extensive experiments demonstrate the effectiveness of the proposed method. The results show that the method has significant energy-saving properties, saving energy by about 11.2%.

Deep Reinforcement Learning Based Decision Making for Complex Jamming Waveforms

With the development of artificial intelligence, intelligent communication jamming decision making is an important research direction of cognitive electronic warfare. In this paper, we consider a complex intelligent jamming decision scenario in which both communication parties choose to adjust physical layer parameters to avoid jamming in a non-cooperative scenario and the jammer achieves accurate jamming by interacting with the environment. However, when the situation becomes complex and large in number, traditional reinforcement learning suffers from the problems of failure to converge and a high number of interactions, which are fatal and unrealistic in a real warfare environment. To solve this problem, we propose a deep reinforcement learning based and maximum-entropy-based soft actor-critic (SAC) algorithm. In the proposed algorithm, we add an improved Wolpertinger architecture to the original SAC algorithm in order to reduce the number of interactions and improve the accuracy of the algorithm. The results show that the proposed algorithm shows excellent performance in various scenarios of jamming and achieves accurate, fast, and continuous jamming for both sides of the communication.

A Path-Planning Approach Based on Potential and Dynamic Q-Learning for Mobile Robots in Unknown Environment

The path-planning approach plays an important role in determining how long the mobile robots can travel. To solve the path-planning problem of mobile robots in an unknown environment, a potential and dynamic Q-learning (PDQL) approach is proposed, which combines Q-learning with the artificial potential field and dynamic reward function to generate a feasible path. The proposed algorithm has a significant improvement in computing time and convergence speed compared to its classical counterpart. Experiments undertaken on simulated maps confirm that the PDQL when used for the path-planning problem of mobile robots in an unknown environment outperforms the state-of-the-art algorithms with respect to two metrics: path length and turning angle. The simulation results show the effectiveness and practicality of the proposal for mobile robot path planning.

Steelmaking Process Optimised through a Decision Support System Aided by Self-Learning Machine Learning

This paper presents the application of a reinforcement learning (RL) algorithm, concretely Q-Learning, as the core of a decision support system (DSS) for a steelmaking subprocess, the Composition Adjustment by Sealed Argon-bubbling with Oxygen Blowing (CAS-OB) from the SSAB Raahe steel plant. Since many CAS-OB actions are selected based on operator experience, this research aims to develop a DSS to assist the operator in taking the proper decisions during the process, especially less experienced operators. The DSS is intended to supports the operators in real-time during the process to facilitate their work and optimise the process, improving material and energy efficiency, thus increasing the operation’s sustainability. The objective is that the algorithm learns the process based only on raw data from the CAS-OB historical database, and on rewards set according to the objectives. Finally, the DSS was tested and validated by a developer engineer from the CAS-OB steelmaking plant. The results show that the algorithm successfully learns the process, recommending the same actions as those taken by the operator 69.23% of the time. The algorithm also suggests a better option in 30.76% of the remaining cases. Thanks to the DSS, the heat rejection due to wrong composition is reduced by 4%, and temperature accuracy is increased to 83.33%. These improvements resulted in an estimated reduction of 2% in CO2 emissions, 0.5% in energy consumption and 1.5% in costs. Additionally, actions taken based on the operator’s experience are incorporated into the DSS knowledge, facilitating the integration of operators with lower experience in the process.

Indoor Emergency Path Planning Based on the Q-Learning Optimization Algorithm

The internal structure of buildings is becoming increasingly complex. Providing a scientific and reasonable evacuation route for trapped persons in a complex indoor environment is important for reducing casualties and property losses. In emergency and disaster relief environments, indoor path planning has great uncertainty and higher safety requirements. Q-learning is a value-based reinforcement learning algorithm that can complete path planning tasks through autonomous learning without establishing mathematical models and environmental maps. Therefore, we propose an indoor emergency path planning method based on the Q-learning optimization algorithm. First, a grid environment model is established. The discount rate of the exploration factor is used to optimize the Q-learning algorithm, and the exploration factor in the ε-greedy strategy is dynamically adjusted before selecting random actions to accelerate the convergence of the Q-learning algorithm in a large-scale grid environment. An indoor emergency path planning experiment based on the Q-learning optimization algorithm was carried out using simulated data and real indoor environment data. The proposed Q-learning optimization algorithm basically converges after 500 iterative learning rounds, which is nearly 2000 rounds higher than the convergence rate of the Q-learning algorithm. The SASRA algorithm has no obvious convergence trend in 5000 iterations of learning. The results show that the proposed Q-learning optimization algorithm is superior to the SARSA algorithm and the classic Q-learning algorithm in terms of solving time and convergence speed when planning the shortest path in a grid environment. The convergence speed of the proposed Q- learning optimization algorithm is approximately five times faster than that of the classic Q- learning algorithm. The proposed Q-learning optimization algorithm in the grid environment can successfully plan the shortest path to avoid obstacle areas in a short time.