Optimal Reactive Power Dispatch in ADNs using DRL and the Impact of Its Various Settings and Environmental Changes

Modern active distribution networks (ADNs) witness increasing complexities that require efforts in control practices, including optimal reactive power dispatch (ORPD). Deep reinforcement learning (DRL) is proposed to manage the network's reactive power by coordinating different resources, including distributed energy resources, to enhance performance. However, there is a lack of studies examining DRL elements' performance sensitivity. To this end, in this paper we examine the impact of various DRL reward representations and hyperparameters on the agent's learning performance when solving the ORPD problem for ADNs. We assess the agent's performance regarding accuracy and training time metrics, as well as critic estimate measures. Furthermore, different environmental changes are examined to study the DRL model's scalability by including other resources. Results show that compared to other representations, the complementary reward function exhibits improved performance in terms of power loss minimization and convergence time by 10-15% and 14-18%, respectively. Also, adequate agent performance is observed to be neighboring the best-suited value of each hyperparameter for the studied problem. In addition, scalability analysis depicts that increasing the number of possible action combinations in the action space by approximately nine times results in 1.7 times increase in the training time.

A novel MADM algorithm for physical education teaching quality evaluation based on 2-tuple linguistic neutrosophic numbers power heronian mean operators

Classroom teaching quality evaluation is an important link in the curriculum quality assurance system. It has important guiding significance for the timely feedback of classroom teaching effects, the achievement of teachers' teaching goals, and the implementation of teaching plans. The evaluation system is scientific, objective and accurate. The classroom teaching quality evaluation is an important way to improve the level of teacher education and teaching and then determine the quality of talent training in various majors. At present, although the evaluation work has played a positive role, the backwardness of the evaluation system has seriously restricted the effectiveness of teaching feedback. The classroom teaching quality evaluation of college basketball training is viewed as the multi-attribute decision-making (MADM). In this article, we combine the generalized Heronian mean (GHM) operator and power average (PA) with 2-tuple linguistic neutrosophic sets (2TLNSs) to propose the generalized 2-tuple linguistic neutrosophic power HM (G2TLNPHM) operator. The G2TLNPHM operator is built for MADM. Finally, an example for classroom teaching quality evaluation of college basketball training is used to show the proposed methods.

A Reinforcement Learning Approach Based on Automatic Policy Amendment for Multi-AUV Task Allocation in Ocean Current

In this paper, the multiple autonomous underwater vehicles (AUVs) task allocation (TA) problem in ocean current environment based on a novel reinforcement learning approach is studied. First, the ocean current environment including direction and intensity is established and a reward function is designed, in which the AUVs are required to consider the ocean current, the task emergency and the energy constraints to find the optimal TA strategy. Then, an automatic policy amendment algorithm (APAA) is proposed to solve the drawback of slow convergence in reinforcement learning (RL). In APAA, the task sequences with higher team cumulative reward (TCR) are recorded to construct task sequence matrix (TSM). After that, the TCR, the subtask reward (SR) and the entropy are used to evaluate TSM to generate amendment probability, which adjusts the action distribution to increase the chances of choosing those more valuable actions. Finally, the simulation results are provided to verify the effectiveness of the proposed approach. The convergence performance of APAA is also better than DDQN, PER and PPO-Clip.

A Hierarchical Decision-Making Method with a Fuzzy Ant Colony Algorithm for Mission Planning of Multiple UAVs

Unmanned aerial vehicles (UAVs) received an unprecedented surge of people’s interest worldwide in recent years. This paper investigates the specific problem of cooperative mission planning for multiple UAVs on the battlefield from a hierarchical decision-making perspective. From the view of the actual mission planning issue, the two key problems to be solved in UAV collaborative mission planning are mission allocation and route planning. In this paper, both of these problems are taken into account via a hierarchical decision-making model. Firstly, we use a target clustering algorithm to divide the original targets into target subgroups, where each target subgroup contains multiple targets. Secondly, a fuzzy ant colony algorithm is used to calculate the global path between target subgroups for a single-target group. Thirdly, a fuzzy ant colony algorithm is also used to calculate the local path between multiple targets for a single-target subgroup. After three levels of decision-making, the complete path for multiple UAVs can be obtained. In order to improve the efficiency of a collaborative task between different types of UAVs, a cooperative communication strategy is developed, which can reduce the number of UAVs performing tasks. Finally, experimental results demonstrate the effectiveness of the proposed cooperative mission planning and cooperative communication strategy for multiple UAVs.

A Situation Assessment Method with an Improved Fuzzy Deep Neural Network for Multiple UAVs

To improve the intelligence and accuracy of the Situation Assessment (SA) in complex scenes, this work develops an improved fuzzy deep neural network approach to the situation assessment for multiple Unmanned Aerial Vehicle(UAV)s. Firstly, this work normalizes the scene data based on time series and use the normalized data as the input for an improved fuzzy deep neural network. Secondly, adaptive momentum and Elastic SGD (Elastic Stochastic Gradient Descent) are introduced into the training process of the neural network, to improve the learning performance. Lastly, in the real-time situation assessment task for multiple UAVs, conventional methods often bring inaccurate results for the situation assessment because these methods don’t consider the fuzziness of task situations. This work uses an improved fuzzy deep neural network to calculate the results of situation assessment and normalizes these results. Then, the degree of trust of the current result, relative to each situation label, is calculated with the normalized results using fuzzy logic. Simulation results show that the proposed method outperforms competitors.

Fuzzy Reinforcement Learning and Curriculum Transfer Learning for Micromanagement in Multi-Robot Confrontation

Multi-Robot Confrontation on physics-based simulators is a complex and time-consuming task, but simulators are required to evaluate the performance of the advanced algorithms. Recently, a few advanced algorithms have been able to produce considerably complex levels in the context of the robot confrontation system when the agents are facing multiple opponents. Meanwhile, the current confrontation decision-making system suffers from difficulties in optimization and generalization. In this paper, a fuzzy reinforcement learning (RL) and the curriculum transfer learning are applied to the micromanagement for robot confrontation system. Firstly, an improved Qlearning in the semi-Markov decision-making process is designed to train the agent and an efficient RL model is defined to avoid the curse of dimensionality. Secondly, a multi-agent RL algorithm with parameter sharing is proposed to train the agents. We use a neural network with adaptive momentum acceleration as a function approximator to estimate the state-action function. Then, a method of fuzzy logic is used to regulate the learning rate of RL. Thirdly, a curriculum transfer learning method is used to extend the RL model to more difficult scenarios, which ensures the generalization of the decision-making system. The experimental results show that the proposed method is effective.