Reactive Power Optimization for Transient Voltage Stability in Energy Internet via Deep Reinforcement Learning Approach

A Review: Machine Learning for Combinatorial Optimization Problems in Energy Areas

Combinatorial optimization problems (COPs) are a class of NP-hard problems with great practical significance. Traditional approaches for COPs suffer from high computational time and reliance on expert knowledge, and machine learning (ML) methods, as powerful tools have been used to overcome these problems. In this review, the COPs in energy areas with a series of modern ML approaches, i.e., the interdisciplinary areas of COPs, ML and energy areas, are mainly investigated. Recent works on solving COPs using ML are sorted out firstly by methods which include supervised learning (SL), deep learning (DL), reinforcement learning (RL) and recently proposed game theoretic methods, and then problems where the timeline of the improvements for some fundamental COPs is the layout. Practical applications of ML methods in the energy areas, including the petroleum supply chain, steel-making, electric power system and wind power, are summarized for the first time, and challenges in this field are analyzed.

Advances in the Application of Machine Learning Techniques for Power System Analytics: A Survey

The recent advances in computing technologies and the increasing availability of large amounts of data in smart grids and smart cities are generating new research opportunities in the application of Machine Learning (ML) for improving the observability and efficiency of modern power grids. However, as the number and diversity of ML techniques increase, questions arise about their performance and applicability, and on the most suitable ML method depending on the specific application. Trying to answer these questions, this manuscript presents a systematic review of the state-of-the-art studies implementing ML techniques in the context of power systems, with a specific focus on the analysis of power flows, power quality, photovoltaic systems, intelligent transportation, and load forecasting. The survey investigates, for each of the selected topics, the most recent and promising ML techniques proposed by the literature, by highlighting their main characteristics and relevant results. The review revealed that, when compared to traditional approaches, ML algorithms can handle massive quantities of data with high dimensionality, by allowing the identification of hidden characteristics of (even) complex systems. In particular, even though very different techniques can be used for each application, hybrid models generally show better performances when compared to single ML-based models.

Flexible Transmission Network Expansion Planning Based on DQN Algorithm

Compared with static transmission network expansion planning (TNEP), multi-stage TNEP is more in line with the actual situation, but the modeling is also more complicated. This paper proposes a new multi-stage TNEP method based on the deep Q-network (DQN) algorithm, which can solve the multi-stage TNEP problem based on a static TNEP model. The main purpose of this research is to provide grid planners with a simple and effective multi-stage TNEP method, which is able to flexibly adjust the network expansion scheme without replanning. The proposed method takes into account the construction sequence of lines in the planning and completes the adaptive planning of lines by utilizing the interactive learning characteristics of the DQN algorithm. In order to speed up the learning efficiency of the algorithm and enable the agent to have a better judgment on the reward of the line-building action, the prioritized experience replay (PER) strategy is added to the DQN algorithm. In addition, the economy, reliability, and flexibility of the expansion scheme are considered in order to evaluate the scheme more comprehensively. The fault severity of equipment is considered on the basis of the Monte Carlo method to obtain a more comprehensive system state simulation. Finally, extensive studies are conducted with IEEE 24-bus reliability test system, and the computational results demonstrate the effectiveness and adaptability of the proposed flexible TNEP method.

Penalty Electricity Price-Based Optimal Control for Distribution Networks

With the integration of large-scale renewable energy and the implementation of demand response, the complexity and volatility of distribution network operations are increasing. This has led to the inconsistency between the actual net power consumption of power users and their optimal dispatching orders. As a result, the distribution networks cannot operate according to their optimization strategy. The study proposed a penalty electricity price mechanism and the optimal control method based on this electricity price mechanism for distribution networks. First, we established the structure of the distribution network optimal control system. Second, aiming at the actual net power consumption (including power generation and consumption) of power users tracking their dispatching orders, we established a penalty electricity price mechanism. Third, we designed an optimal control strategy and process of distribution networks based on the penalty electricity price. Finally, we verified the proposed method by taking the IEEE-33 node system as an example. The verification results showed that the penalty electricity price could effectively limit the net power consumption fluctuations of power users to achieve optimal control of distribution networks.

Multiobjective Reactive Power Optimization of Renewable Energy Power Plants Based on Time-and-Space Grouping Method

The large-scale renewable energy power plants connected to a weak grid may cause bus voltage fluctuations in the renewable energy power plant and even power grid. Therefore, reactive power compensation is demanded to stabilize the bus voltage and reduce network loss. For this purpose, time-series characteristics of renewable energy power plants are firstly reflected using K-means++ clustering method. The time group behaviors of renewable energy power plants, spatial behaviors of renewable energy generation units, and a time-and-space grouping model of renewable energy power plants are thus established. Then, a mixed-integer optimization method for reactive power compensation in renewable energy power plants is developed based on the second-order cone programming (SOCP). Accordingly, power flow constraints can be simplified to achieve reactive power optimization more efficiently and quickly. Finally, the feasibility and economy for the proposed method are verified by actual renewable energy power plants.

Voltage and Reactive Power Optimization Using a Simplified Linear Equations at Distribution Networks with DG

In this paper, the VVO (Volt/Var optimization) is proposed using simplified linear equations. For fast computation, the characteristics of voltage control devices in a distribution system are expressed as a simplified linear equation. The voltage control devices are classified according to the characteristics of voltage control and represented as the simplified linear equation. The estimated voltage of distribution networks is represented by the sum of the simplified linear equations for the voltage control devices using the superposition principle. The voltage variation by the reactive power of distributed generations (DGs) can be expressed as the matrix of reactance. The voltage variation of tap changing devices can be linearized into the control area factor. The voltage variation by capacitor banks can also be expressed as the matrix of reactance. The voltage equations expressed as simplified linear equations are formulated by quadratic programming (QP). The variables of voltage control devices are defined, and the objective function is formulated as the QP form. The constraints are set using operating voltage range of distribution networks and the control ranges of each voltage control device. In order to derive the optimal solution, mixed-integer quadratic programming (MIQP), which is a type of mixed-integer nonlinear programming (MINLP), is used. The optimal results and proposed method results are compared by using MATLAB simulation and are confirmed to be close to the optimal solution.

Reactive Power Optimization of a Distribution System Based on Scene Matching and Deep Belief Network

With a large number of distributed generators (DGs) and electrical vehicles (EVs) integrated into the power distribution system, the complexity of distribution system operation is increased, which arises to higher requirements for online reactive power optimization. This paper proposes two methods for online reactive power optimization, a scene-matching method based on Random Matrix (RM) features and a deep learning method based on Deep Belief Network (DBN). Firstly, utilizing the operation and ambient Big Data (BD) of the distribution system, we construct the high-dimension Random Matrices and extract 57 state features for the subsequent scene-matching and DBN training. Secondly, the feature-based scene-matching method is proposed. Furtherly, to effectively deal with the uncertainty of DGs and to avoid the performance deterioration of the scene-matching method under a new unknown scene, the DBN-based model is constructed and trained, with the former features as the inputs and the conventional reactive power control solutions as the outputs. This DBN model can learn the nonlinear complicated relationship between the system features and the reactive power control solutions. Finally, the comprehensive case studies have been conducted on the modified IEEE-37 nodes active distribution system, and the performances of the proposed two methods are compared with the conventional method. The results show that the DBN-based method possesses the better performance than the others, and it can reduce the network losses and node voltage deviations obviously, even under the new unknown and unmatched scenes. It does not depend on the distribution system model and parameters anymore and can provide online decision-making more quickly. The discussions of the two methods under different DG penetrations and the historical data volume were given, verifying the adaptability, robustness and generalization ability of the DBN-based method.