CN-121978891-A - Excitation control method based on dynamic event triggering and deep reinforcement learning

CN121978891ACN 121978891 ACN121978891 ACN 121978891ACN-121978891-A

Abstract

An excitation control method based on dynamic event triggering and deep reinforcement learning belongs to the technical field of power system automation and control. The method comprises the steps of establishing a synchronous generator and excitation system simulation mathematical model for controller design and training, constructing a deep reinforcement learning training environment, adopting a deep deterministic strategy gradient algorithm to train a deep reinforcement learning controller in an off-line mode, designing a dynamic event triggering mechanism, solidifying actor network parameters of the trained deep reinforcement learning controller, and deploying the actor network parameters and the dynamic event triggering mechanism to a control unit together for online simulation test and performance evaluation. The method has the advantages of safety, reliability, excellent performance, high efficiency of resources, strong self-adaption, and capabilities of obviously reducing the calculation and communication resource consumption of the controller and improving the intelligent level and operation economy of the excitation system in the face of complex disturbance of a modern power grid on the premise of ensuring the stability and control quality of the system.

Inventors

MA QINGGUI
SONG JIANBIN
WANG JINGKUN
XU JIAN
ZONG FEIFEI
HUANG JIANLI
ZHANG XIAOQIANG
Tao Fuguan

Assignees

华润电力登封有限公司

Dates

Publication Date: 20260505
Application Date: 20260204

Claims (5)

1. The excitation control method based on dynamic event triggering and deep reinforcement learning is characterized by comprising the following steps of: step one, establishing a synchronous generator and excitation system simulation mathematical model for controller design and training, and enabling the model to accurately reflect dynamic characteristics of a system under various disturbance, wherein a state space equation of the synchronous generator and the excitation system is used for describing dynamic behaviors of the system; Step two, based on the simulation mathematical model established in the step one, constructing a deep reinforcement learning training environment, and adopting a deep deterministic strategy gradient algorithm to train a deep reinforcement learning controller offline, wherein an actor network of the controller can map out an approximately optimal excitation control auxiliary signal according to an input system state vector after training; Step three, designing a dynamic event triggering mechanism, wherein the triggering condition of the mechanism is determined by the error norm of the current state and the state at the last triggering moment of the system, the change rate norm of the error and a dynamic positive threshold value, and the dynamic threshold value is related to the physical quantity reflecting the disturbance intensity of the system and can be changed reversely along with the physical quantity; And step four, solidifying the actor network parameters of the deep reinforcement learning controller trained in the step two, and deploying the actor network parameters to an excitation regulator control unit together with the dynamic event triggering mechanism designed in the step three, wherein the dynamic event triggering mechanism continuously monitors the states of the generator and the power grid system, and if and only if the triggering conditions are met, the deep reinforcement learning controller is called to calculate and update the current control signal output, otherwise, the control unit keeps the control signal at the last triggering moment unchanged.
2. The excitation control method based on dynamic event triggering and deep reinforcement learning of claim 1, wherein the state variables of the simulation mathematical model in the first step at least include power angle deviation, rotor angular velocity deviation and q-axis transient potential deviation, and the differential equation is described as follows: Wherein, the As the derivative of the power angle deviation with respect to time in units of Is the deviation of the power angle caused by the deviation of the rotating speed from the synchronous speed, and represents the deviation of the power angle Is a rate of change of (2); Is a constant for the reference angular frequency of the system, wherein Is the rated frequency of the system, and has the unit of ; In units of rotor angular velocity deviation Is the actual angular velocity of the rotor Synchronous with angular velocity A difference between; As the derivative of angular velocity deviation with respect to time in units of Is the angular velocity deviation of the rotor Is a rate of change of (2); Is an inertial time constant, and has the unit of The rotor moment of inertia; the unit is electromagnetic power deviation Is the electromagnetic power output by the generator With an initial steady state value Is a difference in (2); Is the damping coefficient, the unit is Is the natural damping action of the generator rotor; The derivative of the transient potential deviation of the q axis with respect to time is expressed in units of Is the rate of change of the transient potential deviation; Is the d-axis open circuit transient time constant, and has the unit of Is the core time constant of the excitation system; Is the exciting electromotive force deviation, and the unit is Is the output of the excitation system; For q-axis synchronous potential deviation in units of Is an algebraic variable.
3. The excitation control method based on dynamic event triggering and deep reinforcement learning as set forth in claim 1, wherein the dynamic event triggering mechanism in the third step has the following mathematical triggering conditions: Wherein, the Is a norm; Indicating the current time System state vector of (2) With the last event trigger time State vector of (a) Error vector between them, dimensionless; a dimensionless time derivative of the error vector; is a weight coefficient larger than zero and is used for adjusting the influence degree of a change rate term in trigger decision The importance of the error change speed in the trigger decision is determined; the dynamic event triggering threshold is dimensionless, is a positive number which changes along with time, and determines the width of the triggering condition.
4. The excitation control method based on dynamic event triggering and deep reinforcement learning as set forth in claim 3, wherein the dynamic event triggering threshold value The specific design formula of (2) is as follows: Wherein, the The threshold value parameter is a threshold value adopted by the system in stable operation, and is dimensionless; For threshold adjustment of sensitivity coefficient, the unit is Is an adjustable parameter greater than zero; The derivative of the active power of the generator with respect to time is dimensionless, and the derivative is the change rate of the active power and is used for representing the size and urgency of disturbance of the system in real time.
5. The excitation control method based on dynamic event triggering and deep reinforcement learning as set forth in claim 1, wherein in the second step, a depth deterministic strategy gradient algorithm is adopted to conduct an off-line training to design a reward function The specific form is as follows: Wherein, the Describing environment feedback of offline training at the moment t for instant rewards, and having no dimension; Is the voltage deviation of the machine end, and the unit is ; In units of rotor angular velocity deviation , As active power deviation in units of ; The unit of the auxiliary excitation control signal output by the deep reinforcement learning controller at the time t is ; 、、、 The weight coefficients are all larger than zero, and are used for balancing the voltage stabilization, damping enhancement and control cost, and the optimization targets are dimensionless.

Description

Excitation control method based on dynamic event triggering and deep reinforcement learning Technical Field The invention belongs to the technical field of automation and control of power systems, and particularly relates to an excitation control method based on dynamic event triggering and deep reinforcement learning. Specifically, the invention relates to an innovative excitation controller design method integrating deep reinforcement learning and advanced event triggering control, which aims to improve the self-adaptive capacity, control quality and operation efficiency of an excitation system in a complex power grid environment through offline learning and online intelligent decision. Background The excitation control system of the synchronous generator is one of core equipment for guaranteeing safe and stable operation of the power system, and the main tasks of the excitation control system include maintaining the generator terminal voltage at a given level, reasonably distributing reactive power among parallel operation units and improving the static and dynamic stability of the power system. Conventional excitation control commonly employs PID (proportional-integral-derivative) regulators based on classical control theory, with the addition of Power System Stabilizers (PSS) to provide positive damping, suppressing low frequency power oscillations. However, with the continued development of power systems, particularly the integration of large-scale renewable energy sources (e.g., wind power, photovoltaic), the operating characteristics of the power grid are becoming increasingly complex. The wide introduction of the power electronic equipment enables the system to present stronger nonlinearity and time variability, the limitations of the traditional linear fixed parameter controller (such as PID+PSS) are gradually exposed when the system faces the novel and strong random disturbance, firstly, the control parameters of the system are usually set for specific typical working conditions, when the system operating points are greatly deviated or disturbance forms which are not covered by the design working conditions occur, the control performance can be obviously reduced, the self-adaption capability is insufficient, secondly, the PSS is mainly designed for low-frequency oscillation of a frequency range of 0.1-2.0 Hz, and the inhibition effect is limited for oscillation modes of higher frequency ranges or complex multimode oscillation. Deep reinforcement learning (English holonomic name: deep Reinforcement Learning, abbreviated as DRL) is an important branch in the field of artificial intelligence, and has great potential in solving the problem of nonlinear system control because of the fact that the deep reinforcement learning can autonomously learn a complex decision strategy through interaction with the environment. Theoretically, a fully trained DRL controller can approach a complex nonlinear optimal control law, potentially exceeding the performance of a conventional linear controller. However, the application of DRL directly to industrial processes with extremely high requirements on safety and reliability, such as excitation control, poses serious challenges, namely firstly, online learning or training phase, where an intelligent agent needs to explore the environment through "trial and error", unstable control actions in the process may form unacceptable risks for practical power equipment and grid safety, and secondly, the DRL controller generally needs high frequency state sampling and control quantity calculation, which will form a heavy burden on hardware computing resources of excitation regulators and system communication bandwidth. The event-triggered control is a resource-aware control method aimed at reducing unnecessary control update and communication, and its basic idea is that execution of a control task and update of a control signal are triggered only when the degree to which the system state deviates from the equilibrium point exceeds a certain preset threshold, otherwise the last control output is maintained. In the prior art, event trigger control is applied to research of an excitation system, and a fixed event trigger threshold value is mostly adopted. The fixed threshold is simple in design, but lacks adaptability, namely the control performance and even stability can be influenced due to untimely updating when the system suffers from large disturbance, and the threshold is close to periodic control when the threshold is too low, so that resources can not be effectively saved. Therefore, a mechanism that can intelligently adjust the trigger threshold according to the dynamic behavior of the system is critical to achieving the best balance between ensuring control performance and optimizing resource usage. In summary, the prior art has the problem to be solved in how to design a new excitation control method which has strong nonlinear processing capability and excellent dynamic