CN-122026398-A - Island micro-grid secondary frequency Q learning control method based on bandwidth sensing event triggering

Abstract

The invention discloses an island micro-grid secondary frequency Q learning control method based on bandwidth sensing event triggering. The invention provides a novel bandwidth sensing event-triggered secondary frequency optimization control method based on reinforcement learning aiming at the problem of micro-grid frequency recovery. Firstly, different from the traditional event triggering mode, the invention provides a bandwidth-aware-based event triggering mechanism, which can realize the self-adaptive adjustment of the triggering threshold according to the real-time bandwidth condition and the system dynamics and reasonably utilize the communication resources. Secondly, the stability analysis is carried out on the micro-grid system by utilizing the Lyapunov stability theory, and the parameter upper boundary condition for ensuring the stability of the system is obtained by solving the linear matrix inequality. Finally, the invention provides a self-adaptive secondary controller gain adjustment method based on Q learning, which can adaptively adjust the secondary frequency controller gain according to the system state so as to realize more efficient secondary frequency adjustment and improve the system stability.

Inventors

• YAN SHEN
• Dou Zehao
• ZHANG KUN

Assignees

• 南京林业大学
• 南京众行能源科技有限公司

Dates

Publication Date

20260512

Application Date

20260114

Claims (2)

1. 1. A novel island micro-grid secondary frequency Q learning control method based on bandwidth sensing event triggering is characterized by comprising the following steps: step 1, establishing a micro-grid frequency secondary control model based on droop control, which specifically comprises the following steps: 1) The micro-grid frequency secondary control model based on droop control of the ith inverter is as follows: ω i ＝ω′-m i (P i -P i ′)+z i , (1) Wherein ω i is the output frequency amplitude of the ith distributed power supply, ω 'is the nominal set point of the frequency amplitude, P i is the output active power of the ith distributed power supply, P i ' is the reference active power value of the ith distributed power supply, and m i is the droop coefficient representing the relationship of power and frequency deviation. 2) Introducing error variables The state space model of the island microgrid is changed to: Wherein z i is the secondary controller of DG i , and the mathematical expression is: In the middle of Is an output frequency controller. Step2, in order to improve the utilization efficiency of a communication network, designing a bandwidth-aware event-triggered island micro-grid secondary frequency controller, wherein the method comprises the following steps of: 1) The triggering process of the event triggering mechanism is shown in the formula (4), and is characterized in that the invention assumes that all data transmission delays epsilon ij are smaller than the sampling period h of the system. When the ith distributed power supply transmits the sampling data to the adjacent distributed power supply through the communication network at the current sampling time d s , the arrival time of the transmission data does not exceed d s+1 , namely d s +ε ij (d s )≤d s+1 , specifically as follows: Wherein: In the middle of Delta i (t) is an event trigger threshold parameter of the ith distributed power supply, d s -1 is the last sampling time; triggering time for the ith event of the ith controller; b i ＝1,b i =0 indicates whether the i-th node can obtain the reference frequency, respectively. Is an error signal; And Represents the upper and lower bounds of delta i (t), c 0 ,c 1 ,c 2 ,c 3 ,c 4 is a positive number, satisfying c 1 -c 2 ＝c 3 +c 4 =1, Gamma (t) ∈0,1 represents a bandwidth status real-time indication signal. 2) The method comprises the following steps of designing an island micro-grid secondary frequency controller based on bandwidth perception type event triggering: Where k ω <0 is the secondary frequency controller gain. And 3, designing a secondary frequency control method of the island micro-grid based on bandwidth sensing event triggering, deriving a frequency control system model of the island micro-grid, giving out comprehensive design conditions of the controller by utilizing a Liapunov theory and a Wirtinger inequality technology, solving the parameters of the controller, and further adjusting the frequency of the island micro-grid. The method comprises the following steps: 1) deriving an island micro-grid closed-loop frequency control system model, combining derivatives of omega i in formulas (2) and (3), Is transformed into: The above equation is converted to for t ε [ d s ,d s+1 ] by equations (2) - (4): where a ij ＝1,a ij =0 indicates whether the i-th node is adjacent to the j-th node, respectively. 2) Order the The following closed loop system is obtained: Wherein the method comprises the steps of 3) In order to solve the parameters of the controller, the following Leidefenov function is first selected: Wherein: in the present invention, < q|x (t) > means x (t) T Qx (t). 4) Calculating 5) Using the Wirtinger inequality, the integral term in equations (17), (18) can be scaled as follows: Wherein: 6) The following augmentation vectors are defined: Wherein: 7) Based on equation (9), constructing the zero term can result in: Wherein, the 8) On the basis of the above step, the method can be as follows: Wherein, the 9) From the event trigger function designed in (4), it is possible to obtain: 10 To stabilize the system (8), the derivative of the li-apunov function needs to satisfy: Wherein: ∑=Γ 11 +Ω 1 +Ω 2 , 11 By solving the matrix inequality Σ < 0), the controller gain and event trigger threshold parameters can be derived: k ω ,δ ω,M ＝diag{δ 1,M ,...,δ n,M }。 12 The controller parameters are brought into a closed-loop frequency control system, and the island micro-grid frequency is adjusted so that the island micro-grid frequency can track and stabilize the reference frequency value.
2. 2. Step 4, according to the bandwidth sensing type event-triggered island micro-grid secondary frequency Q learning control method disclosed by claim 1, the method is characterized in that the micro-grid is regarded as a multi-Agent system, and each distributed power supply corresponds to one Agent. The control system can adaptively adjust the gain of the frequency controller according to different system operation conditions by introducing a Q learning algorithm to interact with the system so as to improve the frequency control performance of the system. The method comprises the following steps: 1) The invention disperses the system average frequency deviation deltaf into n intervals, thereby defining the state of Agent i as S i ＝{s 1 ,s 2 ,...,s n . The expression of the average frequency deviation Δf is as follows: 2) The present invention selects n appropriate controller gains as actions. Motion space a i ＝{a 1 ,a 2 ,...,a m , where a m is the upper bound of the quadratic frequency controller gain for LMI solutions. 3) The invention sets the reward function deltaf of the Q learning module as: Wherein, the 0<G 1 ＜g 2 ＜g 3 ＜g 4 is a positive constant set to divide the Δω i interval. 4) Since the micro-grid generally cannot obtain global information, the invention designs a coordinated average rewards To approximate a global prize, enabling it to achieve consistent global goals and to achieve optimal frequency control performance. Coordinated average rewards The formula of (2) is as follows: Wherein, the And Representing rewards obtained by Agent i and its neighboring Agent j when Agent i selects optimal behavior a i , and alpha i and beta i as weight coefficients. 5) The invention updates the Q value by adopting the following Q-Learning update formula: 6) And obtaining an optimal strategy through Q learning and environment interaction, and importing the trained optimal strategy into a micro-grid system, so that the secondary frequency Q learning controller can adaptively adjust the gain of the secondary frequency controller according to the system environment change.

Description

Island micro-grid secondary frequency Q learning control method based on bandwidth sensing event triggering Technical Field The invention relates to a novel island micro-grid secondary frequency Q learning control method based on bandwidth sensing event triggering, in particular to an island micro-grid bandwidth sensing event triggering control method with a controller gain capable of being adaptively adjusted according to a system environment. Background The micro-grid is a small power generation and distribution system integrating distributed power sources (such as solar energy, wind energy, tidal energy and the like), energy storage devices and loads, and compared with a traditional centralized power grid, the micro-grid is more flexible to operate and improves the utilization rate of sustainable energy. When the micro-grid is operated in a grid-connected working state, the micro-grid can be used for energy adjustment and optimization according to the requirements of the main grid. When the micro-grid operates in an island working state, the micro-grid can independently operate under the condition of grid faults or emergency, and continuous power supply is provided for users. This capability is particularly suitable for those places where high power reliability is required, such as hospitals, communication stations, important industrial places, etc. When the micro-grid operates in the island operation mode, an appropriate control strategy is required to realize coordinated control of the micro-grid system in order to maintain stable operation and power quality of the micro-grid. An event triggering mechanism is introduced in the secondary control link of the micro-grid, so that the communication cost of the micro-grid can be saved, and the waste of communication resources can be reduced. The mechanism allows the micro-grid to communicate data only in the adjacent distributed power sources under certain circumstances. The bandwidth sensing type event triggering mechanism can realize self-adaptive adjustment of the triggering threshold according to the real-time bandwidth condition and the real-time condition of the system. Reinforcement learning is a machine learning method that aims at making decisions to achieve specific goals by an agent learning in interaction with the environment. In reinforcement learning, an agent learns an optimal behavior strategy by observing the state of the environment, performing specific actions, and receiving rewards or penalties, and can be used to solve the adaptive optimization problem of microgrid operation control. In the field of micro-grid secondary frequency control, dynamic adjustment research on control gain is relatively few, and the existing method is difficult to adapt to complex and changeable operating environments. It is important to explore a method for adjusting gain parameters in real time according to system state. Therefore, the invention provides a dynamic micro-grid secondary frequency control method based on Q learning. Disclosure of Invention Based on the analysis, the invention establishes a micro-grid secondary control model based on droop control to construct an island micro-grid bandwidth sensing type event triggering control method based on Q learning, so as to realize higher efficient frequency adjustment efficiency in changeable operation environments, save the utilization of communication network resources and realize theoretical design of system stability analysis and controller parameters. The specific technical scheme of the invention is as follows, and the novel island micro-grid secondary frequency Q learning control method based on bandwidth sensing event triggering is characterized by comprising the following steps: step 1, establishing a micro-grid frequency secondary control model based on droop control; Step 2, designing an island micro-grid secondary frequency controller triggered on the basis of a bandwidth sensing event; Step 3, deriving an island micro-grid frequency control closed-loop system model, solving controller parameters, and adopting the solved controller to carry out frequency adjustment on the island micro-grid so as to enable the island micro-grid to track a given reference frequency; And 4, designing a Q learning-based island micro-grid frequency secondary control method, so that the island micro-grid frequency secondary control method can adaptively adjust the system secondary frequency control gain according to changeable working environments, and the stability of the system is improved. The island micro-grid secondary frequency Q learning control method based on bandwidth sensing event triggering is realized sequentially according to the following steps: firstly, establishing a micro-grid frequency secondary control model based on droop control, which is specifically as follows: ωi＝ω′-mi(Pi-Pi′)+zi, (1) Wherein ω i is the output frequency amplitude of the ith distributed power supply, ω 'is the nominal set poin