CN-122021268-A - Control parameter optimization solving method for network-structured energy storage converter based on data driving

Abstract

The invention discloses a data-driven network-structured energy storage converter control parameter optimization solving method, which combines the high efficiency of performing converter graphical topological programming and simulation verification by applying PSCAD with the real-time performance of performing deep reinforcement learning AI calculation by Python, constructs a data-driven converter PI control parameter on-line optimization solving framework, realizes the data-driven fusion of Python deep reinforcement learning evaluation of PI control parameter optimization and converter effective modeling simulation, and highlights the optimization control of network-structured converter dynamic performance indexes based on voltage, current and SOC small sample sampling data. The invention can improve the response speed and stability index of the new energy + flow battery energy storage system, breaks through the bottleneck limit of the traditional PI control on the dynamic performance index of the grid-built converter, and meets the evaluation requirement of the dynamic PI control parameter of the converter, which is suitable for the PCC power fluctuation following inhibition requirement.

Inventors

• WANG LIGUO
• HE SHENGFEI
• Ran Xianghui

Assignees

• 哈尔滨工业大学

Dates

Publication Date

20260512

Application Date

20260112

Claims (6)

1. 1. A method for optimizing and solving control parameters of a network-structured energy storage converter based on data driving is characterized by comprising the following steps: step 1, establishing a bus power controllable networking type converter main control circuit interactive topology simulation platform based on PSCAD, carrying out modeling analysis on PI control parameters of a power outer ring and a voltage/current inner ring based on optimized control of converter bridge arms powered by a flow battery and PWM driving signals of the flow battery, and solving and determining qualitative influence relation of the PI control parameters on active power, output voltage and current change rate through a control equation and a virtual inner potential equation; Step 2, developing Python solving software of an A2C deep reinforcement learning algorithm suitable for optimizing and solving control parameters of the grid-built converter, thereby realizing evaluation, prediction and verification of PI control parameters of the grid-built converter, providing feasible PI control parameters for the interactive topology simulation platform constructed in the step one in real time, and driving PI control data of a simulation model of the converter; Step 3, completing the environment construction of Python by configuring a virtual environment through Anaconda, realizing online regulation and control of converter control parameters based on Pytorch, and transmitting the obtained PI control parameters to the bus power-controllable networking converter main control circuit interactive topology simulation platform established in the step 1; Step 4, path configuration and engine starting setting of the PSCAD interactive simulation platform and the Python: Step 4-1, initializing a Python environment through pyenv command, and utilizing py.sys.path dynamic addition project path to ensure that the PSCAD simulation platform can be positioned to the Python module; Step 4-2, starting a PSCAD simulation platform engine by the Python through a COM Automation interface, and setting the Python to operate a simulation model in a remote calling mode; Step 5, performing PI Data-driven interactive PSCAD simulation platform and Python cross-platform collaborative simulation, namely extracting simulation Data from an output file of PSCAD, converting the simulation Data into an output. Cvs file through a Data Export module, and realizing closed-loop parameter optimization through linkage of three key links of environment configuration, data processing and parameter feedback; Step 6, deducing a mapping set of converter PI control parameters applicable to different power/current change rates by applying a Python-based A2C deep reinforcement learning algorithm to proportionality coefficients under different load conditions Sum and integral coefficient Performing iterative optimization, and setting the inner/outer loop PI control parameters by the algorithm under the condition of determining load variation 、 As PI control parameters for the system power loop, voltage/current loop.
2. 2. The method for optimizing and solving control parameters of the network-structured energy storage converter based on data driving according to claim 1, wherein the specific steps of the step 1 are as follows: step 1-1, a flow battery serving as a converter busbar energy supply link is equivalent to a controlled voltage source, and a PI control parameter closed-loop control model of a power outer loop and a voltage/current inner loop is established; step 1-2, building a container containing inertia Damping device Active power control and reactive power control submodules, d and q axis voltage/current controllers are built and embedded into the power grid , ) In the link, the step of processing the data, Is a proportional coefficient, Constructing a virtual impedance link to realize the virtual synchronization characteristic of the network-structured converter; step 1-3, constructing a research system comprising PCC voltage/current sampling, 3/2 coordinate transformation, active/reactive power calculation, voltage/current and power distortion rate analysis, flow battery SOC evaluation, converter control equation model derivation and the like , ) Optimizing control parameters; step 1-4, designing a PSCAD and Python data sharing interface ‌, and outputting the PSCAD through ‌ data file analysis And And sending the file to a Python script for processing, and extracting PCC voltage/current distortion rate information.
3. 3. The method for optimizing and solving control parameters of the network-structured energy storage converter based on data driving according to claim 1, wherein in the step 1, a control equation and a virtual internal potential equation are as follows: (1) (2) (3) (4) In the formula, For the magnitude of the virtual internal potential, For the virtual internal potential phase angle, In order to be a damping coefficient, For the virtual inertia to be a virtual inertia, As an actual value of the active power, As a result of the active reference value, Is the reactive power actual value of the reactive power, As a reference value for the reactive power, As a basic value of the voltage, the voltage is, For the virtual rotational speed rating of the VSG, Is the actual value of the virtual rotational speed of the VSG, Is used as a capacitor of a direct-current bus, As a reactive-voltage sag factor, For the d/q axis current reference value, Is the actual value of the d/q axis current, For the d/q axis voltage reference value, Is the actual value of the d/q axis voltage, Is the proportional/integral coefficient of the outer ring of voltages, Is the proportional coefficient/integral coefficient of the current inner loop, For the step-down d/q axis voltage reference value, To generalize the virtual resistance/inductance value.
4. 4. The method for optimizing and solving the control parameters of the network-structured energy storage converter based on data driving according to claim 1, wherein the specific steps of the step 2 are as follows: Step 2-1, A2C main function network structure design suitable for converter control optimization: (1) The shared feature extraction layer is used for extracting features of the voltage/current state through the full connection layer and the ReLU activation, and converting the original state into a high-dimensional feature vector; (2) Strategy network Actor optimizing converter by double output branch processing 、 Coefficients, outputting respective motion probability distributions; (3) The value network Critic takes the minimum voltage/current distortion rate as a criterion, evaluates the weight of the current distortion rate state and outputs a state value through a single full-connection layer; step 2-2, designing an action selection mechanism: (1) Parallel optimization 、 Action of two parameters parallel optimization 、 The optimal balance between dynamic performance and stability can be realized by the converter; (2) Converting output values into probability distributions using Softmax, actor output 、 The initial value is normalized through a Softmax function and converted into probability distribution, so that the interpretability of action selection is ensured; (3) Random sampling based on probability distribution to select specific parameter values for output 、 Sampling parameters are extracted according to the probability distribution of (a), the extracted sampling parameters are output, and new sampling parameters are obtained by combining PCC power fluctuation 、 An estimated value; Step 2-3, entropy calculation: (1) Simultaneous calculation of 、 Entropy values of two policy branches 、 The motion probability distribution of the branches respectively calculates entropy of the two motion distributions for strategy diversity analysis; (2) Will be 、 Is added as a total entropy term to the loss function: In the formula, Is a policy penalty for optimizing the action selection policy of the agent; is a coefficient of a cost function for adjusting the weight of the cost penalty in the total penalty; Is the loss of value, i.e., the error between the predicted value of the cost function and the true return; is a regularized coefficient of entropy, used for adjusting the weight of entropy in the total loss; Is the entropy of the strategy, is 、 The total entropy obtained by adding the two entropy values of the two blocks; Step 2-4, value estimation: (1) Critic processes the output characteristics of the shared characteristic extraction layer through an independent full connection layer and outputs state value; (2) Calculating a Q value through multi-step return; (3) The generalized dominance estimation is used to calculate the dominance function.
5. 5. The method for optimizing and solving the control parameters of the network-structured energy storage converter based on data driving according to claim 1, wherein the specific steps of the step3 are as follows: Step 3-1, python virtual environment configuration, namely inputting an instruction into a PowerShell to acquire a required function library, setting Mixstart modules for configuring and integrating the Python environment, wherein the Im modules are used for realizing cooperative work of inner ring PI parameter self-tuning based on reinforcement learning and an interactive simulation platform, and the Project modules are used for realizing cooperative work of outer ring PID parameter self-tuning based on deep reinforcement learning and on-line verification of Simulink hardware; and 3-2, after the configuration of the Python virtual environment is completed, path configuration of the interactive simulation platform and engine starting are carried out, so that cross-platform interaction between the interactive simulation platform and the Python is realized.
6. 6. The method for optimizing and solving the control parameters of the network-structured energy storage converter based on data driving according to claim 1, wherein the specific steps of the step 5 are as follows: Step 5-1, environment configuration, namely binding a Python interpreter path through Pyenv and Anacondenv, and loading an m script initialization interface; Step 5-2, data processing, namely running A2C.py at the Python end, analyzing data by using a deep reinforcement learning algorithm, and optimizing PI parameters 、 ; Step 5-3, parameter feedback, namely saving the optimization result as final_pid_values.pkl, and passing PSACD Automation Interface 、 And (3) injecting control parameters into the established interactive simulation platform of the main control circuit of the network-structured converter with controllable bus power in the step one, so as to form a closed loop of simulation, training and parameter adjustment.

Description

Control parameter optimization solving method for network-structured energy storage converter based on data driving Technical Field The invention belongs to the technical field of application foundations of a deep reinforcement learning method A2C (ADVANTAGE ACTOR-Critic) in power electronics and artificial intelligence, and aims to develop a control parameter optimization solving method of a grid-built energy storage converter, in particular to a data-driven Python control parameter optimization and PSCAD (Power Systems Computer AIDED DESIGN) effective verification-based grid-built energy storage converter optimization design method. Background As an intermediate medium for energy exchange between the energy storage flow battery and the new energy grid-connected system, the grid-structured converter needs to simultaneously consider SOC (State of Charge) regulation and control of the flow battery and PCC (Point of Common Coupling) new energy grid-connected impact inhibition, and therefore the grid-structured converter is required to have higher response speed and stronger robust inhibition effect. The requirement is expressed from the control level that the converter should have the function of dynamically adjusting proportional integral parameters so as to solve the problem of PI determination,) Problems such as overshoot of response induced by regulation, power oscillation and the like. However, how to define PI control parameters in real time according to PCC actual measurement data in high efficiency is still one of the technical problems that plagues the current grid-connected converters to be solved in the need of high-speed and random fluctuation of the new energy grid-connected system. The deep reinforcement learning method based on data driving can provide a solving thought for solving the problem of adjusting the dynamic PI control parameters in the real-time operation of the grid-structured energy storage converter. According to the development of a deep reinforcement learning A2C (ADVANTAGE ACTOR-Critic) parallel operation algorithm, a shared bottom layer feature extraction module is adopted to process input state information, an action strategy containing proportional integral parameters is generated, dynamic evaluation of state value is carried out, and accordingly the problems of dynamic response lag, overshoot disturbance and the like caused by new energy grid connection are suppressed. The research difficulty is how to conduct PI prediction according to limited sample data and how to give out PI control parameters suitable for PCC actual working conditions in real time according to actual measurement data, so that a research method for conducting PI control parameter online prediction and evaluation based on existing converter voltage, current and SOC small samples is continuously explored. Disclosure of Invention Aiming at the urgent requirement of the energy storage of the flow battery and the new energy grid connection on the improvement of the dynamic performance index of the grid-built converter, the invention provides a grid-built energy storage converter control parameter optimization control method based on data driving so as to meet the dynamic PI control parameter evaluation requirement of the converter which is suitable for the PCC power fluctuation following inhibition requirement. The method combines the high efficiency of performing converter graphical topological programming and simulation verification by using PSCAD with the real-time performance of performing deep reinforcement learning AI calculation by Python, constructs an online optimization solving framework of the PI control parameters of the converter based on data driving, realizes the data driving fusion of Python deep reinforcement learning evaluation of PI control parameter optimization and converter effective modeling simulation, and highlights the optimization control of the dynamic performance index of the networking converter based on voltage, current and SOC small sample sampling data. The invention aims at realizing the following technical scheme: A method for optimizing and solving control parameters of a network-structured energy storage converter based on data driving comprises the following steps: Step 1, establishing a bus power controllable networking type converter main control circuit interactive topology simulation platform based on PSCAD, optimally controlling converter bridge arms powered by a flow battery and PWM driving signals driven by the flow battery, carrying out modeling analysis on PI control parameters of a power outer ring and a voltage/current inner ring, and solving and determining qualitative influence relation of the PI control parameters on active power, output voltage and current change rate through a control equation and a virtual inner potential equation, wherein the method comprises the following specific steps of: step 1-1, a flow battery serving as a converter busbar energy