Search

CN-121984403-A - Model-free self-adaptive motor control method driven by reinforcement learning

CN121984403ACN 121984403 ACN121984403 ACN 121984403ACN-121984403-A

Abstract

The invention relates to the technical field of motor control, in particular to a model-free self-adaptive motor control method driven by reinforcement learning, which comprises the steps of firstly collecting three-phase current of a motor, obtaining dq-axis current through conversion, calculating feedback rotating speed through an observer, filtering, and outputting Q-axis reference current through PI regulation; reconstructing model equation and discretizing, determining observation gain coefficient, calculating dq axis reference voltage, correcting by delay compensation, inputting SVPWM module to generate PWM wave driving motor, constructing discretized state space, action space and rewarding function, and dynamically optimizing control parameters by cooperative updating of strategy network and Q network. The invention greatly reduces the parameter dependence, realizes the parameter adaptive optimization, gives consideration to the situations of rotation speed tracking speed, current harmonic suppression and energy consumption saving, adapts to frequency conversion base stations, small precise air conditioners and the like, and improves the control stability and the batch adaptation efficiency.

Inventors

  • LI XIANGSONG
  • HE FANG
  • FANG HUI
  • WANG XIAOCHONG

Assignees

  • 广东海悟科技有限公司

Dates

Publication Date
20260505
Application Date
20251230

Claims (10)

  1. 1. A model-free self-adaptive motor control method driven by reinforcement learning is characterized by comprising the following steps: s1, collecting three-phase current output by a motor Acquiring dq-axis current And (3) with ; S2, acquiring the motor angle through an observer, and calculating the feedback rotating speed by combining the sampling time of the rotating speed ring ; S3, feeding back the rotating speed Performing low-pass filtering treatment; S4, the target rotating speed and the feedback rotating speed Output q-axis reference current through PI regulation ; S5, constructing a model equation of the motor, wherein the model equation is as follows Wherein And In the differential form of the current in the dq axis; And Is the motor stator voltage; S6, carrying out linearization and discretization processing on the model equation to construct a discretization equation set containing dq-axis prediction current and a predicted value, and determining a dq-axis observation gain coefficient by solving a state space equation 、 、 ; S7, calculating dq axis reference voltage according to the dq axis reference current and the dq axis actual current And ; S8, modulating and correcting the motor stator voltage to obtain corrected voltage And ; S9, reference voltage And And inputting the PWM signals into an SVPWM module to generate PWM waves to drive the switching tube.
  2. 2. The model-free adaptive motor control method driven by reinforcement learning according to claim 1, further comprising the steps of: s10, constructing a discretization state space equation Wherein For the d-axis predicted current to actual current error, For the q-axis predicted current to actual current error, As the rate of change of the d-axis voltage, The q-axis voltage change rate; s11, defining an action space ; S12, setting a reward function Wherein And (3) with As the weight of the material to be weighed, Greater than ; S13, based on state transition The cumulative rewards are maximized by policy network updates, and the loss functions are minimized by Q network updates.
  3. 3. The model-free adaptive motor control method driven by reinforcement learning according to claim 1, wherein in step S1, three-phase current outputted by the motor is collected Three-phase current The dq axis current is obtained after Clarke transformation and Park transformation in sequence And (3) with ; In step S2, the motor angle is acquired by the observer Combining the sampling time of the rotating speed ring Calculating a feedback rotational speed The calculation formula is Wherein For the current motor position, The motor position at time k-1; in step S3, the feedback rotation speed is determined by the Butterworth discrete first order low pass filtering technique Low-pass filtering to obtain filter coefficients meeting 、 Wherein And In order for the filter coefficients to be of a type, Is the bandwidth of the rotating speed ring.
  4. 4. The model-free adaptive motor control method driven by reinforcement learning according to claim 1, wherein in step S5, Comprising stator resistances D-axis inductor Rotational speed of motor Current on d-axis Q-axis inductor Q-axis current ; Comprising stator resistances Q-axis inductor Rotational speed of motor Current on q-axis D-axis inductor Current on d-axis Flux linkage ; Is an inductance with d axis The parameter of the correlation is set to be, For inductance with q axis Related parameters.
  5. 5. The model-free adaptive motor control method driven by reinforcement learning according to claim 1, wherein in step S6, the discretization equation set is 、 ; Wherein the method comprises the steps of And The currents are predicted for the dq axes respectively, And The actual current at the moment k is the actual current, And The currents are predicted for the dq axes at times k +1 respectively, In order to sample the time of the current, And Respectively the predicted value of the k moment, 、 Respectively the predicted value of k+1 time, For the d-axis predicted current to actual current error, For the q-axis predicted current to actual current error, 、 、 And Gain coefficients are observed for the dq axes, respectively.
  6. 6. The model-free adaptive motor control method driven by reinforcement learning according to claim 5, wherein in step S6, the observation gain coefficient Satisfy the formula Wherein For the bandwidth of the d-axis state observer, Is the bandwidth of the q-axis state observer.
  7. 7. The model-free adaptive motor control method driven by reinforcement learning according to claim 5, characterized in that in step S7, the dq-axis reference voltage is obtained And The calculation formula of (2) is Wherein And The dq-axis reference current at k +2 respectively, And The actual current on the dq axis at time k+1, respectively.
  8. 8. The model-free adaptive motor control method of reinforcement learning drive of claim 1, wherein the motor stator voltage is modulated and corrected in step S8 to obtain a corrected voltage And The method comprises the following steps: Obtaining a voltage vector from the SVPWM module, and obtaining the motor stator voltage by using the voltage vector and the acting time And ; Modulation factor Wherein In order to switch on and off the time, Respectively vector acting time and correction voltage And Satisfy the following requirements 、 。
  9. 9. The model-free adaptive motor control method driven by reinforcement learning according to claim 2, wherein in step S13, the objective function of the strategy network is ; The policy gradient of the policy network is that ; Wherein the method comprises the steps of For the network to learn the weights and bias sums, In order to evaluate the value of the action, For the output of the optimal parameter adjustment quantity, As a function of the object to be processed, In order to provide a strategic gradient, Is a discount factor.
  10. 10. A model-free adaptive motor control method driven by reinforcement learning according to claim 2, wherein the loss function of the Q network is ; Target network update satisfaction ; Wherein the method comprises the steps of As a function of the loss, In order for the network to be a target, In order to be able to take time, Is a discount factor.

Description

Model-free self-adaptive motor control method driven by reinforcement learning Technical Field The invention belongs to the technical field of motor control, and particularly relates to a model-free self-adaptive motor control method driven by reinforcement learning. Background In industrial production and civil equipment, a motor is used as a core power component, the control performance of the motor directly influences the running efficiency, stability and energy consumption level of the equipment, and the motor control technology is of great importance in scenes with high requirements on control precision and suitability, such as frequency conversion base stations, small precise air conditioners and the like. First, a conventional field oriented control scheme (FOC) is employed. The technology converts three-phase current of a motor into a dq axis coordinate system through Clarke transformation and Park transformation, performs closed-loop control on the dq axis current and the rotating speed by adopting a proportional-integral (PI) regulator based on an accurate mathematical model of the motor, and finally generates a driving signal through Space Vector Pulse Width Modulation (SVPWM) to control the motor to operate. However, the control mode has extremely high accuracy requirements on motor parameters, core parameters such as stator resistance, d/q axis inductance, flux linkage and the like of the motor need to be accurately acquired, and the fixed PI adjustment parameters are relied on, so that when the motor drifts due to batch difference, working condition change or aging, the control accuracy is obviously reduced, and even system oscillation is caused. Secondly, a second-order supercoiled sliding mode algorithm is adopted to design a speed ring, and a control scheme of a high-order sliding mode observer is combined. According to the technology, load disturbance is restrained through robustness of a sliding mode algorithm, a high-order sliding mode observer is utilized to estimate load change in real time, and a q-axis current reference value is generated through feedforward compensation, so that the purposes of restraining buffeting and improving control stability are achieved. However, the core control logic of the scheme still depends on accurate model parameters of the motor, especially key parameters such as rotational inertia, and when the parameters are not matched, the estimation accuracy of an observer and the robustness of a sliding mode algorithm can be greatly reduced, so that the problems of motor rotation speed tracking lag, current harmonic increase and the like are caused. In summary, in the current motor control field, the prior art generally has the defects of dependence on an accurate motor model, complicated parameter debugging, low batch adaptation efficiency, difficulty in considering multiple performance indexes and the like, and limits the application of the motor control method in scenes with high requirements on control performance and suitability, such as variable frequency base stations, small precise air conditioners and the like. Disclosure of Invention The invention aims to overcome the defects in the prior art and provide a model-free self-adaptive motor control method driven by reinforcement learning. The invention discloses a model-free self-adaptive motor control method driven by reinforcement learning, which comprises the following steps: s1, collecting three-phase current output by a motor Acquiring dq-axis currentAnd (3) with; S2, acquiring the motor angle through an observer, and calculating the feedback rotating speed by combining the sampling time of the rotating speed ring; S3, feeding back the rotating speedPerforming low-pass filtering treatment; S4, the target rotating speed and the feedback rotating speed Output q-axis reference current through PI regulation; S5, constructing a model equation of the motor, wherein the model equation is as followsWhereinAndIn the differential form of the current in the dq axis; And Is the motor stator voltage; S6, carrying out linearization and discretization processing on the model equation to construct a discretization equation set containing dq-axis prediction current and a predicted value, and determining a dq-axis observation gain coefficient by solving a state space equation 、、; S7, calculating dq axis reference voltage according to the dq axis reference current and the dq axis actual currentAnd; S8, modulating and correcting the motor stator voltage to obtain corrected voltageAnd; S9, reference voltageAndAnd inputting the PWM signals into an SVPWM module to generate PWM waves to drive the switching tube. The invention is further arranged to further comprise the steps of: s10, constructing a discretization state space equation WhereinFor the d-axis predicted current to actual current error,For the q-axis predicted current to actual current error,As the rate of change of the d-axis voltage,The q-axis voltage change ra