CN-121978896-A - Servo controller parameter adjustment method and system based on deep reinforcement learning

CN121978896ACN 121978896 ACN121978896 ACN 121978896ACN-121978896-A

Abstract

The invention discloses a servo controller parameter adjustment method and system based on deep reinforcement learning, which relate to the technical field of data analysis, and are used for acquiring a rigidity grade of a servo controller to determine an action space boundary of parameter adjustment, carrying out feature analysis on a motor corresponding to the servo controller to obtain a target steady-state feature and a target dynamic feature, carrying out load parameter analysis on the servo controller based on frequency response, carrying out performance change analysis on the servo controller, generating a target multidimensional state space based on the action space boundary, the target steady-state feature, the target dynamic feature, the target load parameter and the target performance change parameter, analyzing an adjustment quantity of the servo controller parameter by using the target multidimensional state space based on a deep reinforcement learning agent, and updating a PID parameter of the servo controller based on the adjustment quantity. The invention enables the servo controller to constantly maintain the optimal control parameters matched with the current characteristics of the controlled object, and realizes the self-adaptive control of the whole life cycle.

Inventors

Qu Shiquan
SHI BENYAN
HE YILANG

Assignees

佛山德玛特智能装备科技有限公司

Dates

Publication Date: 20260505
Application Date: 20260408

Claims (10)

1. A method for adjusting parameters of a servo controller based on deep reinforcement learning, the method comprising: obtaining a rigidity grade of a set servo controller, and determining an action space boundary adjusted by parameters of the servo controller based on the rigidity grade; performing characteristic analysis on a motor corresponding to the servo controller to obtain a target steady-state characteristic and a target dynamic characteristic; Performing load parameter analysis on the servo controller based on the frequency response to obtain target load parameters, and performing performance change analysis on the servo controller to obtain target performance change parameters; Generating a target multidimensional state space based on the action space boundary, the target steady-state characteristic, the target dynamic characteristic, the target load parameter and the target performance variation parameter; constructing a deep reinforcement learning intelligent agent, and analyzing the adjustment quantity of the servo controller parameters by utilizing the target multidimensional state space based on the deep reinforcement learning intelligent agent; and updating the proportional-integral-derivative PID parameter of the servo controller based on the adjustment quantity.
2. The method for adjusting parameters of a servo controller based on deep reinforcement learning according to claim 1, wherein the determining the action space boundary of the servo controller parameter adjustment based on the stiffness level obtained from the set servo controller comprises: acquiring a rigidity grade instruction set by a user based on an upper computer communication interface, and analyzing the rigidity grade instruction to acquire the rigidity grade of the servo controller; acquiring a rigidity grade-PID parameter adjustment range mapping table, and determining a parameter boundary by utilizing the rigidity grade based on the rigidity grade-PID parameter adjustment range mapping table; And determining an action space boundary of servo controller parameter adjustment based on the parameter boundary.
3. The method for adjusting parameters of a servo controller based on deep reinforcement learning according to claim 1, wherein the performing feature analysis on a motor corresponding to the servo controller to obtain a target steady-state feature and a target dynamic feature comprises: Collecting steady-state data in a steady-state application scene of a motor corresponding to a servo controller, and performing feature analysis on the steady-state data to obtain target steady-state features; and acquiring dynamic data in a dynamic application scene of the motor corresponding to the servo controller, and performing feature analysis on the dynamic data to obtain target dynamic features.
4. The method for adjusting parameters of a servo controller based on deep reinforcement learning according to claim 1, wherein the frequency response-based method for analyzing load parameters of the servo controller to obtain target load parameters, and performing performance change analysis of the servo controller to obtain target performance change parameters comprises: Injecting a first input signal into a motor corresponding to a servo controller, and collecting a first output signal output by the motor under the first input signal; determining frequency response data using a fourier transform based on the first input signal and the first output signal; Constructing a reference load model, performing curve fitting on the frequency response data based on the reference load model to obtain a target curve, and determining an initial load value based on the target curve; Performing iterative optimization on the initial load value to obtain a target load parameter; And constructing a reference performance index library, and carrying out performance change analysis on the servo controller based on the reference performance index library to obtain target performance change parameters.
5. The method for adjusting parameters of a servo controller based on deep reinforcement learning according to claim 4, wherein the performing performance change analysis on the servo controller based on the reference performance index library to obtain the target performance change parameters comprises: Setting a sliding time window, and acquiring performance index information of the operation process of the servo controller based on the sliding time window; analyzing absolute change amount and relative change rate by using the reference performance index library based on the performance index information, and determining performance change amount based on the absolute change amount and the relative change rate; And analyzing the performance degradation trend based on the performance index information, and determining a target performance change parameter based on the performance change amount and the performance degradation trend.
6. The method of claim 1, wherein generating the target multi-dimensional state space based on the action space boundary, the target steady state feature, the target dynamic feature, the target load parameter, and the target performance variation parameter comprises: Normalizing the action space boundary, the target steady-state characteristic, the target dynamic characteristic, the target load parameter and the target performance change parameter to obtain the action space boundary, the target steady-state characteristic, the target dynamic characteristic, the target load parameter and the target performance change parameter after normalization; And generating a target multidimensional state space based on the normalized action space boundary, the target steady-state characteristic, the target dynamic characteristic, the target load parameter and the target performance change parameter.
7. The method for adjusting parameters of a servo controller based on deep reinforcement learning according to claim 1, wherein the constructing a deep reinforcement learning agent and analyzing an adjustment amount of parameters of the servo controller based on the deep reinforcement learning agent using the target multi-dimensional state space comprises: Constructing a depth reinforcement learning agent based on the actor-critic network; And inputting the target multidimensional state space into the deep reinforcement learning intelligent agent to analyze the adjustment quantity of the servo controller parameters.
8. The method for adjusting parameters of a servo controller based on deep reinforcement learning according to claim 1, wherein updating the proportional-integral-derivative PID parameters of the servo controller based on the adjustment amount comprises: acquiring current PID parameters of a servo controller, updating the PID parameters based on the adjustment quantity, and acquiring updated PID parameters; performing multi-level security verification on the updated PID parameters to obtain multi-level security verified PID parameters; Performing smooth transition on the PID parameters subjected to the multi-level security verification to obtain PID parameters subjected to the smooth transition; And controlling a current loop, a speed loop and a position loop of the servo controller based on the PID parameters after the smooth transition.
9. The method for adjusting parameters of a servo controller based on deep reinforcement learning according to claim 8, wherein the performing multi-level security check on the updated PID parameters to obtain multi-level security checked PID parameters includes: performing action space boundary verification on the updated PID parameters to obtain PID parameters after the action space boundary verification; Performing absolute value boundary verification on the PID parameters subjected to the action space boundary verification to obtain PID parameters subjected to the absolute value boundary verification; And performing stability criterion verification on the PID parameters subjected to absolute value boundary verification to obtain PID parameters subjected to multi-level security verification.
10. A servo controller parameter adjustment system based on deep reinforcement learning, which adopts the servo controller parameter adjustment method based on deep reinforcement learning as set forth in any one of claims 1 to 9, characterized in that the system comprises: the action boundary determining module is used for obtaining the rigidity grade of the set servo controller and determining an action space boundary adjusted by the servo controller parameter based on the rigidity grade; The characteristic analysis module is used for carrying out characteristic analysis on the motor corresponding to the servo controller to obtain a target steady-state characteristic and a target dynamic characteristic; The parameter analysis module is used for carrying out load parameter analysis on the servo controller based on the frequency response to obtain target load parameters, and carrying out performance change analysis on the servo controller to obtain target performance change parameters; The state space output module is used for generating a target multidimensional state space based on the action space boundary, the target steady-state characteristic, the target dynamic characteristic, the target load parameter and the target performance change parameter; the adjustment quantity analysis module is used for constructing a deep reinforcement learning intelligent agent and analyzing the adjustment quantity of the servo controller parameters by utilizing the target multidimensional state space based on the deep reinforcement learning intelligent agent; And the parameter updating module is used for updating the proportional-integral-derivative PID parameter of the servo controller based on the adjustment quantity.

Description

Servo controller parameter adjustment method and system based on deep reinforcement learning Technical Field The invention relates to the technical field of data analysis, in particular to a servo controller parameter adjustment method and system based on deep reinforcement learning. Background The servo controller is used as a core execution unit of the industrial robot and the high-end numerical control machine tool, and the control performance of the servo controller directly determines the precision, the efficiency and the stability of production equipment. At present, most servo controllers still adopt a PID control algorithm or a deformation structure thereof, and the tuning quality of PID parameters is a key factor influencing the response speed, overshoot and steady-state precision of the servo system. The traditional parameter setting method mainly relies on a debugging engineer to manually adjust parameters according to the running sound and response curve of the motor. However, the method is time-consuming, the setting quality is seriously dependent on the experience level of debugging personnel, and when the method is faced with complex working conditions such as variable load, variable inertia and the like, the global optimal solution is often difficult to find by manual setting, so that the equipment performance cannot be fully exerted. Disclosure of Invention The invention aims to overcome the defects of the prior art, and provides a servo controller parameter adjustment method and a system based on deep reinforcement learning, so that the servo controller can keep the optimal control parameters matched with the current characteristics of a controlled object at any time, and the self-adaptive control of the whole life cycle is realized. In order to solve the technical problems, the invention provides a servo controller parameter adjustment method based on deep reinforcement learning, which comprises the following steps: obtaining a rigidity grade of a set servo controller, and determining an action space boundary adjusted by parameters of the servo controller based on the rigidity grade; performing characteristic analysis on a motor corresponding to the servo controller to obtain a target steady-state characteristic and a target dynamic characteristic; Performing load parameter analysis on the servo controller based on the frequency response to obtain target load parameters, and performing performance change analysis on the servo controller to obtain target performance change parameters; Generating a target multidimensional state space based on the action space boundary, the target steady-state characteristic, the target dynamic characteristic, the target load parameter and the target performance variation parameter; constructing a deep reinforcement learning intelligent agent, and analyzing the adjustment quantity of the servo controller parameters by utilizing the target multidimensional state space based on the deep reinforcement learning intelligent agent; and updating the proportional-integral-derivative PID parameter of the servo controller based on the adjustment quantity. Optionally, the determining the action space boundary adjusted by the servo controller parameter based on the rigidity grade obtained from the rigidity grade of the set servo controller includes: acquiring a rigidity grade instruction set by a user based on an upper computer communication interface, and analyzing the rigidity grade instruction to acquire the rigidity grade of the servo controller; acquiring a rigidity grade-PID parameter adjustment range mapping table, and determining a parameter boundary by utilizing the rigidity grade based on the rigidity grade-PID parameter adjustment range mapping table; And determining an action space boundary of servo controller parameter adjustment based on the parameter boundary. Optionally, the performing feature analysis on the motor corresponding to the servo controller to obtain a target steady-state feature and a target dynamic feature includes: Collecting steady-state data in a steady-state application scene of a motor corresponding to a servo controller, and performing feature analysis on the steady-state data to obtain target steady-state features; and acquiring dynamic data in a dynamic application scene of the motor corresponding to the servo controller, and performing feature analysis on the dynamic data to obtain target dynamic features. Optionally, the performing load parameter analysis on the servo controller based on the frequency response to obtain a target load parameter, performing performance change analysis on the servo controller to obtain a target performance change parameter, including: Injecting a first input signal into a motor corresponding to a servo controller, and collecting a first output signal output by the motor under the first input signal; determining frequency response data using a fourier transform based on the first input signal and the