CN-116494239-B - Soft manipulator control method and device, soft manipulator and electronic equipment

CN116494239BCN 116494239 BCN116494239 BCN 116494239BCN-116494239-B

Abstract

The invention relates to a soft manipulator control method, a device, a soft manipulator and electronic equipment, wherein the method comprises the following steps: and receiving first data and a first image, wherein the first data are corresponding pressure sensor data and IMU sensor data on the manipulator, and the first image is image data measured by an image sensor inside the manipulator. And processing the first data to obtain second data, wherein the second data is load distribution data corresponding to the pressure sensor data and vibration errors corresponding to the IMU sensor data. And identifying the first image after the image edge detection processing through a convolutional neural network so as to acquire volume distribution data of different positions inside the manipulator. And calling a deep reinforcement learning algorithm to analyze and process the second data and the volume distribution data so as to obtain a first control parameter, wherein the first control parameter is a PID control parameter and is used for controlling the air inlet and outlet quantity of an air pump connected with the manipulator, so that the fragile object can be reliably and stably grabbed.

Inventors

ZHANG JICHEN
ZHANG XI
ZHANG YU
Li Xiaojue
LI JUNYAN

Assignees

深圳大学

Dates

Publication Date: 20260505
Application Date: 20230523

Claims (10)

1. A method of controlling a soft manipulator, the method comprising: Receiving first data and a first image, wherein the first data are corresponding pressure sensor data and IMU sensor data on a manipulator, and the first image is image data measured by an image sensor inside the manipulator; Processing the first data to obtain second data, wherein the second data is load distribution data corresponding to the pressure sensor data and vibration errors corresponding to the IMU sensor data; identifying the first image after the image edge detection processing through a convolutional neural network so as to acquire volume distribution data of different positions inside the manipulator; And calling a deep reinforcement learning algorithm to analyze and process the second data and the volume distribution data so as to obtain a first control parameter, wherein the first control parameter is a PID control parameter and is used for controlling the air inlet and outlet quantity of an air pump connected with the manipulator.
2. The soft manipulator control method of claim 1, wherein the processing the first data to obtain second data further comprises: acquiring a first vibration signal based on the IMU sensor data, wherein the first vibration signal is a vibration signal of a gripped object gripped by the manipulator; And acquiring a loss function corresponding to the vibration error based on the first vibration signal, wherein the loss function is used for describing the inhibition effect of the first vibration signal.
3. The soft manipulator control method of claim 2, further comprising: acquiring a first state of the gripped object based on the first data and the first image, wherein the first state is the current state of the manipulator; When the object to be grasped is in a first state and the value of the corresponding loss function exceeds a first threshold value, the convolution neural network is called to fit the first state so as to obtain a first behavior, wherein the first behavior is the optimal behavior when the loss function of the manipulator in the first state exceeds the first threshold value; Wherein the first behavior is used to adjust the first control parameter.
4. The soft manipulator control method of claim 3, further comprising: when the object to be grasped is in a first state and the value of the corresponding loss function is lower than a second threshold value, the convolution neural network is called to fit the first state so as to acquire a second behavior, wherein the second behavior is the best behavior of the manipulator when the loss function is lower than the second threshold value in the first state; Wherein the second behavior is for tuning down the first control parameter.
5. The method according to claim 1, wherein the step of identifying the first image after the image edge detection process by using the convolutional neural network to obtain the volume distribution data of different positions inside the manipulator further comprises: acquiring image data before the grasping movement of a manipulator and image data after the grasping movement; And performing edge detection on the image data before the gripping movement of the manipulator and the image data after the gripping movement so as to acquire an image to be identified after the edge detection.
6. The method according to claim 5, wherein the identifying the first image after the image edge detection processing by the convolutional neural network to obtain the volume distribution data of different positions inside the manipulator comprises: And calling the convolutional neural network to identify the image to be identified so as to acquire volume distribution data before and after the grasping movement of the manipulator.
7. The method of claim 1, wherein invoking the deep reinforcement learning algorithm performs an analysis process on the second data and the volume distribution data, and then comprises: Invoking a PID algorithm to calculate an analysis processing result of the deep reinforcement learning algorithm so as to acquire pulse width modulation parameters; and sending the pulse width modulation parameters to a lower computer connected with the mechanical flashlight through a transmission control protocol so as to control the manipulator to carry out gripping movement.
8. A soft manipulator control device, the device comprising: the data receiving module is used for receiving first data and first images, the first data are corresponding pressure sensor data and IMU sensor data on the manipulator, and the first images are image data measured by an image sensor inside the manipulator; The first processing module is used for processing the first data to obtain second data, wherein the second data is load distribution data corresponding to the pressure sensor data and vibration errors corresponding to the IMU sensor data; The image recognition module is used for recognizing the first image after the image edge detection processing through the convolutional neural network so as to acquire volume distribution data of different positions inside the manipulator; And the second processing module is used for calling a deep reinforcement learning algorithm to analyze and process the second data and the volume distribution data so as to obtain a first control parameter, wherein the first control parameter is a PID control parameter and is used for controlling the air outlet quantity of an air pump connected with the manipulator.
9. A soft manipulator controlled by the soft manipulator control method of any one of claims 1 to 7, comprising: The manipulator body consists of soft fingers, a supporting structure and a pneumatic driving tube, wherein the supporting structure is used for supporting and fixing a plurality of soft fingers, the pneumatic driving tube is connected with the soft fingers, and the soft fingers are controlled to straighten and bend through air inlet and air outlet; the lower computer is electrically connected with the manipulator body and is used for receiving sensor data from the manipulator body and directly controlling the grasping movement of the manipulator body by controlling the air inlet and the air outlet of the pneumatic driving pipe; The upper computer is electrically connected with the lower computer and is used for receiving sensor data from the lower computer through a transmission control protocol and processing the sensor data so as to acquire control parameters for controlling the grasping movement of the manipulator body and sending the control parameters to the lower computer through the transmission control protocol.
10. An electronic device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any one of claims 1 to 7 when the computer program is executed.

Description

Soft manipulator control method and device, soft manipulator and electronic equipment Technical Field The present invention relates to the field of manipulator control technologies, and in particular, to a method and an apparatus for controlling a soft manipulator, and an electronic device. Background The manipulator can simulate certain action functions of hands and arms, is used for grabbing and carrying objects or operating tools according to fixed procedures, and is a novel device developed in the mechanized and automatic production process. In the modern production process, the manipulator is widely applied to an automatic production line, development and production of the manipulator become an emerging technology which is rapidly developed in the high technical field, and development of the manipulator is promoted, so that the manipulator can be better organically combined with mechanization and automation. Although the manipulator is not as flexible as a human hand, the manipulator has the characteristics of repeated work and labor, no fatigue, danger resistance and larger force for grabbing and lifting the weight than a human hand. Therefore, robots have been paid attention to many departments and are increasingly used. Currently, most of commonly used manipulators are rigid manipulators, and functions and forms of conventional rigid manipulators have been widely used in human social production. These rigid manipulators are mainly made of metallic materials and are often responsible for structural, repetitive and other working procedures in industrial sites. However, modern manipulators are evolving towards high positioning accuracy, high flexibility and high man-machine interaction. Because the rigid manipulator itself is too rigid in material, it is difficult to achieve reliable gripping for some delicate or fragile items. Therefore, it is difficult for conventional rigid manipulators to achieve reliable and stable gripping of fragile objects. Disclosure of Invention In view of the above, it is necessary to provide a method and apparatus for controlling a soft manipulator, and an electronic device, which can reliably and stably grasp a fragile object. The invention provides a soft manipulator control method, which comprises the following steps: Receiving first data and a first image, wherein the first data are corresponding pressure sensor data and IMU sensor data on a manipulator, and the first image is image data measured by an image sensor inside the manipulator; Processing the first data to obtain second data, wherein the second data is load distribution data corresponding to the pressure sensor data and vibration errors corresponding to the IMU sensor data; identifying the first image after the image edge detection processing through a convolutional neural network so as to acquire volume distribution data of different positions inside the manipulator; And calling a deep reinforcement learning algorithm to analyze and process the second data and the volume distribution data so as to obtain a first control parameter, wherein the first control parameter is a PID control parameter and is used for controlling the air outlet quantity of an air pump connected with the manipulator. In one embodiment, the processing the first data to obtain second data further includes: acquiring a first vibration signal based on the IMU sensor data, wherein the first vibration signal is a vibration signal of a gripped object gripped by the manipulator; and acquiring a loss function corresponding to the vibration error based on the first vibration signal, wherein the loss function is used for describing the inhibition effect of the first vibration effect. In one embodiment, the method further comprises: acquiring a first state of the gripped object based on the first data and the first image, wherein the first state is the current state of the manipulator; When the object to be grasped is in a first state and the value of the corresponding loss function exceeds a first threshold value, the convolution neural network is called to fit the first state so as to obtain a first behavior, wherein the first behavior is the optimal behavior when the loss function of the manipulator in the first state exceeds the first threshold value; Wherein the first behavior is used to adjust the first control parameter. In one embodiment, the method further comprises: when the object to be grasped is in a first state and the value of the corresponding loss function is lower than a second threshold value, the convolution neural network is called to fit the first state so as to acquire a second behavior, wherein the second behavior is the best behavior of the manipulator when the loss function is lower than the second threshold value in the first state; Wherein the second behavior is for tuning down the first control parameter. In one embodiment, the identifying, by using a convolutional neural network, the first image after the image edge d