CN-121989258-A - Visual feedback-based mechanical arm self-adaptive hierarchical grabbing control method and system

CN121989258ACN 121989258 ACN121989258 ACN 121989258ACN-121989258-A

Abstract

The invention provides a visual feedback-based mechanical arm self-adaptive hierarchical grabbing control method and system, which belong to the technical field of robot control and comprise the steps of acquiring target position information through a visual sensor, acquiring contact state information of the tail end of a mechanical arm and a target through a force sensor, judging a current control stage of the mechanical arm according to the target position information and the contact state information, wherein the control stage comprises a visual leading stage of the mechanical arm approaching the target but not contacting the target, a force leading stage of the mechanical arm grabbing the target and a transition stage from the visual leading stage to the force leading stage, respectively generating a first control component and a second control component based on the target position information, generating a third control component based on the contact state information, dynamically distributing weights of the first control component, the second control component and the third control component according to the current control stage of the mechanical arm, and fusing to generate a comprehensive control instruction, and driving the mechanical arm to grab the target according to the comprehensive control instruction, so that jump of the control instruction is avoided.

Inventors

LI JINHAN
PAN TIANHONG
TIAN JIAQIANG
FAN YUAN
HUANG XUNDE
DAI XUELEI
HUANG YU
NI LIPING
LAO LI

Assignees

安徽大学

Dates

Publication Date: 20260508
Application Date: 20260408

Claims (10)

1. The mechanical arm self-adaptive hierarchical grabbing control method based on visual feedback is characterized by comprising the following steps of: acquiring target position information through a visual sensor, and acquiring contact state information of the tail end of the mechanical arm and a target through a force sensor; Judging a current control stage of the mechanical arm according to the target position information and the contact state information, wherein the control stage comprises a vision leading stage of the mechanical arm approaching the target but not contacting the target, a force leading stage of the mechanical arm grabbing the target and a transition stage from the vision leading stage to the force leading stage; according to the current control stage of the mechanical arm, dynamically distributing weights of the first control component, the second control component and the third control component, and fusing to generate a comprehensive control instruction; And driving the mechanical arm to grab the target according to the comprehensive control instruction.
2. The method for adaptively classifying grabbing control of a mechanical arm based on visual feedback according to claim 1, wherein the method for judging the current control stage of the mechanical arm according to the target position information and the contact state information is as follows: according to the target position information, calculating the distance between the target and the mechanical arm, and according to the contact state information, calculating the contact force between the tail end of the mechanical arm and the target; when the distance between the target and the mechanical arm is larger than or equal to a distance threshold value, the current control stage of the mechanical arm is a vision leading stage, when the distance between the target and the mechanical arm is smaller than the distance threshold value and the contact force exceeds a first acting force threshold value, the current control stage of the mechanical arm enters a transition stage from the vision leading stage, and when the contact force exceeds a second acting force threshold value, the current control stage of the mechanical arm is a force leading stage, and the second acting force threshold value is larger than the first acting force threshold value.
3. The method for adaptively grading a gripper control of a robot arm based on visual feedback as set forth in claim 2, wherein the generating of the first control component and the second control component based on the target position information and the generating of the third control component based on the contact state information comprises: the method comprises the steps of generating a first control component based on the distance between a target and a mechanical arm and the deviation of the target distance by adopting an image-based visual servo method, converting target position information into a space target position, generating a second control component by means of inverse kinematics solution, and generating a third control component by converting the deviation of the contact force between the tail end of the mechanical arm and the target and the deviation of the target acting force into a position correction quantity by adopting an impedance control method.
4. The method for adaptively grading grabbing control of a mechanical arm based on visual feedback according to claim 1, wherein the process of dynamically distributing weights of the first control component, the second control component and the third control component according to the current control stage of the mechanical arm comprises the following steps: When the current control stage of the mechanical arm is a vision leading stage, the weight of the third control component is 0, and the weight of the first control component Weights of the second control component Is kept constant and is made to be a constant, And is also provided with ; When the current control stage of the mechanical arm is a transition stage, the weight of the first control component From the weight Gradually decrease to weight Weights of the third control component Gradually increasing from 0 to weight Weights of the second control component The method meets the following conditions: , In order to enter the time after the transition phase, , The total duration of the transition stage; When the current control stage of the mechanical arm is a force leading stage, the weight of the first control component Weights of the second control component Weights of third control component Is kept constant and is made to be a constant, And is also provided with 。
5. The method for adaptively grading grabbing control of a mechanical arm based on visual feedback as set forth in claim 4, wherein the weight of the first control component Weights of the second control component The following relationships are satisfied: , Weight of Weight of The method meets the following conditions: , 。
6. the method for adaptively grading grabbing control of a mechanical arm based on visual feedback as set forth in claim 4, wherein the weight of the first control component From the weight Gradually reducing to weight according to linear or exponential relation Weights of the third control component Gradually increasing from 0 to weight in a linear or exponential relationship 。
7. The method for adaptively classifying and grabbing a manipulator based on visual feedback as set forth in claim 4, wherein the command is comprehensively controlled when the current control stage of the manipulator is a vision dominant stage The method comprises the following steps: When the current control stage of the mechanical arm is a transition stage, the control instruction is comprehensively controlled The method comprises the following steps: when the current control stage of the mechanical arm is a force leading stage, the command is comprehensively controlled The method comprises the following steps: Wherein, the 、、 The first control component, the second control component and the third control component are respectively.
8. The mechanical arm self-adaptive hierarchical grabbing control system based on visual feedback is characterized by comprising the following components: The sensing module is used for acquiring target position information through a visual sensor and acquiring contact state information of the tail end of the mechanical arm and a target through a force sensor; The decision module is used for judging the current control stage of the mechanical arm according to the target position information and the contact state information, wherein the control stage comprises a vision leading stage of the mechanical arm approaching the target but not contacting the target, a force leading stage of the mechanical arm grabbing the target and a transition stage from the vision leading stage to the force leading stage; The fusion control module is used for respectively generating a first control component and a second control component based on the target position information and generating a third control component based on the contact state information, dynamically distributing weights of the first control component, the second control component and the third control component according to the current control stage of the mechanical arm, and fusing to generate a comprehensive control instruction; And the execution module is used for driving the mechanical arm to grab the target according to the comprehensive control instruction.
9. The visual feedback-based adaptive hierarchical grabbing control system of claim 8, wherein the fusion control module comprises: A visual servo controller for generating a first control component based on a distance between a target and the mechanical arm and a deviation of the target distance by adopting an image-based visual servo method; The position controller is used for converting the target position information into a space target position and generating a second control component through inverse kinematics solution; and the force controller is used for converting the deviation of the contact force between the tail end of the mechanical arm and the target acting force into a position correction amount by adopting an impedance control method to generate a third control component.
10. The visual feedback-based mechanical arm self-adaptive hierarchical grabbing control system of claim 8, wherein the execution module comprises a servo driver, a motor, a mechanical arm and joints positioned on the mechanical arm, the servo driver receives the comprehensive control instruction and converts the comprehensive control instruction into a current instruction to be input into the motor, and torque is transmitted through a joint speed reducer to drive each joint to rotate so as to drive the tail end of the mechanical arm to grab a target.

Description

Visual feedback-based mechanical arm self-adaptive hierarchical grabbing control method and system Technical Field The invention relates to the technical field of robot control, in particular to a method and a system for controlling self-adaptive hierarchical grabbing of a mechanical arm based on visual feedback. Background The mechanical arm grabbing control is one of core technologies for realizing automatic operation. In practical application, the grabbing control needs to comprehensively utilize various sensor information, namely, a visual sensor can provide position information of a target and is used for guiding a mechanical arm to approach the target, and a force sensor can provide contact state information and is used for realizing flexible grabbing. How to effectively fuse the two heterogeneous information is a key technical problem for realizing intelligent grabbing. The prior art mainly adopts the following two ideas to solve the problems: (1) The serial use mode is that the thought adopts a serial mode of firstly using vision and then controlling force, namely firstly using a vision guiding mechanical arm to approach a target, and switching to the force control mode after approaching. The serial mode has the obvious technical defect that when switching moments exist between the two control modes, control instructions jump, so that the mechanical arm moves unstably and even generates impact vibration. The different control modes are regarded as mutually exclusive choices or combinations thereof, rather than as cooperative relationships that can cooperatively and dynamically adjust the degree of contribution. This concept limits further improvement in control performance. (2) And in a parallel use mode, the thought tries to use the visual information and the force information for control decision simultaneously, and generates a comprehensive control instruction through a certain fusion algorithm. However, the visual information and the force information have different physical meanings, different dimensions and different updating frequencies, the ideal effect is difficult to achieve by direct fusion, the setting of fusion parameters is difficult, and the stability of the system is difficult to ensure. In the method, in the contact establishment stage, a contact event is detected through a touch sensor, coordinate system integration is carried out on fusion vision and touch data, vision-touch consistency score is calculated, initial grabbing force is applied, sliding detection is carried out by adopting wavelet packet energy entropy and pressure gradient in the stable stage, grabbing force and impedance parameters are dynamically adjusted by combining self-adaptive impedance control, and the whole process intelligent control from approaching and contacting to stable is realized; the method is characterized in that depth fusion of a perception layer is achieved, geometrical characteristics obtained through vision and pressure data obtained through touch sense are accurately registered through a complex algorithm, the problem that how the positions seen by eyes and the forces sensed by touch sense are aligned under the same coordinate system is solved, and in order to ensure perception accuracy, an expensive depth camera and a high-resolution touch sensor are needed, and calculation complexity is high. Furthermore, this approach to ensure reliability of tracking, visual information is typically given a high or even constant weight throughout the process, which may result in insufficient utilization of force sensor information during close contact phases to achieve optimal compliance control. Disclosure of Invention The invention aims to solve the technical problem of how to solve the problem of command jump when the mechanical arm is guided from vision main to force main in the grabbing process. The invention solves the technical problems by adopting the following technical scheme that the mechanical arm self-adaptive hierarchical grabbing control method based on visual feedback comprises the following steps: acquiring target position information through a visual sensor, and acquiring contact state information of the tail end of the mechanical arm and a target through a force sensor; Judging a current control stage of the mechanical arm according to the target position information and the contact state information, wherein the control stage comprises a vision leading stage of the mechanical arm approaching the target but not contacting the target, a force leading stage of the mechanical arm grabbing the target and a transition stage from the vision leading stage to the force leading stage; according to the current control stage of the mechanical arm, dynamically distributing weights of the first control component, the second control component and the third control component, and fusing to generate a comprehensive control instruction; And driving the mechanical arm to grab the target according to the