Search

CN-121995830-A - Grabbing control method and system for robot vision guidance

CN121995830ACN 121995830 ACN121995830 ACN 121995830ACN-121995830-A

Abstract

The application relates to the technical field of artificial intelligence and discloses a grabbing control method and a grabbing control system for robot vision guidance, wherein the method comprises the following steps of executing global approach rough positioning; the method comprises the steps of entering a preset distance threshold value, then lifting the frame rate, switching to a high-frequency visual servo mode, extracting feature point geometric vectors, calculating pixel residual errors, introducing Kalman filtering to smooth the residual errors, dynamically adjusting an observation noise covariance matrix according to a real-time image signal-to-noise ratio, mapping the smooth residual errors into six-degree-of-freedom speed compensation quantity by using an image jacobian matrix, synthesizing the compensation quantity and an original motion instruction vector, weighting by an S-shaped curve, normalizing and limiting, and then driving a joint actuator until the residual errors are converged. The system comprises a vision acquisition module, a feature extraction module, a residual calculation module, a vision servo controller and a motion execution module. According to the scheme, high-precision and strong-robustness dynamic grabbing is realized, and dependence on camera calibration and three-dimensional reconstruction is remarkably reduced.

Inventors

  • CAO CHENGCHENG
  • WANG XIAOFENG

Assignees

  • 成都锦城学院
  • 成都欧澎科技有限公司

Dates

Publication Date
20260508
Application Date
20260130

Claims (10)

  1. 1. The grabbing control method for the robot vision guidance is characterized by comprising the following steps of: the method comprises the following steps of (1) executing a global approach stage, namely acquiring an initial image of a target object through a vision acquisition device arranged on an end effector of a robot, performing coarse positioning, generating an original motion instruction moving towards a target area, and enabling the end effector to approach the target to a preset distance threshold range; After entering the preset distance threshold, the visual acquisition frame rate is increased to a preset high-frequency frame rate, the visual acquisition frame rate is switched to a high-frequency visual servo mode, continuous image streams are acquired in real time, and geometric feature vectors of at least 4 feature points are extracted from each frame of image; step (3) calculating pixel residual errors, namely comparing the currently extracted geometric feature vector with a pre-stored expected feature vector, and calculating pixel level deviation of the currently extracted geometric feature vector and the pre-stored expected feature vector in an image plane to form a multi-dimensional pixel residual error vector; Step (4) mapping a speed control vector, namely introducing Kalman filtering to smooth the multi-dimensional pixel residual vector, wherein an observed noise covariance matrix of the Kalman filtering is dynamically adjusted according to a real-time image signal-to-noise ratio, and the observed noise covariance matrix is in inverse mapping relation with the image signal-to-noise ratio, and then mapping the smoothed pixel residual into six-degree-of-freedom speed compensation quantity of a robot end effector in a Cartesian space in real time by utilizing an image jacobian matrix and pseudo-inverse thereof, wherein the six-degree-of-freedom speed compensation quantity comprises three linear speed components and three angular speed components; And (5) performing dynamic track correction, namely vector synthesis is performed on the speed compensation quantity and the original motion instruction, normalized amplitude limiting processing is performed on the synthesized joint angular speeds, and a composite control instruction is generated to drive a robot joint actuator to adjust the motion track until pixel residual errors are smaller than a set convergence threshold.
  2. 2. The method according to claim 1, wherein the vector synthesis process in step (5) weights the speed compensation amount by using an S-shaped curve function, the weight coefficient is linearly increased from 0 to 1 in a predetermined period after switching to the high-frequency visual servo mode, and the output speed is the sum of the original motion command and the weighted speed compensation amount.
  3. 3. The method for controlling grabbing of robot vision guidance according to claim 1, wherein in the step (4), when calculating the six-degree-of-freedom speed compensation amount, the pose is solved by adopting EPnP algorithm to obtain depth information of the feature points, and the depth value of the previous control period is used as the current iteration initial value to be input into the solver, when the feature points are coplanar and result in the reprojection error being greater than a preset pixel threshold value, the center reference depth is obtained by a laser displacement sensor integrated on the end effector, and the depth of each point is corrected by utilizing the center reference depth and the cosine value of the included angle of each feature point relative to the center.
  4. 4. The method for controlling the grabbing of the robot vision guidance according to claim 1, wherein in the step (2), if the number of feature points successfully matched by the current frame is less than 3, the end effector is controlled to execute a spiral line search track with a preset step length by taking the current coordinate as a center, and the vision module performs template matching at a preset high-frequency frame rate, and when the matching score exceeds a preset confidence threshold, the target center is locked and the state covariance matrix of the kalman filter is reset.
  5. 5. The method for controlling the grabbing of the robot vision guidance according to claim 1, wherein in the step (4), when the six-degree-of-freedom speed compensation amount is mapped, the gain coefficient is set within a preset interval, when the euclidean norm of the pixel residual is greater than a first preset pixel threshold value, the gain coefficient is set to a first preset value, and when the euclidean norm of the pixel residual is less than or equal to the first preset pixel threshold value, the gain coefficient is set to a second preset value.
  6. 6. The method for controlling the grabbing of the robot vision guidance according to claim 1, wherein the observed noise covariance matrix in the step (4) is a diagonal matrix, diagonal elements of the diagonal matrix are equal to a preset constant plus a value of a negative multiple of the natural exponential function image signal to noise ratio, and when the local gray variance is lower than a preset gray variance threshold, the image signal to noise ratio is judged to be in a low signal to noise ratio state, and the value of the observed noise covariance matrix is increased to a preset upper limit value.
  7. 7. The method for controlling grasping of a robot vision guide according to claim 1, wherein the normalization limiting processing in the step (5) comprises the specific steps of judging whether the synthesized angular velocities of the joints exceed a rated maximum value, and if the maximum value of the absolute values of the angular velocities of the joints exceeds the rated maximum value, scaling down all the joint velocities in the same proportion, wherein the scaling factor is the ratio of the rated maximum value to the maximum value of the absolute values of the angular velocities of the joints.
  8. 8. A robotic vision-guided grip control system, comprising: the visual acquisition module is used for acquiring an initial image of a target object at a first frame rate in a global approach stage, and acquiring a continuous image stream at a second frame rate after entering a preset distance threshold; The feature extraction module is used for extracting geometric feature vectors of at least 4 feature points from each frame of image; the residual calculation module is used for comparing the current geometric feature vector with a pre-stored expected feature vector to generate a multi-dimensional pixel residual vector; The visual servo controller is used for carrying out Kalman filtering smoothing on the multi-dimensional pixel residual vector, wherein an observation noise covariance matrix is dynamically adjusted according to a real-time image signal-to-noise ratio and is in inverse mapping relation with the image signal-to-noise ratio, and the smoothed pixel residual is mapped into six-degree-of-freedom speed compensation quantity by utilizing an image jacobian matrix and pseudo-inverse thereof; And the motion execution module is used for synthesizing the speed compensation quantity and the original motion instruction vector, executing normalized amplitude limiting processing on the synthesized joint angular speeds, and generating a composite control instruction to drive a robot joint executor to adjust a motion track until the pixel residual error is smaller than a set convergence threshold value.
  9. 9. The robot vision-guided gripping control system of claim 8, wherein the vision acquisition module and the laser displacement sensor are integrated together on the robot end effector to form a hand-eye system, and the laser displacement sensor is used for providing a center reference depth when feature points are coplanar to cause pose solving failure so as to correct depth information of each feature point.
  10. 10. The robot vision-guided gripping control system of claim 8, wherein the vision servo controller employs a first gain factor when the euclidean norm of the pixel residual is greater than a first preset pixel threshold and a second gain factor when the euclidean norm of the pixel residual is less than or equal to the first preset pixel threshold, and wherein the motion execution module weights the speed compensation amount using an S-shaped curve function for a predetermined period of time after the mode switching, and the weight factor increases linearly from 0 to 1.

Description

Grabbing control method and system for robot vision guidance Technical Field The invention belongs to the field of artificial intelligence, and particularly relates to a grabbing control method and system for robot vision guidance. Background Along with the continuous improvement of the industrial automation level, the visual guiding grabbing technology directly determines the operation precision and the environment adaptability of the robot. In typical scenes such as intelligent manufacturing, logistics sorting and flexible assembly, a robot needs to position and grasp a target object in an unstructured or semi-structured environment with high precision, which puts a very high requirement on the coordination capability of visual perception and motion control. The grabbing control method based on visual servo becomes a key technical path for improving the robustness of the system due to the closed loop feedback characteristic of the grabbing control method. The method generally constructs a control signal through the deviation between the real-time image characteristics and the expected target, and drives the robot end effector to approach the target pose. However, the conventional visual servo scheme still faces multiple technical bottlenecks in actual deployment, namely, most systems depend on high-precision camera-robot hand-eye calibration parameters, calibration errors or external parameter drift can directly lead to three-dimensional pose resolving misalignment to further cause grabbing failure, secondly, the conventional scheme generally adopts an open-loop coarse positioning superposition static image processing mode, when a target is subjected to micro dynamic displacement, the conventional visual servo scheme cannot respond timely to cause closed loop failure, and thirdly, the conventional visual servo scheme is sensitive to image noise, characteristic jump is easily generated under complex working conditions such as illumination mutation, reflection or partial shielding to cause severe oscillation of control instructions, and finally, a main stream method lacks a smooth transition mechanism in a mode switching (such as global approach to fine servo) process, mechanical jitter is often caused by speed instruction mutation to influence system stability and service life. The problems limit the large-scale reliable application of the vision-guided gripping system in the real industrial field. Aiming at the core defects of strong calibration dependence, weak dynamic response capability, missing noise suppression mechanism, discontinuous control instruction and the like in the prior art, a novel grabbing control scheme integrating self-adaptive filtering, pixel-level closed-loop mapping and dynamic track synthesis is needed to realize the unification of low calibration dependence, high noise immunity and smooth response. Disclosure of Invention The invention aims to provide a grabbing control method and a grabbing control system for robot vision guidance, which can effectively solve the problems in the background technology. In order to achieve the above purpose, the technical scheme adopted by the invention is as follows: A grabbing control method for robot vision guidance comprises the following specific steps: The global approach stage is executed, namely an initial image of a target object is acquired through a visual acquisition device, coarse positioning is carried out, an original motion instruction moving to a target area is generated, and an end effector approaches to the target to be within a preset distance threshold range; After entering the preset distance threshold, the visual acquisition frame rate is increased to a preset high-frequency frame rate, the visual acquisition frame rate is switched to a high-frequency visual servo mode, continuous image streams are acquired in real time, and geometric feature vectors of at least 4 feature points are extracted from each frame of image; step (3) calculating pixel residual errors, namely comparing the currently extracted geometric feature vector with a pre-stored expected feature vector, and calculating pixel level deviation of the currently extracted geometric feature vector and the pre-stored expected feature vector in an image plane to form a multi-dimensional pixel residual error vector; Step (4) mapping a speed control vector, namely introducing Kalman filtering to smooth the multi-dimensional pixel residual vector, wherein an observed noise covariance matrix of the Kalman filtering is dynamically adjusted according to a real-time image signal-to-noise ratio, and the observed noise covariance matrix is in inverse mapping relation with the image signal-to-noise ratio, and then mapping the smoothed pixel residual into six-degree-of-freedom speed compensation quantity of a robot end effector in a Cartesian space in real time by utilizing an image jacobian matrix and pseudo-inverse thereof, wherein the six-degree-of-freedom speed compensation