CN-122009640-A - Automatic labeling system and method for robot in complex industrial environment

CN122009640ACN 122009640 ACN122009640 ACN 122009640ACN-122009640-A

Abstract

The invention belongs to the field of robot automation control, and discloses an automatic labeling system and method for a robot applied to a complex industrial environment. The system comprises a labeling executor module, a mechanical arm module, a visual perception module and a control module, wherein the visual perception module adopts a depth camera for collecting a plurality of groups of checkerboard calibration plate images and collecting target object images, and the depth camera is integrated with an internal reference calibration and error correction unit, a target identification unit and a pose resolving unit. The system combines the internal reference calibration and the error correction unit to complete the camera internal reference and the hand-eye calibration to establish a high-precision coordinate system mapping relation, adopts the preprocessing subunit and the positioning subunit to calculate the precise three-dimensional pose of the labeling position through the visual perception module, and integrates the mixed visual servo control unit through the control module to ensure that the mechanical arm moves stably and accurately, so that the labeling efficiency is greatly improved manually, and the system has high-precision positioning performance.

Inventors

ZHANG ZIJUN
ZHAO WENHAO
ZHAO XINGWEI
LIU MOYUN
LI ZHIRUN
TAO BO

Assignees

华中科技大学

Dates

Publication Date: 20260512
Application Date: 20260331

Claims (10)

1. A robotic automatic labeling system for use in a complex industrial environment, comprising: A labeling executor module; the mechanical arm module comprises a six-axis cooperative mechanical arm, and the tail end of the six-axis cooperative mechanical arm is connected with the labeling executor module; The visual perception module adopts a depth camera and is connected to the tail end of the six-axis cooperative mechanical arm; the depth camera is integrated with an internal reference calibration and error correction unit, a target identification unit and a pose resolving unit; the depth camera is used for collecting a plurality of groups of checkerboard calibration plate images and target object images, and calculating the output of the target recognition unit based on the depth value and the back projection of camera internal parameters to obtain three-dimensional point cloud data of the target object images; the target recognition unit comprises a preprocessing subunit and a positioning subunit, wherein the preprocessing subunit is used for preprocessing the corrected target image through Gaussian filtering noise reduction and HSV color enhancement, the positioning subunit is used for carrying out area recognition on the preprocessed target image through a U 2 -Net semantic segmentation network and then detecting and determining a boundary through a Canny operator edge; The control module is respectively connected with other modules and integrates a hand-eye matrix conversion unit and a mixed vision servo control unit, the hand-eye matrix conversion unit is used for converting the output of the pose resolving unit into a conversion matrix between the coordinate system of the labeling executor module and the coordinate system of the target object, and the mixed vision servo control unit is used for fusing the output of the hand-eye matrix conversion unit with a PID parameter self-setting algorithm and Kalman filtering processing and outputting a movement speed instruction of the six-axis cooperative mechanical arm.
2. The automated robot labeling system for use in complex industrial environments of claim 1, wherein the gaussian filtering is implemented by the following formula: ; Wherein, the Expressed as the current pixel point X and y are respectively expressed as the horizontal and vertical distances from the pixel point of the corrected target object image to the center of the convolution kernel; Expressed as standard deviation, 2 is taken.
3. The automatic labeling system for robots in complex industrial environments according to claim 1, wherein the boundary is determined by Canny operator edge detection, specifically, gradient strength is calculated from the target object image output by the U2-Net semantic segmentation network by the following formula Then comparing with the determined high and low threshold values T high and T low , if M > T high , judging that the pixel belongs to the boundary and needs to be reserved, if M < T low , judging that the pixel does not belong to the boundary and does not remain, if T low <M<T high , judging that whether reserved pixels exist around the pixel, and if reserved pixels exist around the pixel, judging that the pixel still needs to be reserved; ; Wherein, the The gradient intensity expressed as the current pixel point is 180 at T high and 100 at T low ; Represented as the partial derivative of the pixel value in the horizontal direction; represented as the partial derivative of the pixel value in the vertical direction.
4. The automatic labeling system of the robot applied to the complex industrial environment according to claim 1, wherein the six-axis cooperative mechanical arm is further integrated with a joint angle feedback unit and a singular solution detection module, the joint angle feedback unit is used for collecting angle information of each joint in the six-axis cooperative mechanical arm, and the singular solution detection module is used for constructing a closed-loop correction mechanism according to the angle information of each joint collected by the six-axis cooperative mechanical arm and the control module.
5. The automatic labeling system of the robot applied to the complex industrial environment, which is disclosed in claim 1, is characterized in that the depth camera is used for collecting a plurality of groups of checkerboard calibration plate images with different directions and distances, and the internal reference calibration and error correction unit is used for solving the checkerboard calibration plate images based on an OpenCV vision library according to a Zhang Zhengyou calibration method to obtain the internal reference matrix of the depth camera.
6. The automatic labeling system of the robot applied to the complex industrial environment according to claim 1, wherein the pose solving subunit is configured to perform voxel grid downsampling and statistical outlier rejection on the three-dimensional point cloud data corresponding to the optimized depth camera, then perform quadric surface fitting by using a least square method to obtain a normal vector at a center point of the cloth, and then construct a coordinate system by combining with an optimal straight line of a product edge, and output a three-dimensional pose of a target image under the depth camera coordinate system.
7. The automatic labeling system of a robot applied in a complex industrial environment according to claim 1, wherein the control modules are respectively connected with other modules through heterogeneous communication links.
8. The automatic labeling system of the robot applied to the complex industrial environment according to claim 1, wherein the labeling actuator module comprises a push rod, a vacuum chuck and a label adsorption platform, wherein the tail end of the push rod is connected with the vacuum chuck through a spring, and the label adsorption platform is provided with a linear supporting round corner and a waste base paper recycling cylinder.
9. The automated labeling system of claim 1 for use in complex industrial environments, wherein the Ubuntu-based operating system is implemented.
10. The automatic labeling method of the robot applied to the complex industrial environment is characterized by comprising the following steps of: s1, collecting a plurality of groups of checkerboard calibration plate images and target object images; S2, performing internal reference calibration on the depth camera by utilizing the checkerboard calibration plate image, and then performing image correction on the target object image by combining a dynamic distortion correction algorithm; S3, preprocessing the target object image corrected in the S2 through Gaussian filtering noise reduction and HSV color enhancement, carrying out region identification on the preprocessed target object image through a U 2 -Net semantic segmentation network, and then determining a boundary through Canny operator edge detection; s4, obtaining three-dimensional point cloud data of the target object image based on the depth value and back projection calculation of the camera internal reference according to the output of the S3; s5, resolving the three-dimensional point cloud data of the target object image to obtain a three-dimensional pose of the target image under the depth camera coordinate system, updating the process noise covariance and the observation noise covariance of the three-dimensional pose of the target image through Kalman filtering, and outputting the three-dimensional pose of the target image after optimization; S6, converting the output of the S5 into a conversion matrix between a labeling executor module coordinate system and a target object coordinate system, fusing a PID parameter self-tuning algorithm and Kalman filtering processing, and outputting a movement speed instruction of the six-axis cooperative mechanical arm. And S7, labeling according to the movement speed instruction of the six-axis cooperative mechanical arm.

Description

Automatic labeling system and method for robot in complex industrial environment Technical Field The invention belongs to the technical field of robot automation control and visual servo, and particularly relates to an automatic labeling system and method for a robot applied to a complex industrial environment. Background In an intelligent manufacturing production line, automatic labeling is a key link of product information identification and tracing. Currently, a robot automatic labeling system based on machine vision has become a mainstream, but generally adopts a two-stage visual paradigm of 'coarse positioning before fine correction', has inherent defects, and restricts further improvement of precision and stability. Specifically, the mainstream technical process is generally that a general target detection network such as YOLO and fast R-CNN is adopted to identify a product, and a rectangular bounding box containing the product and a background is output. The bounding box is positioned roughly, cannot provide an accurate product-background demarcation, and its coordinates are susceptible to illumination and occlusion to produce jitter. Then, within the bounding box region, geometric features for pose calculation are extracted by relying on traditional image processing algorithms (such as Canny edge detection and Hough transformation). The method is extremely dependent on image quality, is extremely sensitive to interference such as uneven illumination, product reflection, background texture and the like, is unstable in feature extraction, and often causes edge breakage or false detection, so that the finally calculated three-dimensional pose has larger errors and fluctuation, and cannot meet the requirements of sub-millimeter labeling precision. In addition, the existing scheme generally enables the vision module and the control module to be relatively independent, the vision result is used as a disposable guiding signal, the real-time and high-frequency closed loop feedback capability in the movement process of the robot is lacked, and the positioning drift of the robot or the tiny movement of a workpiece cannot be compensated. The method has the following core pain points that the visual perception precision is insufficient, stable and accurate pixel level outline information cannot be provided by relying on the manual characteristics of a regional level detection frame and fragile, the system robustness is poor, the traditional image processing algorithm is sensitive to environmental changes, the stability is poor in a complex industrial scene, the control closed loop bandwidth is low, the visual perception is not deeply integrated into a real-time control loop, and dynamic correction and high-precision servo are difficult to realize. Therefore, a set of cooperative control system needs to be constructed, and the core problems of accurate positioning, stable movement, reliable attachment and the like in automatic labeling in a complex industrial scene are systematically solved. Disclosure of Invention Aiming at the defects or improvement demands of the prior art, the invention provides an automatic labeling system and method for a robot, which are applied to a complex industrial environment, and solve the technical defects of insufficient positioning precision, weak environment anti-interference capability, poor suitability in the complex industrial environment caused by poor control closed loop instantaneity and the like commonly existing in the existing automatic labeling system for the robot based on machine vision. To achieve the above object, according to one aspect of the present invention, there is provided a robot automatic labeling system applied to a complex industrial environment, comprising: A labeling executor module; the mechanical arm module comprises a six-axis cooperative mechanical arm, and the tail end of the six-axis cooperative mechanical arm is connected with the labeling executor module; The visual perception module adopts a depth camera and is connected to the tail end of the six-axis cooperative mechanical arm; the depth camera is integrated with an internal reference calibration and error correction unit, a target identification unit and a pose resolving unit; the depth camera is used for collecting a plurality of groups of checkerboard calibration plate images and target object images, and calculating the output of the target recognition unit based on the depth value and the back projection of camera internal parameters to obtain three-dimensional point cloud data of the target object images; the target recognition unit comprises a preprocessing subunit and a positioning subunit, wherein the preprocessing subunit is used for preprocessing the corrected target image through Gaussian filtering noise reduction and HSV color enhancement, the positioning subunit is used for carrying out area recognition on the preprocessed target image through a U 2 -Net semantic segmentation