CN-121973161-A - 3D vision-based intelligent grabbing system anomaly detection and processing method and device

CN121973161ACN 121973161 ACN121973161 ACN 121973161ACN-121973161-A

Abstract

The invention discloses an intelligent grabbing system abnormality detection and processing method and device based on 3D vision, and belongs to the technical field of industrial robot 3D vision guiding grabbing. The method comprises the steps of collecting 3D point cloud data of stacked flexible plates, carrying out point cloud preprocessing and template matching, determining the positions and the positions of the plates and grabbing points, generating a cushion block detection area and a deformation detection area based on the positions and the positions of the grabbing points, counting the number of the point clouds in the detection area and comparing the number with a threshold value, judging whether cushion block residues or excessive deformation of the plates exist, triggering the processing if the cushion block residues or the excessive deformation of the plates are abnormal, and outputting grabbing instructions if the cushion block residues or the excessive deformation of the plates are normal. The device comprises a truss manipulator, an end effector, a 3D vision sensor, a vision processing unit and a robot control unit. According to the invention, the self-adaptive detection and treatment of excessive deformation of the cushion blocks and the plates are realized by dynamically generating the detection areas and fusing the point cloud threshold value judgment, so that the robustness, the safety and the automation level of the grabbing system are obviously improved, and the method is suitable for automatic unstacking operation of stacking flexible plates.

Inventors

ZHOU TIANLEI
GUO ZICHEN
ZHANG ZHONGXIN
CAO QIANG
WU LEQUN
LI ZHANGMIN
LI YANXIANG
ZHAO JINGYONG
ZHANG ZHEN

Assignees

徐州徐工道金特种机器人技术有限公司

Dates

Publication Date: 20260505
Application Date: 20260330

Claims (10)

1. The method for detecting and processing the abnormality of the intelligent grabbing system based on the 3D vision is characterized by being applied to grabbing of stacked flexible plates and comprising the following steps of: s1, acquiring original 3D point cloud data of a target grabbing area of stacked flexible plates through a 3D vision sensor; s2, preprocessing the original 3D point cloud data to obtain processed point cloud data; s3, matching the preprocessed point cloud data with a preset flexible board 3D template, and determining the six-degree-of-freedom pose and the grabbing point pose of the flexible board to be grabbed currently; S4, dynamically generating a three-dimensional detection area for anomaly detection based on the grabbing point pose; S5, counting point cloud characteristics in the three-dimensional detection area, comparing the counting result with a preset threshold value, and judging whether abnormality exists or not; and S6, triggering an exception handling mechanism if the exception exists, and outputting the grabbing point pose to the robot to execute grabbing if the exception does not exist.
2. The method according to claim 1, wherein in the step S4, the three-dimensional detection area includes a pad detection area for detecting a residual pad and a deformation detection area for detecting excessive deformation of the flexible sheet material, the pad detection area is a prohibited placement area defined under a local coordinate system of the flexible sheet material to be grasped, and the deformation detection area is a three-dimensional cuboid area generated with the grasping point as a center.
3. The method according to claim 2, wherein the defining of the pad detection area comprises the steps of generating a plurality of three-dimensional cuboid areas in the local coordinate system of the flexible board to be grabbed according to a preset forbidden placement rule, wherein the rule comprises a distance range, an edge extension range, a hole or a notch extension range from a board center line, the height range of each three-dimensional cuboid area in the direction perpendicular to the board surface is from a preset pad detection initial height to a preset pad maximum height, and the pad detection areas are converted to a robot base coordinate system through a transformation matrix.
4. The method of claim 2, wherein the defining of the deformation detection region includes shifting the set distance upward and/or downward in a direction normal to the grabbing pose centered on the grabbing point and expanding the set length and width in a direction parallel to the grabbing plane to generate a three-dimensional cuboid detection region.
5. The method according to claim 1, wherein in the step S5, the point cloud is a point cloud number, the preset threshold includes a pad existence threshold and an excessive deformation threshold, the pad abnormality is determined to exist if the point cloud number in any pad detection area exceeds the pad existence threshold, and the excessive deformation abnormality of the flexible board is determined to exist if the point cloud number in the deformation detection area exceeds the excessive deformation threshold.
6. The method of claim 1, wherein the exception handling mechanism comprises immediately stopping the grabbing action, sounding an audible and visual alarm, displaying the type and location of the exception on a human-machine interface, triggering the robot to perform an obstacle avoidance action, or waiting for manual intervention.
7. 3D vision-based intelligent grabbing system abnormality detection and processing device for grabbing stacked flexible plates, and is characterized by comprising: The truss manipulator (1) is erected above a workstation, and a main body frame of the truss manipulator constructs a three-axis orthogonal rectangular coordinate system and is used for driving the end effector to move in the X, Y, Z direction; the end effector (2) is rigidly connected to the tail end of the Z axis of the truss manipulator (1) and is in a flat cuboid structure, the bottom surface of the end effector is parallel to the horizontal plane, and a plurality of independently-operable vacuum suckers are uniformly arranged on the end effector in an array manner and used for adsorbing the upper surfaces of the stacked flexible plates; The 3D vision sensor (3) is fixedly arranged on the side face of the Z axis of the truss manipulator (1) or the side face of the end effector (2), the optical axis of the lens is vertically downward, and the field of view covers the area where the flexible plates are stacked; a vision processing unit (4) connected with the 3D vision sensor (3) through a data cable for receiving and processing point cloud data, performing the method of any of claims 1 to 6; and the robot control unit (5) is in communication connection with the vision processing unit (4) through an industrial bus or Ethernet, and is used for receiving grabbing instructions or abnormal signals and controlling the truss manipulator to move.
8. The device according to claim 7, wherein the 3D vision sensor (3) adopts an 'eye-in-hand' follow-up installation mode, and the vision processing unit (4) and the robot control unit (5) are independently arranged in a safety cabinet outside a working area and acquire point cloud data along with the movement of the truss manipulator (1) to an optimal shooting position.
9. The apparatus of claim 7, further comprising a human-machine interface for displaying in real time point cloud images, abnormal area highlighting, alarm logs, and manual confirmation buttons for an operator to monitor system status and to intervene.
10. The apparatus of claim 7, wherein the end effector has four vacuum chucks distributed in a rectangular array, and the truss manipulator is a four-axis rectangular robot having a gripping pose represented by a translation component and a rotation angle about a Z-axis.

Description

3D vision-based intelligent grabbing system anomaly detection and processing method and device Technical Field The invention belongs to the technical field of intelligent grabbing of robots, and particularly relates to an intelligent grabbing system abnormality detection and processing method and device based on 3D vision. Background In automated warehouse and production lines, robots have been widely used to unstacke stacked flexible sheets layer by layer using 3D vision. However, the field environment is complicated and changeable, and after the upper layer plate is removed, the surface of the lower layer plate may remain with a spacer for separation, or excessive deformation such as obvious warping, sagging and the like may occur due to poor rigidity of the plate itself. The traditional 3D vision grabbing system mainly focuses on pose recognition of a target object, and lacks active sensing capability on abnormal states of grabbing areas, so that the problems that cushion block residues are not found in time, a sucker collides with the cushion block, grabbing fails and even the sucker is damaged are caused, excessive deformation of a plate is not recognized in advance, the sucker cannot establish effective vacuum, a piece is dropped or the plate is torn, the system cannot automatically process after the abnormality occurs, and manual intervention must be stopped, so that continuous production is affected. Therefore, a solution is needed for simultaneously performing three-dimensional quantitative detection and automatic processing on the cushion block and the deformation before grabbing. Disclosure of Invention Aiming at the defects of the prior art, the invention aims to provide the method and the device for detecting and processing the abnormality of the intelligent grabbing system based on the 3D vision, which realize the self-adaptive detection and processing of the cushion block and the excessive deformation by dynamically generating the detection area and fusing the point cloud threshold judgment and remarkably improve the safety, the robustness and the automation level of the grabbing system. The technical scheme is that the 3D vision-based intelligent grabbing system abnormality detection and processing method is applied to grabbing of stacked flexible plates and comprises the following steps of: s1, acquiring original 3D point cloud data of a target grabbing area of stacked flexible plates through a 3D vision sensor; s2, preprocessing the original 3D point cloud data to obtain processed point cloud data; S3, matching the preprocessed point cloud data with a preset flexible board 3D template, and determining the six-degree-of-freedom pose and the grabbing point pose of the current robot base coordinate system of the flexible board to be grabbed; S4, dynamically generating a three-dimensional detection area for anomaly detection based on the grabbing point pose; S5, counting point cloud characteristics in the three-dimensional detection area, comparing the counting result with a preset threshold value, and judging whether abnormality exists or not; and S6, triggering an exception handling mechanism if the exception exists, and outputting the grabbing point pose to the robot to execute grabbing if the exception does not exist. Further, in the step S4, the three-dimensional detection area includes a pad detection area for detecting a residual pad and a deformation detection area for detecting excessive deformation of the flexible board, where the pad detection area is a prohibited placement area defined under a local coordinate system of the flexible board to be grabbed, and the deformation detection area is a three-dimensional cuboid area generated with the grabbing point as a center. Further, the definition of the cushion block detection area comprises the steps of generating a plurality of three-dimensional cuboid areas in the local coordinate system of the flexible plate to be grabbed according to a preset forbidden placement rule, wherein the rule comprises a distance range from the center line of the plate, an edge extension range, a hole or notch extension range, the height range of each three-dimensional cuboid area in the direction perpendicular to the surface of the plate is from a preset cushion block detection initial height to a preset cushion block maximum height, and the cushion block detection areas are converted to a robot base coordinate system through a transformation matrix. Further, the definition of the deformation detection area comprises the steps of taking the grabbing point as the center, shifting upwards and/or downwards along the normal direction of the grabbing pose by a set distance, expanding the set length and width in the direction parallel to the grabbing plane, and generating a three-dimensional cuboid detection area. Further, in the step S5, the point cloud features are the number of point clouds, the preset threshold includes a cushion block existence threshold a