CN-117001659-B - Automatic grabbing method, device and equipment for rubber blocks and storage medium

CN117001659BCN 117001659 BCN117001659 BCN 117001659BCN-117001659-B

Abstract

The invention discloses an automatic grabbing method, device, equipment and storage medium for rubber blocks, which are used for acquiring target area images and point cloud data of a target area in real time; the method comprises the steps of inputting a target area image into a pre-trained example segmentation model, enabling the example segmentation model to output rubber block image data, obtaining a target area global depth image based on point cloud data, determining whether a target area is an empty frame area according to the target area global depth image after determining that a first rubber block does not exist in the target area based on the rubber block image data, analyzing the rubber block image data and the target area global depth image after determining that the target area is not the empty frame area to obtain the space position of a target rubber block in the target area, and controlling a mechanical arm to grasp the target rubber block according to the space position.

Inventors

YUAN YE
WU GUODONG
WAN LIHONG
ZHANG ZEYANG

Assignees

河南中原动力智能制造有限公司

Dates

Publication Date: 20260508
Application Date: 20230712

Claims (8)

1. An automatic grabbing method for rubber blocks is characterized by comprising the following steps: acquiring a target area image and point cloud data of a target area in real time; inputting the target area image into a pre-trained instance segmentation model so that the instance segmentation model outputs rubber block image data, and acquiring a target area global depth image based on the point cloud data; determining whether the target area is an empty frame area according to the global depth image of the target area after determining that the first rubber block does not exist in the target area based on the rubber block image data; after the target area is determined not to be an empty frame area, analyzing the rubber block image data and the global depth image of the target area to obtain the spatial position of a target rubber block in the target area; according to the space position, the mechanical arm is controlled to grasp the target rubber block; Further comprises: based on the rubber block image data, judging whether the rubber block type exists in the rubber block image data or not as NG type after detecting that the first rubber block exists in the rubber block image data; When the rubber block type is determined to be NG type in the rubber block image data, analyzing the rubber block image data and the global depth image of the target area to obtain the space position of the target rubber block in the target area; When the rubber block type in the rubber block image data is determined to be NG type, confirming that the NG type rubber block appears, and outputting an alarm prompt; and marking the rubber blocks with the inclination angles exceeding the preset threshold value as NG types, otherwise marking the rubber blocks with the inclination angles not exceeding the preset threshold value as OK types.
2. The method for automatically grabbing a rubber block according to claim 1, wherein after determining whether the target area is a blank area, further comprising: After the target area is determined to be an empty frame area, confirming that no graspable rubber block exists in the target area, and outputting a manual material changing prompt.
3. The method for automatically grabbing a rubber block according to claim 1, wherein the pre-training of the instance segmentation model specifically comprises: acquiring exposure rubber block images corresponding to different exposure values; rubber block identification is carried out on each exposure rubber block image, and a rubber block mask image corresponding to each rubber block in the exposure rubber block image is obtained; Marking rubber blocks in all the rubber blocks in a rubber block category, so that rubber blocks with the inclination angles exceeding a preset threshold value in all the rubber blocks are marked as NG categories, otherwise, rubber blocks with the inclination angles not exceeding the preset threshold value in all the rubber blocks are marked as OK categories; Generating an initial training data set based on the exposure rubber block image and the rubber block mask image, and performing data expansion on the initial training data set to obtain an expanded training data set; and training the initial neural network model based on the extended training data set until the model converges to obtain an example segmentation model.
4. The method for automatically grabbing a rubber block according to claim 1, wherein the step of acquiring the global depth image of the target area based on the point cloud data specifically comprises: mapping the point cloud data into a depth map based on a mapping relation between the point cloud of the rubber block and the depth map to obtain a global depth image of the target region, wherein the mapping relation is as follows: ; in the formula, 、 Is an arbitrary coordinate point of the image, 、 Is the center coordinate of the image and, 、、 Represents three-dimensional coordinate points of a point cloud, The z-axis value representing the camera coordinates, i.e. the distance of the object from the camera, , , Is the camera's own parameters.
5. The method for automatically grabbing a rubber block according to claim 1, wherein analyzing the rubber block image data and the global depth image of the target area to obtain the spatial position of the target rubber block in the target area specifically comprises: the rubber block image data comprises a first rubber block mask image, a first rubber block category and a first confidence; Performing alignment integration processing on the first rubber block mask image and the target area global depth image to obtain a rubber block space position in the target area; And selecting the rubber block with the highest first confidence from the highest layer of the spatial positions of the rubber blocks based on the first confidence as a target rubber block, and obtaining the spatial position of the target rubber block in the target area.
6. The automatic grabbing device for the rubber block is characterized by comprising a target area data acquisition module, a data processing module, a first target area judging module, a target rubber block position determining module and a target rubber block grabbing module; the target area data acquisition module is used for acquiring a target area image and point cloud data of a target area in real time; The data processing module is used for inputting the target area image into a pre-trained instance segmentation model so that the instance segmentation model outputs rubber block image data and acquires a target area global depth image based on the point cloud data; The first target area judging module is used for judging whether the target area is an empty frame area according to the global depth image of the target area after determining that the first rubber block does not exist in the target area based on the rubber block image data; the target rubber block position determining module is used for analyzing the rubber block image data and the target area global depth image after determining that the target area is not an empty frame area to obtain the spatial position of the target rubber block in the target area; The target rubber block grabbing module is used for controlling the mechanical arm to grab the target rubber block according to the space position; the system also comprises a second target area judging module; The second target area judging module is used for judging whether the rubber block type exists in the rubber block image data or not as NG type after detecting that the first rubber block exists in the rubber block image data based on the rubber block image data; when the rubber block type is determined to be NG type in the rubber block image data, analyzing the rubber block image data and the global depth image of the target area to obtain the space position of the target rubber block in the target area; when the rubber block type in the rubber block image data is determined to be NG type, confirming that the NG type rubber block appears, and outputting an alarm prompt; and marking the rubber blocks with the inclination angles exceeding the preset threshold value as NG types, otherwise marking the rubber blocks with the inclination angles not exceeding the preset threshold value as OK types.
7. A terminal device comprising a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, the processor implementing the method of automatic gripping of rubber blocks according to any one of claims 1 to 5 when executing the computer program.
8. A computer readable storage medium, characterized in that the computer readable storage medium comprises a stored computer program, wherein the computer program, when run, controls a device in which the computer readable storage medium is located to perform the method for automatically grabbing rubber blocks according to any one of claims 1 to 5.

Description

Automatic grabbing method, device and equipment for rubber blocks and storage medium Technical Field The present invention relates to the field of image recognition processing technologies, and in particular, to a method, an apparatus, a device, and a storage medium for automatically grabbing a rubber block. Background The rubber block is a common material in industrial production, and is mainly used for manufacturing rubber articles such as automobile tires, wherein the width of the common rubber block raw material is generally 300-500mm, the length is generally 400-700mm, and the weight is about 35kg in the automobile tire manufacturing industry. At present, the rubber block raw material is fed to the next process by means of manual carrying, obviously, the traditional rubber block feeding mode by means of manual carrying is low in efficiency, is unfavorable for manufacturers to promote self competitiveness, and virtually improves production cost, and along with the development of artificial intelligence technology, the degree of automation of a rubber block feeding production line is improved to be a current urgent requirement. Disclosure of Invention The invention aims to solve the technical problem of providing an automatic grabbing method, device and equipment for rubber blocks and a storage medium, so that the feeding efficiency of the rubber blocks is improved, and meanwhile, the production cost is reduced. In order to solve the technical problems, the invention provides an automatic grabbing method of rubber blocks, which comprises the following steps: acquiring a target area image and point cloud data of a target area in real time; inputting the target area image into a pre-trained instance segmentation model so that the instance segmentation model outputs rubber block image data, and acquiring a target area global depth image based on the point cloud data; determining whether the target area is an empty frame area according to the global depth image of the target area after determining that the first rubber block does not exist in the target area based on the rubber block image data; after the target area is determined not to be an empty frame area, analyzing the rubber block image data and the global depth image of the target area to obtain the spatial position of a target rubber block in the target area; and controlling the mechanical arm to grasp the target rubber block according to the space position. The invention provides an automatic grabbing method of a rubber block, which further comprises the following steps: based on the rubber block image data, judging whether the rubber block type exists in the rubber block image data or not as NG type after detecting that the first rubber block exists in the rubber block image data; When the rubber block type is determined to be NG type in the rubber block image data, analyzing the rubber block image data and the global depth image of the target area to obtain the space position of the target rubber block in the target area; and when the rubber block type in the rubber block image data is determined to be NG type, confirming that the NG type rubber block appears, and outputting an alarm prompt. In one possible implementation manner, after determining whether the target area is a blank area, the method further includes: After the target area is determined to be an empty frame area, confirming that no graspable rubber block exists in the target area, and outputting a manual material changing prompt. In one possible implementation manner, when the example segmentation model is pre-trained, the method specifically includes: acquiring exposure rubber block images corresponding to different exposure values; rubber block identification is carried out on each exposure rubber block image, and a rubber block mask image corresponding to each rubber block in the exposure rubber block image is obtained; Marking the rubber blocks in a class of rubber blocks so that the rubber blocks with the inclination angles exceeding a preset threshold value in the rubber blocks are marked as NG classes, otherwise, marking the rubber blocks with the inclination angles not exceeding the preset threshold value in the rubber blocks as OK classes; Generating an initial training data set based on the exposure rubber block image and the rubber block mask image, and performing data expansion on the initial training data set to obtain an expanded training data set; and training the initial neural network model based on the extended training data set until the model converges to obtain an example segmentation model. In one possible implementation manner, based on the point cloud data, acquiring a global depth image of the target area specifically includes: mapping the point cloud data into a depth map based on a mapping relation between the point cloud of the rubber block and the depth map to obtain a global depth image of the target region, wherein the mapping relation is as follows: