CN-116494232-B - Use method of coal and gangue rapid sorting mechanical arm system based on improved mask-rcnn algorithm

CN116494232BCN 116494232 BCN116494232 BCN 116494232BCN-116494232-B

Abstract

The invention discloses a use method of a coal and gangue rapid sorting mechanical arm system based on an improved mask-rcnn algorithm, which comprises an algorithm for detecting and dividing coal and gangue, an algorithm for calculating depth information of coal and gangue images acquired by a binocular camera and an algorithm for calculating an optimal grabbing angle of the divided images, wherein the algorithm is realized by an improved mask-rcnn model. The invention solves the problems of the detection and segmentation of coal and gangue, realizes the measurement of the distance between the coal and the gangue, and calculates the optimal grabbing angle. The system improves the efficiency of sorting coal and gangue, and has important significance for green development of coal industry.

Inventors

LI ZHUOQIN
CAO ZHENGUAN
LI JINBIAO
FANG LIAO
YANG XUN
LI RUI

Assignees

安徽理工大学

Dates

Publication Date: 20260508
Application Date: 20230427

Claims (3)

1. The application method of the coal and gangue rapid sorting mechanical arm system based on the improved mask-rcnn algorithm is characterized by comprising the following steps of: (1) Sending the collected coal and gangue data set into an improved mask-rcnn model for training, and storing the trained optimal weight; (2) Detecting and dividing pictures acquired by the binocular camera by using a trained improved mask-rcnn algorithm, reading parallax among the left and right pictures from the detected coal and gangue, and calculating depth information; (3) Calculating the optimal clamping angle between the tail end of the robot arm and the coal and the gangue according to the shape and the position of the separated coal and gangue; In the content (1), the feature extraction network architecture in the back plane layer of the mask-rcnn is improved, the structure in layer1 of ResNet is changed into a Conv+BN structure, a residual error structure is introduced, the structure in layer2 is changed into a Conv+BN+ReLU structure, a residual error structure and an attention mechanism are further added in the structures of layer3 and layer4, after the steps, semantic information of high-level features and detailed information of bottom features of a feature map obtained by a feature extraction part are more abundant, and a foundation is laid for subsequent detection and segmentation; In the content (1), the detection and classification are carried out on coal and gangue, and the environment and the measured object are single, so that 256×256 target frames of the FPN layer structure of mask-rcnn are reduced, only the conditions that the number of the 6 frames is 128×128, 256×256, 256×128, 512×512, 1024×1024 and 1024×512 respectively are reserved, the total number of the frames is reduced by 0.67 times, and the training and reasoning time is greatly reduced.
2. The method for using the mechanical arm system for quickly sorting coal and gangue based on the improved mask-rcnn algorithm according to claim 1, wherein in the content (2), a pair of images acquired by a binocular camera are sequentially sent into a trained improved mask-rcnn network for reasoning, aiming at an anchor frame for identifying coal and gangue, the central position of a target frame is read out, the coordinates of a central point in a left eye camera picture and the values of pixel points R, G, B on the central point position are reserved, the coordinates of the central point of the target frame in a right eye camera are respectively expanded by 10 pixel points upwards and downwards, 49 pixel points are expanded leftwards and rightwards, a rectangular frame with the height and width of 21×99 is formed, then a template with the width of 5×5 is sequentially traversed, R, G, B values of the 25 pixel points covered by the template are added to be averaged to replace R, G, B pixel values of the central point of the template, finally the pixel values in the points are compared with the coordinate values of the pixel points on the central point of the left eye camera, the coordinate values of the pixel points are selected by the coordinate values of the left eye camera, the coordinate values of the left eye camera and the coordinate values of the left eye camera are the coordinate values of the two points, and the difference of the coordinate values of the two images are the image is the image of the triangle, and the difference is obtained.
3. The method for using the mechanical arm system for quickly sorting the coal and the gangue based on the improved mask-rcnn algorithm according to the method for using the mechanical arm system for quickly sorting the coal and the gangue, which is disclosed by the invention, is characterized in that in the content (3), according to the shapes of the separated coal and gangue, the edge coordinates of the separated coal and the gangue are output, the distances between the points and surrounding points are traversed in sequence, two points with the largest distances are selected, the slope of the points in an image is calculated, and the included angle of the mechanical claws at the tail end of the mechanical arm is vertical to the included angle, so that the effective grabbing of the coal and the gangue can be realized.

Description

Use method of coal and gangue rapid sorting mechanical arm system based on improved mask-rcnn algorithm Technical Field The invention relates to the field of cooperative mechanical arms, in particular to a use method of a coal and gangue rapid sorting mechanical arm system based on an improved mask-rcnn algorithm. Background With continuous research in the field of vision, machine vision is beginning to gradually increase the variety of colors in the fields of object recognition, classification, segmentation, and the like. The color, size, shape and position of the object are visually available and the vision provides 80% of the external information, where depth information for obtaining a given target is one of the hot spots currently under investigation. Traditional coal and gangue sorting relies on manual sorting by workers, but the complex environment of underground roadways and a large amount of dust generated during coal collection have great harm to the health of workers, and along with the development of coal mine intellectualization, coal and gangue sorting enters a new stage. Therefore, a rapid sorting mechanical arm system for coal and gangue based on an improved mask-rcnn algorithm is needed to realize the functions, the coal and the gangue can be automatically identified in a set environment, the depth information and the shape of the coal and the gangue are calculated, the mechanical arm is guided to sort the coal and the gangue according to the acquired information, and the sorting efficiency of the coal and the gangue is improved. Disclosure of Invention The invention aims to provide a use method of a coal and gangue rapid sorting mechanical arm system based on an improved mask-rcnn algorithm, which firstly collects a large number of coal and gangue pictures, and performing operations such as cutting, rotating, noise reduction, amplifying and shrinking on the pictures, expanding a data set, and dividing a training set, a verification set and a test set according to the ratio of 10:3:2. Then, the mask-rcnn network model is improved, the feature extraction network architecture in the back plane layer is improved, the structure in layer1 of ResNet is changed into a Conv+BN structure, a residual structure is introduced, the structure in layer2 is changed into a Conv+BN+ReLU structure, a residual structure and an attention mechanism are further added in the structures of layer3 and layer4, and semantic information of high-level features of a feature map and detailed information of bottom-level features obtained by a feature extraction part are improved. And then the number of anchors in the FPN layer is reduced from 9 to 6, so that the total anchors are reduced by 0.67 times, and the training and reasoning time is greatly reduced. And sending the data set of the user to an improved mask-rcnn model for training, and storing the trained weight. The method comprises the steps of sending a pair of images shot by a binocular camera into an improved mask-rcnn model for reasoning, respectively obtaining target frames corresponding to recognized coal and gangue, reading out the central positions of the target frames, reserving the central point coordinates in a left-eye camera picture and the values of pixel points R, G, B at the central point positions, expanding the coordinates of the central points of the target frames in a right-eye camera upwards and downwards respectively for 10 pixel points, expanding 49 pixel points leftwards and rightwards, traversing each pixel point in a 21X 99 rectangular frame in sequence by using a 5X 5 square template, respectively adding R, G, B values of the pixel points covered by the template to average values, replacing R, G, B values of pixels of the central points of the original template, comparing the pixel point values of the points with the pixel point values at the central point positions of the left-eye camera, selecting a matched pixel point with the smallest difference between the pixel values, obtaining the coordinate values of the images of the pixels, obtaining absolute values of differences on the x coordinates of the two pixel points, namely obtaining a parallax value, and obtaining depth information according to an imaging principle and a triangle similarity theorem. And outputting edge coordinates of the coal and gangue separated by the improved mask-rcnn according to the shapes of the coal and gangue, traversing the distances between the points and surrounding points in sequence, selecting two points with the largest distance, calculating the slope of the points in an image, enabling the included angle of a mechanical claw at the tail end of the mechanical arm to be at a vertical angle with the included angle, and guiding the mechanical arm to reach a designated position to finish the grabbing task by combining depth information. Drawings FIG. 1 is a system scheme of the present invention FIG. 2 is a flow chart of an algorithm of the present inventio