CN-121999303-A - Method and system for detecting foreign matters of coal conveying belt based on visible light image

CN121999303ACN 121999303 ACN121999303 ACN 121999303ACN-121999303-A

Abstract

The invention relates to the technical field of image recognition and discloses a method and a system for detecting foreign matters of a coal conveying belt based on a visible light image, wherein the method comprises the steps of smoothly denoising the visible light image of a coal conveying belt running area to obtain a standard image, obtaining a contour enhancement and texture suppression image through multistage enhancement, and fusing to obtain fusion characteristics; the method comprises the steps of carrying out semantic segmentation on a standard graph to obtain a segmented mask, obtaining three-dimensional point cloud through binocular vision, carrying out statistics projection to obtain a region surface normal distribution map, and carrying out coupling evaluation on fusion characteristics, mask shape characteristics and normal distribution map geometric characteristics to generate a comprehensive foreign matter detection report of a coal conveying belt operation region.

Inventors

ZHOU ZHOU
CHEN BO
TONG TAOTAO

Assignees

湖南华电平江发电有限公司

Dates

Publication Date: 20260508
Application Date: 20260408

Claims (10)

1. The method for detecting the foreign matters of the coal conveying belt based on the visible light image is characterized by comprising the following steps of: S1, carrying out smooth denoising on a visible light image of a coal conveying belt running area to obtain a standard belt area image of the coal conveying belt running area; S2, carrying out multi-stage image enhancement on the standard belt region image to obtain a contour enhancement image and a texture suppression image of the coal conveying belt running region; S3, carrying out collaborative reinforcement fusion on the contour enhancement image and the texture suppression image to obtain fusion characteristics of the coal conveying belt running region, and carrying out semantic segmentation quantization on the standard belt region image based on the fusion characteristics to obtain a segmentation mask of the standard belt region image; s4, based on the segmentation mask, binocular vision perception is carried out on the standard belt region image so as to obtain three-dimensional point cloud data of the coal conveying belt operation region; s5, carrying out statistical projection on the spatial distribution characteristics of the surface normals in the three-dimensional point cloud data to obtain an area surface normals distribution map of the coal conveying belt running area; S6, carrying out foreign matter coupling evaluation on the fusion characteristics, the shape characteristics of the segmentation mask and the geometric characteristics of the area surface normal distribution diagram to obtain a comprehensive foreign matter detection report of the coal conveying belt running area.
2. The method for detecting foreign matters in a coal conveying belt based on visible light images according to claim 1, wherein the step of smoothly denoising the visible light images of the coal conveying belt running area to obtain standard belt area images of the coal conveying belt running area comprises the steps of: taking a visible light image of a receiving coal conveying belt running area as original image data; performing Gaussian filtering on the original image data to obtain an intermediate filtering image of the coal conveying belt running area; performing edge detection sensing on the intermediate filtering image to obtain an edge retention mask of the coal conveying belt running area; And carrying out parameter self-adaptive adjustment on the Gaussian filtering operation based on the edge retention mask so as to execute secondary filtering processing and obtain a standard belt area image of the coal conveying belt running area.
3. The method for detecting foreign matters in a coal conveyor belt based on visible light images according to claim 1, wherein the step of performing multi-level image enhancement on the standard belt region image to obtain a contour enhanced image and a texture suppressed image of the coal conveyor belt running region comprises the steps of: Performing brightness contrast adjustment on the standard belt region image to obtain a brightness balance image of the coal conveying belt running region; Carrying out a Laplace edge enhancement operator on the brightness balance image to obtain a contour enhancement image of the coal conveying belt running area; And performing multidirectional texture analysis on the brightness balance image to generate a texture suppression image of the coal conveying belt running area.
4. The method for detecting foreign matters in a coal conveying belt based on visible light images according to claim 1, wherein the performing collaborative reinforcement fusion on the contour enhanced image and the texture suppression image to obtain fusion features of the coal conveying belt running region, and performing semantic segmentation quantization on the standard belt region image based on the fusion features to obtain a segmentation mask of the standard belt region image comprises: based on the historical foreign matter image detection record of the coal conveying belt running area, learning and mapping the contour enhanced image and the texture suppression image by combining an attention mechanism to construct a fusion weight generation network of the coal conveying belt running area; Inputting the contour enhancement image and the texture suppression image into the fusion weight generation network to obtain a pixel-level contour enhancement weight map of the contour enhancement image and a pixel-level texture suppression weight map of the texture suppression image; Respectively carrying out weighted quantization on the contour enhancement image and the texture suppression image according to the pixel-level contour enhancement weight map and the pixel-level texture suppression weight map to obtain a contour contribution component of the contour enhancement image and a texture contribution component of the texture suppression image; pixel superposition is carried out on the contour contribution component and the texture contribution component, and a fusion feature diagram of the coal conveying belt running area is obtained; Based on the fusion feature map, carrying out semantic tag system construction on a belt region, a background region and a region to be identified of the coal conveying belt operation region, and carrying out pixel-level semantic classification on the standard belt region image to obtain an initial semantic segmentation result of the standard belt region image; and performing space context optimization on the initial semantic segmentation result to obtain a segmentation mask of the standard belt region image.
5. The method for detecting foreign matters in a coal conveying belt based on a visible light image according to claim 4, wherein the step of constructing a semantic tag system for a belt region, a background region and a region to be identified of the coal conveying belt running region based on the fusion feature map, and performing pixel-level semantic classification for the standard belt region image to obtain an initial semantic segmentation result of the standard belt region image comprises the following steps: performing multi-scale convolution operation on the fusion feature map to obtain a context feature map of the coal conveying belt running area; Defining a semantic tag class set of the coal conveying belt running area based on the context feature map, wherein the semantic tag class set comprises a class corresponding to a belt area, a class corresponding to a background area and a class corresponding to a foreign object area to be identified; inputting the context feature map to a pixel class classification network in the fusion weight generation network to obtain a probability vector of a semantic tag class set; Based on the pixel positions of the fusion feature images, carrying out numerical screening on the probability vectors to generate an initial semantic tag image of the coal conveying belt running area; And carrying out connected domain analysis on the initial semantic label graph, and merging pixel areas which are adjacent in space and have the same label in the coal conveying belt running area according to an analysis result to obtain an initial semantic segmentation result of the standard belt area image.
6. The method for detecting foreign matters in a coal conveyor belt based on a visible light image according to claim 1, wherein the performing binocular vision on the standard belt area image based on the segmentation mask to obtain three-dimensional point cloud data of the coal conveyor belt running area comprises: According to preset binocular camera parameters, carrying out three-dimensional correction on the standard belt region image and a corresponding synchronous image of another view angle to obtain a three-dimensional image pair of the coal conveying belt running region; Extracting an image pair of a region to be matched corresponding to the coal conveying belt running region from the stereoscopic image pair based on the segmentation mask; performing dense stereo matching on the image pairs of the region to be matched to obtain an initial parallax image of the coal conveying belt running region; Performing triangulation calculation on the coal conveying belt running area according to the binocular camera parameters and the initial parallax map to obtain an initial three-dimensional space point set of the coal conveying belt running area; and performing outlier filtering and noise smoothing on the initial three-dimensional space point set to obtain three-dimensional point cloud data of the coal conveying belt operation area.
7. The method for detecting foreign matters in a coal conveyor belt based on visible light images according to claim 6, wherein the calculation formula of the triangulation calculation is as follows: ; in the formula, For the pixel coordinates of the initial disparity map, The three-dimensional space coordinates of the target of the coal conveying belt running area are obtained, Is a pixel coordinate point Is provided for the weighted projection matrix of (c), As principal point coordinates of the binocular camera parameters, For the computation of disparities in the dense stereo matching, Binocular camera baseline distance for the binocular camera parameters, And the equivalent focal length of the camera is the binocular camera parameter.
8. The method for detecting foreign matters in a coal conveyor belt based on visible light images according to claim 1, wherein the statistical projection is performed on spatial distribution characteristics of surface normals in three-dimensional point cloud data to obtain an area surface normals distribution map of the coal conveyor belt running area, and the method comprises the following steps: Performing surface normal estimation on the three-dimensional point cloud data to obtain a three-dimensional normal field of the three-dimensional point cloud data; Carrying out direction normalization coding on the normal vector in the three-dimensional normal field to obtain a normal direction characteristic diagram of a coal conveying belt running area; Projecting the three-dimensional point cloud data of the coal conveying belt running area to a two-dimensional plane, and establishing a mapping relation between the normal direction feature map and a two-dimensional projection grid in the two-dimensional plane according to the projection relation; based on the mapping relation, mapping the normal direction characteristics of the normal direction characteristic diagram to independent grids of the two-dimensional projection grid for statistical histogram, and obtaining a normal distribution descriptor of the normal direction characteristic diagram; And carrying out visual fusion rendering on the normal distribution descriptors according to the spatial sequence of the two-dimensional projection grid to obtain an area surface normal distribution map of the coal conveying belt running area.
9. The method for detecting foreign matters in a coal conveyor belt based on visible light images according to claim 1, wherein the step of performing foreign matter coupling evaluation on the fusion feature, the shape feature of the division mask and the geometric feature of the area surface normal distribution map to obtain a comprehensive foreign matter detection report of the coal conveyor belt running area comprises the steps of: carrying out space feature analysis on connected domains of the region to be identified in the segmentation mask to obtain shape features of the connected domains; Based on the spatial position of the communicating region, carrying out attribute characteristic analysis on the surface normal distribution map of the region to obtain the geometric characteristics of the coal conveying belt running region; Performing feature cascading on the fusion features, the shape features and the geometric features to construct a comprehensive description vector of the region to be identified; and detecting the foreign matters in the region to be identified according to the comprehensive description vector to obtain a comprehensive foreign matter detection report of the coal conveying belt running region.
10. A visible light image-based foreign matter detection system for a coal conveyor belt, for implementing the visible light image-based foreign matter detection method for a coal conveyor belt of claim 1, the system comprising: the image smoothing denoising module is used for smoothing and denoising the visible light image of the coal conveying belt running area to obtain a standard belt area image of the coal conveying belt running area; the multi-stage image enhancement module is used for carrying out multi-stage image enhancement on the standard belt region image to obtain a contour enhancement image and a texture suppression image of the coal conveying belt running region; The collaborative reinforcement fusion and semantic segmentation module is used for performing collaborative reinforcement fusion on the contour reinforcement image and the texture suppression image to obtain fusion characteristics of the coal conveying belt running area, and performing semantic segmentation quantization on the standard belt area image based on the fusion characteristics to obtain a segmentation mask of the standard belt area image; The binocular vision perception module is used for carrying out binocular vision perception on the standard belt region image based on the segmentation mask so as to obtain three-dimensional point cloud data of the coal conveying belt running region; the surface normal distribution statistics module is used for carrying out statistics projection on the spatial distribution characteristics of the surface normal in the three-dimensional point cloud data so as to obtain an area surface normal distribution map of the coal conveying belt running area; and the foreign matter coupling evaluation module is used for performing foreign matter coupling evaluation on the fusion characteristic, the shape characteristic of the segmentation mask and the geometric characteristic of the area surface normal distribution diagram so as to obtain a comprehensive foreign matter detection report of the coal conveying belt running area.

Description

Method and system for detecting foreign matters of coal conveying belt based on visible light image Technical Field The invention relates to the technical field of image recognition, in particular to a method and a system for detecting foreign matters of a coal conveying belt based on visible light images. Background In the field of detection of foreign matters of coal conveying belts, a traditional detection method based on visible light images has a remarkable short plate in an image preprocessing stage. While the traditional smooth denoising technology is difficult to completely retain key information of the edge of the belt and potential foreign matters while suppressing noise, loss of effective characteristics is easy to cause, the single-mode image enhancement means cannot pointedly optimize the contrast effect of the outline of the foreign matters and the texture of the belt, so that a clear and reliable detection basis is difficult to be provided for a preprocessed image, and the effect of a subsequent detection link is directly influenced. The existing detection technology has the defects of the cooperativity of feature extraction and foreign matter identification and the lack of deep fusion and comprehensive evaluation of multidimensional features. The traditional method often relies on a single feature to judge the foreign matters, and the outline feature, the texture feature and the space geometric feature of the image cannot be fully combined, so that the semantic segmentation accuracy is limited, and the three-dimensional space information is not fully utilized. The foreign matters are difficult to distinguish from the belt and the background area, the phenomena of missed detection and false detection are frequently generated in the detection process, the detection efficiency and the detection precision can not meet the actual requirements of continuous and efficient operation of the coal conveying system, and the equipment faults and the safety risks caused by the foreign matters are difficult to effectively avoid. Disclosure of Invention The invention provides a method and a system for detecting foreign matters of a coal conveying belt based on visible light images, which are used for solving the problems in the background technology. In order to achieve the above object, the invention provides a method for detecting foreign matters in a coal conveying belt based on visible light images, which comprises the following steps: S1, carrying out smooth denoising on a visible light image of a coal conveying belt running area to obtain a standard belt area image of the coal conveying belt running area; S2, carrying out multi-stage image enhancement on the standard belt region image to obtain a contour enhancement image and a texture suppression image of the coal conveying belt running region; S3, carrying out collaborative reinforcement fusion on the contour enhancement image and the texture suppression image to obtain fusion characteristics of the coal conveying belt running region, and carrying out semantic segmentation quantization on the standard belt region image based on the fusion characteristics to obtain a segmentation mask of the standard belt region image; s4, based on the segmentation mask, binocular vision perception is carried out on the standard belt region image so as to obtain three-dimensional point cloud data of the coal conveying belt operation region; s5, carrying out statistical projection on the spatial distribution characteristics of the surface normals in the three-dimensional point cloud data to obtain an area surface normals distribution map of the coal conveying belt running area; S6, carrying out foreign matter coupling evaluation on the fusion characteristics, the shape characteristics of the segmentation mask and the geometric characteristics of the area surface normal distribution diagram to obtain a comprehensive foreign matter detection report of the coal conveying belt running area. In a preferred embodiment, the smoothing denoising the visible light image of the coal conveying belt running area to obtain a standard belt area image of the coal conveying belt running area includes: taking a visible light image of a receiving coal conveying belt running area as original image data; performing Gaussian filtering on the original image data to obtain an intermediate filtering image of the coal conveying belt running area; performing edge detection sensing on the intermediate filtering image to obtain an edge retention mask of the coal conveying belt running area; And carrying out parameter self-adaptive adjustment on the Gaussian filtering operation based on the edge retention mask so as to execute secondary filtering processing and obtain a standard belt area image of the coal conveying belt running area. In a preferred embodiment, the multi-stage image enhancement of the standard belt region image to obtain a contour enhanced image and a texture suppression image of