CN-116721066-B - Metal surface defect detection method, device and storage medium

CN116721066BCN 116721066 BCN116721066 BCN 116721066BCN-116721066-B

Abstract

The invention discloses a metal surface defect detection method, a metal surface defect detection device and a storage medium, and belongs to the technical field of defect detection. The method comprises the steps of collecting images of high-light metal calibration balls in different illumination directions by using a light intensity three-dimensional image collecting device, obtaining an illumination direction matrix according to the collected images and a calibration formula, placing metal to be detected into the light intensity three-dimensional image collecting device, sequentially lighting up all light sources to obtain object surface image sequences in different illumination directions, fusing a plurality of object surface images in the object surface image sequences to obtain fused images, calculating a divergence map of the object surface according to the object surface image sequences and the illumination direction matrix, and respectively inputting the fused images and the divergence map into a preset double-branch feature fusion network to extract fusion features of the images and output a final defect detection result. The invention improves the defect detection capability by improving the contrast between the defect and the background and capturing the geometric shape information of the surface.

Inventors

HU GUANGHUA
TU QIANXI
LI ZHENDONG

Assignees

华南理工大学

Dates

Publication Date: 20260505
Application Date: 20230526

Claims (9)

1. A method for detecting defects on a metal surface, comprising the steps of: Acquiring images of the high-light metal calibration balls in different illumination directions by using a luminosity three-dimensional image acquisition device, and acquiring an illumination direction matrix according to the acquired images and a calibration formula, wherein the luminosity three-dimensional image acquisition device comprises a plurality of light sources in different directions; Placing the metal to be detected into the luminosity three-dimensional image acquisition device, and sequentially lighting each light source to obtain object surface image sequences in different illumination directions; Fusing a plurality of object surface images in the object surface image sequence to obtain a fused image; Calculating a divergence map of the object surface according to the object surface image sequence and the illumination direction matrix; respectively inputting the fusion image and the divergence image into a preset double-branch feature fusion network to extract fusion features of the image and output a final defect detection result; The calculating the divergence map of the object surface according to the object surface image sequence and the illumination direction matrix comprises the following steps: Calculating a divergence map and a curvature map of the surface of the object by utilizing a photometric stereo method, a divergence formula and a curvature formula according to the object surface image sequence and the illumination direction matrix; the calculation formula of the photometric stereo method is as follows: wherein I is the brightness value of a certain position of the image, L is the illumination direction matrix, N is the unit normal vector of the position, which is the reflectivity of the surface of the object; Will be Considering as the whole N, the normal vector calculation formula is: 。
2. The method for detecting defects on a metal surface according to claim 1, wherein the photometric stereo image capturing device comprises: an industrial camera; Four light sources; the light source frame is used for installing the light sources, and the four light sources are respectively arranged on the cross beams in the four directions of the light source frame; the objective table is arranged in the middle of the aperture and used for placing metal to be measured; And the camera bracket is used for fixing the industrial camera, and the industrial camera is positioned right above the objective table and shoots vertically downwards.
3. The method for detecting metal surface defects according to claim 1, wherein the expression of the calibration formula is: wherein N is the normal vector of the center point position of the highlight region, V is the reflected light vector, For illumination direction vectors of a single light source, the illumination direction vectors of all light sources are combined into an illumination direction matrix.
4. The method of claim 1, wherein fusing the plurality of object surface images in the sequence of object surface images comprises: fusing the plurality of images into one image by using a contrast pyramid fusion algorithm; The contrast pyramid fusion algorithm firstly obtains a Laplacian pyramid of an image, and the calculation formula is as follows: In the formula, In the form of a gaussian filter, For the layer i image of the gaussian pyramid, For the image after interpolation expansion of the first layer image, Z is the highest layer number of the Laplacian pyramid, A first layer image of the Laplacian pyramid; Will be Seen as The contrast pyramid is defined as: In the formula, For the value at the first layer contrast pyramid position x of the kth image, 、 And respectively representing the values of the positions corresponding to the fused contrast pyramid and the Gaussian pyramid, taking the value with the largest absolute value of the M images at the position to construct a new contrast pyramid, taking the average value of the Gaussian pyramid of the M images to construct a new Gaussian pyramid, and recovering the fused image by using the new contrast pyramid and the Gaussian pyramid.
5. The method of claim 1, wherein the divergence formula is expressed as: The expression of the curvature formula is: Wherein p is Representing the gradient of the object surface along the x-direction, q being Representing the gradient of the surface in the y-direction, div representing the divergence at various locations of the object, Representing the curvature at various locations of the object.
6. The method for detecting metal surface defects according to claim 1, wherein the double-branch feature fusion network is a double-branch yolov s feature fusion network, and a residual space channel attention module is introduced into the network, and the expression is as follows: In the formula, 、 Representing a channel attention map and a spatial attention map, respectively, F being an input feature, For Sigmoid activation functions, MLP is a shared multi-layer perceptron, For stacking operations AvgPool, maxPool represents average pooling and maximum pooling, In order for the convolution operation to be performed, Is an output feature.
7. The method for detecting metal surface defects according to claim 1, wherein the dual-branch feature fusion network performs feature fusion on features extracted by the dual-branch through the following formula: In the formula, For features extracted from the branches of the divergence map, For the features extracted by the RGB image branches, conv is a convolution operation with halving the number of channels, In order to be a double-branch fusion characteristic, Is a stacking operation.
8. A metal surface defect detection apparatus, comprising: At least one processor; At least one memory for storing at least one program; the at least one program, when executed by the at least one processor, causes the at least one processor to implement the method of any one of claims 1-7.
9. A computer readable storage medium, in which a processor executable program is stored, characterized in that the processor executable program is for performing the method according to any of claims 1-7 when being executed by a processor.

Description

Metal surface defect detection method, device and storage medium Technical Field The present invention relates to the field of defect detection technologies, and in particular, to a method and an apparatus for detecting a metal surface defect, and a storage medium. Background The metal plate and the sheet metal part are important products in modern production, and have important application in various fields such as mechanical manufacture, automobile production, aerospace and the like. However, various surface defects such as scratches, pit protrusions, etc. are inevitably generated during the manufacturing process of the product, and these defects not only affect the appearance of the product, but also may shorten the service life and even pose a safety threat, so that in order to prevent defective products from flowing into the market, it is necessary to perform product appearance inspection before shipping. The traditional manual visual inspection method has long time and high cost, and the detection effect is influenced by subjective factors, so that an automatic detection method based on vision is needed. The defect detection method mainly comprises two large modules of image acquisition and detection algorithm, wherein the image acquisition is the basis of the whole detection flow, the quality of the image is an important factor for determining the final detection effect, if the defect characteristics are highlighted as much as possible during imaging, the difficulty of subsequent detection can be greatly reduced, the defect detection method is greatly dependent on illumination conditions, certain types of defects such as scratches, scratches and the like are generally slender and have obvious directionality, high contrast can be displayed only in specific illumination directions and angles, most of the current detection methods only use a single light source for illumination, when the defect azimuth is not matched with the illumination directions, the defects cannot be well highlighted from the background, thus missed detection is caused, defects such as pits and protrusions only have depth fluctuation, have no obvious difference from the background in color, reflectivity and the like, and have the trouble of not high detection rate when the RGB image detection is simply relied on. In addition, in terms of detection algorithms, most of the current mainstream methods are based on a general deep learning target detection framework, and due to different defect forms, different sizes and very similar partial defects to the background, the models can not capture defect characteristic information well, so that the problems of low recognition rate, low detection precision and the like are caused. Disclosure of Invention In order to solve at least one of the technical problems existing in the prior art to a certain extent, the invention aims to provide a metal surface defect detection method, device and storage medium based on photometric stereo imaging and dual-branch feature fusion network. The technical scheme adopted by the invention is as follows: a method for detecting defects on a metal surface, comprising the steps of: Acquiring images of the high-light metal calibration balls in different illumination directions by using a luminosity three-dimensional image acquisition device, and acquiring an illumination direction matrix according to the acquired images and a calibration formula, wherein the luminosity three-dimensional image acquisition device comprises a plurality of light sources in different directions; Placing the metal to be detected into the luminosity three-dimensional image acquisition device, and sequentially lighting each light source to obtain object surface image sequences in different illumination directions; Fusing a plurality of object surface images in the object surface image sequence to obtain a fused image; Calculating a divergence map of the object surface according to the object surface image sequence and the illumination direction matrix; And respectively inputting the fusion image and the divergence map into a preset double-branch characteristic fusion network to extract fusion characteristics of the image and output a final defect detection result. Further, the photometric stereo image capturing device includes: an industrial camera; Four light sources; the light source frame is used for installing the light sources, and the four light sources are respectively arranged on the cross beams in the four directions of the light source frame; the objective table is arranged in the middle of the aperture and used for placing metal to be measured; And the camera bracket is used for fixing the industrial camera, and the industrial camera is positioned right above the objective table and shoots vertically downwards. Further, the expression of the calibration formula is: L=2(V·N)N-V Wherein N is a normal vector of the center point position of the highlight region, V is a reflected li