CN-115439446-B - Appearance defect detection method and device, storage medium and electronic equipment

CN115439446BCN 115439446 BCN115439446 BCN 115439446BCN-115439446-B

Abstract

The application provides an appearance defect detection method, an appearance defect detection device, a storage medium and electronic equipment, wherein a feature extraction branch performs feature extraction on defective pixel points in a target image according to a preset scale to acquire N dimensional image features, a target image is an acquisition image feature fusion branch of a detection object, fusion is performed on image features of adjacent dimensions to acquire N-1 fusion features, a decoding branch decodes based on the N-1 fusion features to acquire a decoding image result, the decoding image result comprises first type feature pixel points and second type feature pixel points, the first type feature pixel points represent defective pixel points, and the second type feature pixel points represent non-defective pixel points. The defect can be accurately detected under the condition of low contrast, namely under the condition that the difference between the defect and the background is not obvious, the interference of interference factors such as stains, shadows, uneven illumination and the like on the defect detection can be eliminated, and the defect detection device can be suitable for defect detection of various shapes and sizes.

Inventors

XU MINGLIANG
JIANG XIAOHENG
GU NINGBO
ZHANG YUNXIA
LU YANG
HE SHUO
ZHANG WENJIE

Assignees

北京航空航天大学杭州创新研究院

Dates

Publication Date: 20260512
Application Date: 20220906

Claims (8)

1. An appearance defect detection method, applied to an electronic device, the electronic device deployed with a pre-trained network model, the network model including a feature extraction branch, a feature fusion branch, and a decoding branch, the method comprising: the feature extraction branch performs feature extraction on defective pixel points in a target image according to a preset scale to obtain image features of N dimensions, wherein the target image is an acquired image of a detection object; the feature fusion branch fuses the image features of adjacent dimensions to obtain N-1 fusion features; The decoding branch decodes based on the N-1 fusion features to obtain a decoded image result, wherein the decoded image result comprises first-type feature pixel points and second-type feature pixel points, the first-type feature pixel points represent defective pixel points, and the second-type feature pixel points represent non-defective pixel points; The decoding branch decodes based on the N-1 fusion feature to obtain a decoded image result, wherein the decoding branch obtains a decoding result corresponding to the N-1 fusion feature based on the N-1 fusion feature and a decoding reference feature, the decoding reference feature is a reference feature obtained after space and channel enhancement is carried out on the image feature of the N dimension, the decoding branch obtains a decoding result corresponding to the i fusion feature based on the i fusion feature and a decoding result corresponding to the i+1 fusion feature, wherein i is not less than 1 and not more than N-2, and the decoding result corresponding to the 1 fusion feature is the decoded image result; The formula of the decoding result corresponding to the ith fusion feature is: Wherein, the And The splicing result of (2) represents the decoding result, F h represents the ith fusion feature, F l represents the decoding result corresponding to the (i+1) th fusion feature, up represents upsampling, maxp represents pooling downsampling, conv (1×1) represents subsequent batch normalized 1×1 convolution, delta represents an activation function, as-channel represents element-by-element multiplication, Characterization adds element by element.
2. The method of claim 1, wherein the feature fusion branch fuses the image features of adjacent dimensions to obtain N-1 fusion features, comprising: the feature fusion branch sums the features of the image features of adjacent dimensions; the feature fusion branches determine a weight matrix based on similarity of image features of adjacent dimensions; and the feature fusion branch performs weighted fusion on the feature sum based on the weight matrix to obtain the fusion feature.
3. The visual inspection method according to claim 2, wherein the characteristic sum is calculated as: ; Wherein V h +V l characterizes the feature sum, dc characterizes the depth separable convolution, maxp characterizes the pooled downsampling, X h characterizes the high-dimensional image features in the adjacent dimensions, and X l characterizes the low-dimensional image features in the adjacent dimensions.
4. The method of claim 3, wherein the fused feature is calculated as: Wherein F characterizes the fusion feature, softmax is the activation function, d k characterizes the channel dimension of K.
5. The visual defect detecting method of claim 1, wherein the decoding branches include N-1 decoders, the ith decoder is configured to obtain a decoding result corresponding to the ith fusion feature, and the joint loss function of the network model is: ; wherein L total characterizes a joint loss function, The binary cross entropy penalty of the ith decoder is characterized, The cross-over ratio IoU characterizing the i-th decoder is lost.
6. An appearance defect detection apparatus, characterized by being applied to an electronic device, comprising: the feature extraction unit is used for extracting features of defective pixel points in a target image according to a preset scale to obtain image features of N dimensions, wherein the target image is an acquisition image of a detection object; The feature fusion unit is used for fusing the image features of adjacent dimensions to obtain N-1 fusion features; The decoding unit is used for decoding based on the N-1 fusion features to obtain a decoded image result, wherein the decoded image result comprises first-class feature pixel points and second-class feature pixel points, the first-class feature pixel points represent defective pixel points, and the second-class feature pixel points represent non-defective pixel points; The decoding unit obtains a decoding result corresponding to an ith fusion feature based on the (N-1) th fusion feature and a decoding reference feature, wherein the decoding result corresponding to the (N-1) th fusion feature is obtained based on the (N-1) th fusion feature and the decoding reference feature, the decoding reference feature is a reference feature obtained after space and channel enhancement is carried out on the image feature of the (N) th dimension, and the decoding unit obtains the decoding result corresponding to the (i) th fusion feature based on the decoding result corresponding to the (i) th fusion feature and the (i+1) th fusion feature, wherein the i is not less than 1 and not more than N-2, and the decoding result corresponding to the (1) th fusion feature is the decoding image result; The formula of the decoding result corresponding to the ith fusion feature is: Wherein, the And The splicing result of (2) represents the decoding result, F h represents the ith fusion feature, F l represents the decoding result corresponding to the (i+1) th fusion feature, up represents upsampling, maxp represents pooling downsampling, conv (1×1) represents subsequent batch normalized 1×1 convolution, delta represents an activation function, as-channel represents element-by-element multiplication, Characterization adds element by element.
7. A computer readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the method according to any of claims 1-5.
8. An electronic device comprising a processor and a memory for storing one or more programs, which when executed by the processor, implement the method of any of claims 1-5.

Description

Appearance defect detection method and device, storage medium and electronic equipment Technical Field The present application relates to the field of images, and in particular, to an appearance defect detection method, an appearance defect detection device, a storage medium, and an electronic device. Background Along with the development of social science and technology and the improvement of living standard of people, the requirements of people on industrial product quality are higher and higher. The surface quality detection is a key link before the product leaves the factory. The conventional inspection method is to perform visual inspection on the product by an experienced worker, and record the number of the defective product for the next process. The method has a plurality of problems, such as missed detection and false detection caused by fatigue of long-time naked eyes of workers, great subjective influence on manual detection, lower detection efficiency and the like. Therefore, advanced automatic, accurate and efficient surface defect detection systems are urgently needed to be applied in the industrial production at present, so that manual labor force is effectively liberated, and the industrial production efficiency is improved. Disclosure of Invention The present application aims to provide an appearance defect detection method, an appearance defect detection device, a storage medium and an electronic device, so as to at least partially improve the problems. In order to achieve the above object, the technical scheme adopted by the embodiment of the application is as follows: In a first aspect, an embodiment of the present application provides an appearance defect detection method applied to an electronic device, where the electronic device is deployed with a pre-trained network model, the network model includes a feature extraction branch, a feature fusion branch, and a decoding branch, and the method includes: the feature extraction branch performs feature extraction on defective pixel points in a target image according to a preset scale to obtain image features of N dimensions, wherein the target image is an acquired image of a detection object; the feature fusion branch fuses the image features of adjacent dimensions to obtain N-1 fusion features; The decoding branch decodes based on the N-1 fusion features to obtain a decoded image result, wherein the decoded image result comprises first-type feature pixel points and second-type feature pixel points, the first-type feature pixel points represent defective pixel points, and the second-type feature pixel points represent non-defective pixel points. In a second aspect, an embodiment of the present application provides an appearance defect detection apparatus, applied to an electronic device, including: the feature extraction unit is used for extracting features of defective pixel points in a target image according to a preset scale to obtain image features of N dimensions, wherein the target image is an acquisition image of a detection object; The feature fusion unit is used for fusing the image features of adjacent dimensions to obtain N-1 fusion features; The decoding unit is used for decoding based on the N-1 fusion features to obtain a decoded image result, wherein the decoded image result comprises first-type feature pixel points and second-type feature pixel points, the first-type feature pixel points represent defective pixel points, and the second-type feature pixel points represent non-defective pixel points. In a third aspect, an embodiment of the present application provides a storage medium having stored thereon a computer program which, when executed by a processor, implements the method described above. In a fourth aspect, an embodiment of the present application provides an electronic device, where the electronic device includes a processor and a memory, where the memory is configured to store one or more programs, and when the one or more programs are executed by the processor, the method is implemented. Compared with the prior art, the appearance defect detection method, the device, the storage medium and the electronic equipment provided by the embodiment of the application have the advantages that the feature extraction branch performs feature extraction on defective pixel points in the target image according to the preset scale to acquire N dimensional image features, wherein the target image is an acquisition image feature fusion branch of a detection object, the fusion of the image features of adjacent dimensions is performed to acquire N-1 fusion features, the decoding branch decodes based on the N-1 fusion features to acquire a decoded image result, the decoded image result comprises first type feature pixel points and second type feature pixel points, the first type feature pixel points represent defective pixel points, and the second type feature pixel points represent non-defective pixel points. The defect detection