EP-4742150-A1 - DEFECT DETECTION METHOD AND APPARATUS, ELECTRONIC DEVICE, STORAGE MEDIUM AND PROGRAM PRODUCT

EP4742150A1EP 4742150 A1EP4742150 A1EP 4742150A1EP-4742150-A1

Abstract

Disclosed in the embodiments of the present application are a defect detection method and apparatus, an electronic device, a storage medium and a program product. The method comprises: acquiring an image under test of a target workpiece and a template image of the target workpiece, and comparing the image under test with the template image so as to obtain a differential image; inputting the image under test and the differential image into a defect detection model, and executing the following processing: on the basis of the image under test and the differential image, extracting a target image feature and, on the basis of the target image feature, performing defect detection on the image under test to obtain defect detection box information of the image under test; positioning a defect area in the differential image and, according to the defect area, determining a first detection result of the target workpiece; and, according to the defect detection box information and the first detection result, determining a second detection result of the target workpiece.

Inventors

ZHAN, Jiawei

Assignees

Tencent Technology (Shenzhen) Company Limited

Dates

Publication Date: 20260513
Application Date: 20240911

Claims (20)

A defect detection method, executable by an electronic device, and comprising: obtaining an image of a workpiece and a template image of the workpiece, and comparing the image with the template image to obtain a differential image; inputting the image and the differential image to a pre-trained defect detection model, to perform the following processing: extracting a target image feature from the image and the differential image; and performing defect detection on the image based on the target image feature to obtain information of a bounding box for annotating a first defect region in the image; locating a second defect region in the differential image, and determining a first detection result of the workpiece based on whether the second defect region meets a pre-defined defect feature condition; and determining a second detection result of the workpiece based on the information of the bounding box and the first detection result.
The defect detection method according to claim 1, wherein the extracting a target image feature from the image and the differential image comprises: extracting a first image feature from the image; extracting a second image feature from the differential image; and fusing the first image feature and the second image feature to obtain the target image feature.
The defect detection method according to claim 2, wherein the image is a multi-channel image, the differential image is a single-channel image, the inputting the image and the differential image to a pre-trained defect detection model comprises: obtaining an expanded differential image by replicating data of the differential image, a quantity of channels of the expanded differential image being equal to a quantity of channels of the image; and inputting the image and the expanded differential image to the defect detection model.
The defect detection method according to claim 2, wherein the extracting a first image feature from the image comprises: sequentially performing convolution on the image in a plurality of stages, wherein each stage comprises at least one convolutional branch; a first stage of the plurality of stages comprises one convolutional branch taking the image as input; for each of remaining stages after the first stage, a new convolutional branch is added upon entering a next stage from the respective stage; a resolution of the new convolutional branch is less than a resolution of any existing convolutional branch in the respective stage; and a feature inputted to each convolutional branch in the respective stage is obtained by fusing features outputted by all convolutional branches in a previous stage; fusing initial convolutional features outputted by convolutional branches in a last stage of the plurality of stages, to obtain a first convolutional feature; and downsampling the first convolutional feature to obtain the first image feature.
The defect detection method according to claim 4, wherein the fusing initial convolutional features outputted by convolutional branches in a last stage of the plurality of stages, to obtain a first convolutional feature comprises: fusing the initial convolutional features to obtain a fusion feature; for each of the convolutional branches in the last stage, performing convolution on the fusion feature using a resolution of the respective convolutional branch, to obtain a second convolutional feature of the respective convolutional branch; and fusing second convolutional features of the convolutional branches in the last stage to obtain the first convolutional feature.
The defect detection method according to claim 2, wherein the fusing the first image feature and the second image feature to obtain the target image feature comprises: obtaining a third image feature, by concatenating a feature value of each pixel in the first image feature with a feature value of a corresponding pixel in the second image feature; reducing a dimension of the third image feature to obtain a fourth image feature; and activating the fourth image feature to obtain the target image feature.
The defect detection method according to any one of claims 1 to 6, wherein the defect detection model comprises a plurality of concatenated network heads, a local region feature extractor connected to each of the plurality of concatenated network heads, and a region proposal network, RPN, and the performing defect detection on the image based on the target image feature to obtain information of a bounding box for annotating a first defect region in the image comprises: inputting the target image feature to the RPN for region extraction, to obtain coordinates of a reference bounding box; for a first network head of the plurality of concatenated network heads, inputting the coordinates of the reference bounding box and the target image feature to a local region feature extractor connected to the first network head to obtain a pooling feature, inputting the pooling feature to the first network head for defect detection, and outputting coordinates of a detection bounding box; for each of remaining network heads following the first network head, inputting the coordinates of the detection bounding box outputted by a previous network head and the target image feature to a local region feature extractor connected to the respective network head to obtain a pooling feature, inputting the pooling feature to the respective network head for defect detection, and outputting coordinates of a detection bounding box; and determining coordinates of a detection bounding box outputted by a last network head as the information of the bounding box.
The defect detection method according to any one of claims 1 to 7, wherein the information of the bounding box comprises coordinates of the bounding box, a confidence probability associated with the bounding box, and a defect category associated with the bounding box, and the determining a second detection result of the workpiece based on the information of the bounding box and the first detection result comprises: when the first detection result indicates that the workpiece has no defect, obtaining a pre-set confidence probability threshold and a pre-set area threshold that correspond to the defect category; determining a defect area based on the coordinates, and determining a first comparison result between the defect area and the pre-set area threshold, and a second comparison result between the confidence probability and the pre-set confidence probability threshold; and determining the second detection result based on the first comparison result and the second comparison result.
The defect detection method according to claim 8, further comprising: when the first detection result indicates that the workpiece has a defect, determining that the second detection result is that the workpiece has a defect.
The defect detection method according to any one of claims 1 to 9, wherein the comparing the image with the template image to obtain a differential image comprises: determining a transformation parameter between the image and the template image; aligning a workpiece object displayed in the template image with a workpiece object displayed in the image based on the transformation parameter, to obtain an aligned image; comparing the aligned image with the template image to obtain an initial differential image; and filtering the initial differential image to obtain the differential image.
The defect detection method according to claim 10, wherein the determining a transformation parameter between the image and the template image comprises: determining, by using a sliding window, a plurality of window regions in the image; for each of the plurality of window regions, calculating a correlation coefficient between image content in the respective window region and image content in a window region corresponding to the respective window region in the template image; determining a target region in the image based on correlation coefficients of the plurality of window regions; determining a transformation matrix used for transforming the target region into a region corresponding to the target region in the template image; and determining an inverse matrix of the transformation matrix as the transformation parameter.
The defect detection method according to claim 1, wherein the extracting a target image feature from the image and the differential image comprises: concatenating the image and the differential image to obtain a concatenated image; and inputting the concatenated image to the defect detection model to extract the target image feature.
The defect detection method according to any one of claims 1 to 12, wherein the defect detection model is trained through the following operations: obtaining a training image of the workpiece, the training image being labeled with a defect category label; comparing the training image with the template image to obtain a sample differential image; inputting the training image and the sample differential image to the defect detection model to obtain information of a sample bounding box in the training image, the information of the sample bounding box comprising a sample defect category of the workpiece; determining an initial classification loss based on the sample defect category and the defect category label; adjusting the initial classification loss based on a pre-set first adjustment parameter and a pre-set second adjustment parameter to obtain a target classification loss, the pre-set first adjustment parameter being related to a quantity of positive samples and a quantity of negative samples corresponding to the defect category label, and the pre-set second adjustment parameter being related to a classification difficulty level corresponding to the defect category label; and training the defect detection model based on the target classification loss.
A defect detection apparatus, comprising: a first processing module, configured to obtain an image of a workpiece and a template image of the workpiece, and compare the image with the template image to obtain a differential image; a second processing module, configured to input the image and the differential image to a defect detection model, to perform the following processing: extracting a target image feature from the image and the differential image; and perform defect detection on the image based on the target image feature to obtain information of a bounding box for annotating a first defect region in the image; a third processing module, configured to locate a second defect region in the differential image, and determine a first detection result of the workpiece based on whether the second defect region meets a pre-defined defect feature condition; and a fourth processing module, configured to determine a second detection result of the workpiece based on the information of a bounding box and the first detection result.
The defect detection apparatus according to claim 14, wherein the second processing module is configured to: extract a first image feature from the image; extract a second image feature from the differential image; and fuse the first image feature and the second image feature to obtain the target image feature.
The defect detection apparatus according to claim 14 or 15, wherein the defect detection model comprises a plurality of concatenated network heads, a local region feature extractor connected to each of the plurality of concatenated network heads, and a region proposal network, RPN; and the second processing module is configured to: input the target image feature to the RPN for region extraction, to obtain coordinates of a reference bounding box; for a first network head of the plurality of concatenated network heads, input the coordinates of the reference bounding box and the target image feature to a local region feature extractor connected to the first network head to obtain a pooling feature, input the pooling feature to the first network head for defect detection, and output coordinates of a detection bounding box; for each of remaining network heads following the first network head, input the coordinates of the detection bounding box outputted by a previous network head and the target image feature to a local region feature extractor connected to the respective network head to obtain a pooling feature, input the pooling feature to the respective network head for defect detection, and output coordinates of a detection bounding box; and determine coordinates of a detection bounding box outputted by a last network head as the information of the bounding box.
The defect detection apparatus according to any one of claims 14 to 16, wherein the information of a bounding box comprises coordinates of the bounding box, a confidence probability associated with the bounding box, and a defect category associated with the bounding box; and the fourth processing module is configured to: when the first detection result indicates that the workpiece has no defect, obtain a pre-set confidence probability threshold and a pre-set area threshold that correspond to the defect category; determine a defect area based on the coordinates, and determine a first comparison result between the defect area and the pre-set area threshold, and a second comparison result between the confidence probability and the pre-set confidence probability threshold; and determine the second detection result based on the first comparison result and the second comparison result.
An electronic device, comprising a memory and a processor, the memory having a computer program stored therein, and the processor, when executing the computer program, implementing the defect detection method according to any one of claims 1 to 13.
A computer-readable storage medium, the storage medium having a computer program stored therein, and the computer program, when executed by a processor, implementing the defect detection method according to any one of claims 1 to 13.
A computer program product, comprising a computer program, the computer program, when executed by a processor, implementing the defect detection method according to any one of claims 1 to 13.

Description

RELATED APPLICATION This disclosure claims priority to Chinese Patent Application No. 202311498351.5, entitled "DEFECT DETECTION METHOD AND APPARATUS, ELECTRONIC DEVICE, AND STORAGE MEDIUM" filed with the China National Intellectual Property Administration on November 10, 2023. FIELD OF THE TECHNOLOGY This disclosure relates to the field of artificial intelligence technologies, and in particular, to a defect detection method and apparatus, an electronic device, a storage medium, and a program product. BACKGROUND OF THE DISCLOSURE In some applications, product defect detection is usually implemented by using a deep learning algorithm. However, accuracy of defect detection performed by using the deep learning algorithm relies on a large quantity of manually labeled defect samples. A larger quantity of defect types or lower distinguishability of defect features indicates that a larger quantity of defect samples may be needed, leading to high costs of obtaining samples and low training efficiency. In addition, because supervised training is usually used in the deep learning algorithm, accuracy of defect detection may be reduced when training samples are insufficient. SUMMARY Embodiments of this disclosure provide a defect detection method and apparatus, an electronic device, a storage medium, and a program product, to improve accuracy of defect detection and reduce dependency on a training sample. According to an aspect, an embodiment of this disclosure provides a defect detection method, including: obtaining an image of a workpiece and a template image of the workpiece, and comparing the image with the template image to obtain a differential image;inputting the image and the differential image to a pre-trained defect detection model, to perform the following processing: extracting a target image feature from the image and the differential image; andperforming defect detection on the image based on the target image feature to obtain information of a bounding box for annotating a first defect region in the image;locating a second defect region in the differential image, and determining a first detection result of the workpiece based on whether the second defect region meets a pre-defined defect feature condition; anddetermining a second detection result of the workpiece based on the information of the bounding box and the first detection result. According to another aspect, an embodiment of this disclosure further provides a defect detection apparatus, including: a first processing module, configured to obtain an image of a workpiece and a template image of the workpiece, and compare the image with the template image to obtain a differential image;a second processing module, configured to input the image and the differential image to a defect detection model, to perform the following processing: extracting a target image feature from the image and the differential image; and perform defect detection on the image based on the target image feature to obtain information of a bounding box for annotating a first defect region in the image;a third processing module, configured to locate a second defect region in the differential image, and determine a first detection result of the workpiece based on whether the second defect region meets a pre-defined defect feature condition; anda fourth processing module, configured to determine a second detection result of the workpiece based on the information of a bounding box and the first detection result. According to another aspect, an embodiment of this disclosure further provides an electronic device, including a memory and a processor, the memory having a computer program stored therein, and the processor, when executing the computer program, implementing the foregoing defect detection method. According to another aspect, an embodiment of this disclosure further provides a computer-readable storage medium, the storage medium having a computer program stored therein, and the computer program, when executed by a processor, implementing the foregoing defect detection method. According to another aspect, an embodiment of this disclosure further provides a computer program product, the computer program product including a computer program, and the computer program being stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, to enable the computer device to perform the foregoing defect detection method. BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings are intended to provide a further understanding of the technical solutions of this disclosure, constitute a part of the specification, and are used to explain the technical solutions of this disclosure together with the embodiments of this disclosure, but do not constitute a limitation on the technical solutions of this disclosure. FIG. 1 is a schematic diagram of an exemplary imp