JP-7138753-B2 - Image defect detection method, device, electronic device, storage medium and product

JP7138753B2JP 7138753 B2JP7138753 B2JP 7138753B2JP-7138753-B2

Inventors

肖慧慧
▲聶▼ 磊
▲鄒▼ 建法
黄 ▲鋒▼

Assignees

ベイジンバイドゥネットコムサイエンステクノロジーカンパニーリミテッド
ベイジンバイドゥネットコムサイエンステクノロジーカンパニーリミテッド

Dates

Publication Date: 20220916
Application Date: 20210906
Priority Date: 20201225

Claims (17)

acquiring a detection-waiting image; obtaining at least one inpainted image corresponding to the awaiting detection image, based on the awaiting detection image, at least one mask image group, and a plurality of defect-free positive sample images; locating the defect on the detection-waiting image based on the detection-waiting image and each of the repair images; each mask image group includes at least two complementary binary images; different mask image groups have different image scales, Image defect detection method.
Obtaining a restored image corresponding to the detection-waiting image based on the detection-waiting image, at least one mask image group, and a plurality of defect-free positive sample images, obtaining a target mask image currently being processed and combining the awaiting detection image with each of the binary images in the target mask image to obtain a plurality of inpainted data pairs; inputting each of the plurality of inpainted data pairs into a pre-trained image interpolation model to form a plurality of inpainted images corresponding to the target mask images; merging a plurality of the inpainted images to obtain an inpainted image corresponding to the target mask images; The image interpolation model is obtained by training a plurality of the positive sample images, The image defect detection method according to claim 1.
Obtaining a restored image corresponding to the target mask image group by merging a plurality of the restored images includes: multiplying each inpainted image with a corresponding binary image in the set of target mask images to obtain each reference image; summing each of the reference images to obtain an inpainted image corresponding to the target mask images; 3. The image defect detection method according to claim 2.
Before inputting each of said inpainted data pairs into a pre-trained image interpolation model, Randomly generating a plurality of binary images; performing unsupervised training on a predetermined machine learning model using each said positive sample image and each said binary image to generate said image interpolation model; 3. The image defect detection method according to claim 2.
unsupervised training a given machine learning model using each of the positive sample images and each of the binary images to generate the image interpolation model; selecting one positive sample image and one binary image and correspondingly multiplying each pixel point of these images to obtain one defect image; merging the defect image and the selected binary image to obtain a four-channel target image; inputting the target image into the machine learning model and performing model training on the machine learning model; Input the target image to the machine learning model from the operation of selecting one positive sample image and one binary image until the training end condition is met to obtain the image interpolation model, and repeatedly executing up to the action of performing model training on 5. The image defect detection method according to claim 4.
wherein the at least one inpainted image includes a plurality of inpainted images; Identifying a defect position for the detection-waiting image based on the detection-waiting image and each of the repair images includes: difference comparing each of the inpainted images with the awaiting detection image to obtain a plurality of difference heat maps; choosing the maximum value of each heat point in each said difference heat map to construct a target heat map; performing binarization segmentation on the target heatmap to obtain defect regions in the detection-waiting image; The image defect detection method according to claim 1.
comparing each of the repaired images with the detection-waiting image to obtain a plurality of difference heat maps; Determining a gradient intensity similarity deviation between each inpainted image and the awaiting detection image respectively, and obtaining a plurality of difference heat maps based on the determined similarity deviation results. 7. The image defect detection method according to claim 6.
a detection waiting image acquisition module for acquiring a detection waiting image; a repaired image generation module for obtaining a repaired image corresponding to the detection-waiting image, based on the detection-waiting image, at least one mask image group, and a plurality of defect-free positive sample images; a defect location identification module for performing defect location identification on the detection-waiting image based on the detection-waiting image and each of the repair images; each mask image group includes at least two complementary binary images; different mask image groups have different image scales, Image defect detector.
The restored image generation module includes: a restoration data pair generation sub-module for obtaining a target mask image group currently being processed and combining the awaiting detection image with each of the binary images in the target mask image group to obtain a plurality of restoration data pairs; a plurality of inpainted image generation sub-modules for inputting each of the inpainted data pairs into a pre-trained image interpolation model to form a plurality of inpainted images corresponding to the target mask images; a restored image generation submodule for merging a plurality of the restored images to obtain a restored image corresponding to the target mask image group; The image interpolation model is obtained by training a plurality of the positive sample images, The image defect detection device according to claim 8.
The restored image generation sub-module includes: multiplying each inpainted image with a corresponding binary image in the set of target mask images to obtain each reference image; summing each of the reference images to obtain an inpainted image corresponding to the target mask images; The image defect detection apparatus according to claim 9.
The restored image generation module includes: Randomly generating a plurality of binary images; performing unsupervised training on a predetermined machine learning model using each of the positive sample images and each of the binary images to generate the image interpolation model; and an image interpolation model generation submodule used for prepare further, The image defect detection apparatus according to claim 9.
The image complement model generation submodule includes: selecting one positive sample image and one binary image and correspondingly multiplying each pixel point of these images to obtain one defect image; merging the defect image with a selected binary image to obtain a four-channel target image; inputting the target image into the machine learning model and performing model training on the machine learning model; back to selecting one positive sample image and one binary image until the training end condition is met to obtain the image interpolation model; The image defect detection apparatus according to claim 11.
The defect location module includes: a plurality of difference heat map generation sub-modules for comparing each of the repaired images with the detection waiting image to obtain a plurality of difference heat maps; a target heatmap construction module for selecting the maximum value of each heatpoint in each said difference heatmap to construct a target heatmap; a defect area determination sub-module for performing binarization division on the target heat map and obtaining a defect area in the detection waiting image; The image defect detection device according to claim 8.
The plurality of difference heat map generation sub-modules include: Determine the gradient intensity similarity deviation between each restoration image and the detection waiting image, and obtain a plurality of difference heat maps based on the determination result of the similarity deviation. The image defect detection apparatus according to claim 13.
at least one processor; a storage device communicatively coupled to the at least one processor; instructions executable by the at least one processor are stored in the storage device; The instructions are executed by the at least one processor such that the at least one processor can implement the image defect detection method according to any one of claims 1-7, Electronics.
A program non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the image defect detection method according to any one of claims 1 to 7.
A computer program for causing a computer to execute the image defect detection method according to any one of claims 1 to 7.

Description

FIELD OF THE DISCLOSURE The present disclosure relates to the field of image processing technology, and in particular, but not exclusively, to its application in the field of defect detection technology, and in particular to image defect detection methods, apparatus, electronics, storage media and products. . In the traditional industrial manufacturing production scenario, such as computer, communication and consumer electronics parts manufacturing, steel production, automobile manufacturing, battery manufacturing, solar panel manufacturing, etc., product appearance defect detection is important in the production process. It is a part of How to detect defects in collected product images is an important issue that attracts industry attention. The drawings are provided for better understanding of the present technical solution and do not limit the present disclosure. 1 is a schematic diagram of an image defect detection method according to an embodiment of the present disclosure; FIG.FIG. 4 is a schematic diagram of another image defect detection method according to an embodiment of the present disclosure;FIG. 4 is a schematic diagram of a further image defect detection method according to embodiments of the present disclosure;FIG. 5 is a schematic diagram of yet another image defect detection method according to an embodiment of the present disclosure;1 is a flow diagram of an image defect detection method according to an embodiment of the present disclosure; FIG.1 is a schematic diagram of a network structure of an image interpolation model according to an embodiment of the present disclosure; FIG.FIG. 4 is an architecture diagram of defect location for an image awaiting detection according to an embodiment of the present disclosure;1 is a structural schematic diagram of an image defect detection device according to an embodiment of the present disclosure; FIG.1 is a block diagram of an electronic device that implements an image defect detection method according to an embodiment of the present disclosure; FIG. Illustrative embodiments of the present disclosure are described below with reference to the drawings. This content includes various details of embodiments of the disclosure to aid understanding and should be considered exemplary only. Accordingly, those skilled in the art should recognize that various changes and modifications can be made to the examples described herein without departing from the scope and spirit of this disclosure. Similarly, for the sake of clarity and clarity, the following description omits descriptions of well-known functions and structures. FIG. 1 is a schematic diagram of an image defect detection method according to an embodiment of the present disclosure, which is applied to the situation of detecting and locating defects in an image awaiting detection, wherein the method includes image defect detection. It can be executed by a device, which can be implemented in the form of software and/or hardware, and can be integrated in an electronic device. The electronic device according to this embodiment may be a server, a computer, a smart phone, a tablet PC, or the like. Specifically, referring to FIG. 1, the method specifically includes the following. In S110, an image waiting for detection is acquired. Here, detection-waiting images are collected in an industrial manufacturing production scenario, such as, for example, a Body-in-White image, a washing machine exterior image, a gear exterior image, or an electronic product packaging image. It may be an image of any product, and this embodiment is not limited to this. Note that the detection waiting image according to the present embodiment may or may not include defects, and the present embodiment is not limited to this. Here, the defects included in the detection waiting image may be scratches, stains, uneven paint, etc., and the present embodiment is not limited to these. In the preferred implementation of this embodiment, the image collection device may collect the detection-waiting images in real time, or the detection-waiting images may be obtained from the image database, which is not limited in this embodiment. ， Illustratively, in this embodiment, a camera acquires an image of an automobile body in real time, and subsequent defect detection is performed on the acquired image of the vehicle body in real time, so that the acquired vehicle body contains defects. In addition, if the acquired vehicle body image contains a defect, the defect area in the image can be localized. At S120, a restoration image corresponding to the image awaiting detection is obtained based on the image awaiting detection, the group of at least one mask image, and the plurality of defect-free positive sample images. Here, each mask image group includes at least two complementary binary images, and different mask image groups have different image scales. Illustratively, one mask image group in this embodiment may include four binary images having a c