CN-119810017-B - Defect detection method and device, and super-parameter optimization method and device

CN119810017BCN 119810017 BCN119810017 BCN 119810017BCN-119810017-B

Abstract

The method comprises the steps of firstly obtaining a sample image set, then training a target task model by using the sample image set for each super parameter combination in a plurality of super parameter combinations, evaluating and recording performance indexes of the trained target task model when executing the target task, and finally selecting the super parameter combination corresponding to the trained target task model with the best performance indexes as the super parameter combination finally used for training the target task model. In the process of training the model, if the prediction error of the target task model on any sample image is smaller than an error threshold value, the sample image is abandoned, and the subsequent training of rounds is not participated, so that the sample amount used in the super-parameter optimization process is reduced, the time consumption is reduced, the most important and representative sample is utilized by the model, and the optimal super-parameter combination is obtained efficiently and accurately.

Inventors

ZENG LIHONG
YANG YANG
HUANG GAN
HUANG TAO
ZHAI AITING

Assignees

深圳市华汉伟业科技有限公司

Dates

Publication Date: 20260512
Application Date: 20241125

Claims (13)

1. A defect detection method, comprising: acquiring an image of a detected object; inputting the image of the detected object into an encoder in a pre-trained defect segmentation model to perform multi-scale feature extraction to obtain a plurality of layers of first feature images with different resolutions, wherein the defect segmentation model further comprises a decoder and a classification head module, and the decoder comprises a first convolution layer and a self-attention module; Fusing the first characteristic diagrams with different resolutions, inputting the fused characteristic diagrams into the first convolution layer for convolution processing to obtain a first multi-category score diagram, wherein the number of channels of the first multi-category score diagram is the same as the number of preset category classes, and at least one category class is used for representing defects; using the self-attention module, and using the first multi-category score map to enhance context information of the fused feature map to obtain a fused feature map; the classification head module utilizes the fusion feature map to conduct pixel-level classification prediction so as to obtain a defect segmentation result; Wherein the defect segmentation model is trained by: Acquiring a first sample image set, wherein the first sample image set comprises a plurality of first sample images and corresponding defect segmentation labels, and the defect segmentation labels are used for indicating defect areas in the first sample images; For each super-parameter combination in a plurality of preset super-parameter combinations, training a defect segmentation model by using the first sample image set by adopting the super-parameter combination, and evaluating defect segmentation performance of the trained defect segmentation model; recording the defect segmentation performance of a defect segmentation model trained by adopting each super-parameter combination, wherein in the process of training the defect segmentation model by utilizing the first sample image set, if the defect segmentation error of the defect segmentation model for any one first sample image is smaller than a first error threshold value, the first sample image is abandoned, and the training of subsequent rounds is not participated; selecting a super-parameter combination corresponding to the trained defect segmentation model with the best defect segmentation performance as a final super-parameter combination, and training the defect segmentation model by adopting the final super-parameter combination to obtain a trained defect segmentation model.
2. The defect detection method of claim 1, wherein the acquiring the first set of sample images comprises: acquiring an original sample image set, wherein the original sample image set comprises a plurality of original sample images and corresponding original defect segmentation labeling diagrams, the original defect segmentation labeling diagrams are provided with defect marking areas, and the areas corresponding to the defect marking areas in the corresponding original defect segmentation labeling diagrams in the original sample images are defect areas; The method comprises the steps of obtaining an original defect segmentation label image, obtaining a first defect segmentation label image by intercepting part of images including the defect marking area in the original defect segmentation label image, intercepting the area corresponding to the first defect segmentation label image intercepted in the corresponding original defect segmentation label image in the original sample image, and obtaining a first sample image corresponding to the first defect segmentation label image; unifying the resolutions of all the first sample images and the first defect segmentation annotation map; and forming the first sample image set by all the first sample images and the corresponding first defect segmentation annotation graph, wherein the first defect segmentation annotation graph is the defect segmentation annotation.
3. The defect detection method of claim 1 or 2, wherein training the defect segmentation model using the first set of sample images and evaluating defect segmentation performance of the trained defect segmentation model comprises: Dividing the first sample image set into a training set and a verification set according to a preset proportion, training a defect segmentation model by using the training set, and evaluating the defect segmentation performance of the trained defect segmentation model by using the verification set after training.
4. The defect detection method of claim 1, wherein, The defect segmentation result comprises a classification result of whether each pixel in the image of the detected object belongs to a defective pixel or not; The defect detection method further includes determining a defective region in the image of the detected object based on defective pixels.
5. The defect detection method of claim 1, wherein fusing the first feature maps of the multiple layers of different resolutions, and inputting the fused feature maps into the first convolution layer for convolution processing, to obtain a first multi-category score map, comprises: Unifying the first feature images with different resolutions to the resolution of the first feature image with the largest resolution, and performing channel splicing to obtain a spliced feature image; And inputting the spliced feature map into the first convolution layer to carry out convolution processing to obtain a first multi-category score map.
6. The defect detection method of claim 5, wherein the utilizing the self-attention module to perform context information enhancement on the stitched feature map using the first multi-category score map to obtain the fused feature map comprises: performing target enhancement on the spliced feature images by using the first multi-category score images to obtain target enhanced feature images; And inputting the spliced characteristic diagram and the target enhanced characteristic diagram into the self-attention module to process a self-attention mechanism to obtain the fusion characteristic diagram, wherein the spliced characteristic diagram is used for forming a query matrix in the self-attention mechanism, and the target enhanced characteristic diagram is used for forming a key matrix and a value matrix in the self-attention mechanism.
7. The defect detection method of claim 6, wherein the performing target enhancement on the stitched feature map using the first multi-category score map to obtain a target enhanced feature map comprises: Performing shape transformation on the spliced feature images to form a two-dimensional matrix F, performing shape transformation on the first multi-category fractional images to form a two-dimensional matrix P, and performing matrix multiplication on the two-dimensional matrix F and the two-dimensional matrix P; multiplying the two-dimensional matrix F and the two-dimensional matrix P, and carrying out shape transformation on the multiplied result to restore the multiplied result into image data, thereby obtaining the target enhancement feature map.
8. The defect detection method of claim 6 wherein the self-attention module comprises a second convolution layer, a third convolution layer, a fourth convolution layer, and a fifth convolution layer, wherein inputting the stitching feature map and the target enhancement feature map into the self-attention module for self-attention mechanism processing to obtain the fusion feature map comprises: Reducing the channel number of the spliced feature map through the second convolution layer to obtain a second convolution feature map, reducing the channel number of the target enhancement feature map through the third convolution layer to obtain a third convolution feature map, and reducing the channel number of the target enhancement feature map through the fourth convolution layer to obtain a fourth convolution feature map; performing shape transformation on the second convolution feature map to form a query matrix M q , and performing shape transformation on the third convolution feature map to form a key matrix M k , wherein the query matrix M q and the key matrix M k are capable of matrix multiplication; Multiplying the query matrix M q by the key matrix M k , and performing Softmax processing on the multiplied result to obtain a relation matrix M r ; Performing shape transformation on the fourth convolution feature map to form a value matrix M v , wherein the value matrix M v and the relationship matrix M r are capable of performing matrix multiplication; Multiplying the value matrix M v by the relation matrix M r , performing shape transformation on the multiplied result to restore the multiplied result to image data with the same resolution as the spliced feature map, and restoring the multiplied result to the same channel number as the spliced feature map through the fifth convolution layer to obtain a fifth convolution feature map; and performing element-level addition operation on the fifth convolution feature map and the spliced feature map to obtain the fusion feature map.
9. The defect detection method of claim 1 wherein the defect segmentation result comprises a defect segmentation result map, wherein pixel values of pixels in the defect segmentation result map are used to represent whether corresponding pixels in an image of the inspected object belong to defective pixels; The step of performing pixel-level classification prediction by the classification head module by using the fusion feature map to obtain a defect segmentation result includes: inputting the fusion feature map into the sixth convolution layer for convolution processing, and then up-sampling to the resolution of the image of the detected object to obtain a second multi-category score map, wherein the number of channels of the second multi-category score map is the same as the number of preset category categories, and the numerical value of each channel at each pixel position in the second multi-category score map respectively represents the probability that the corresponding pixel in the image of the detected object belongs to each category, wherein at least one category is used for representing defects; And taking the classification category corresponding to the channel with the largest value as the final classification category for each pixel of the second multi-category score map, and obtaining the defect segmentation result map.
10. The defect detection method of claim 9 wherein the loss function of the defect segmentation model consists of a first loss function and a second loss function, the first loss function representing a difference between the defect segmentation result map and the defect segmentation label, the second loss function representing a difference between a first multi-class score map obtained by inputting the first sample image into the defect segmentation model and the defect segmentation label.
11. The defect detection method of claim 10, wherein the expression of the first loss function is: , Wherein IoU 1 represents the intersection ratio of the defect segmentation result graph and the defect segmentation label; the expression of the second loss function is: , IoU 2 represents the intersection ratio of the intermediate segmentation result and the defect segmentation label, wherein the intermediate segmentation result is obtained by upsampling a first multi-category score map to the resolution of the image of the detected object, and taking the category corresponding to the channel with the largest value from each pixel after upsampling; The expression of the loss function of the defect segmentation model is as follows: , Wherein the method comprises the steps of And Is a preset weight coefficient.
12. A defect detection apparatus, comprising: The image acquisition module is used for acquiring an image of the detected object; the defect segmentation module is used for inputting the image of the detected object into an encoder in a pre-trained defect segmentation model to perform multi-scale feature extraction to obtain a plurality of layers of first feature images with different resolutions, wherein the defect segmentation model further comprises a decoder and a classification head module, the decoder comprises a first convolution layer and a self-attention module, the plurality of layers of first feature images with different resolutions are fused, the fused feature images are input into the first convolution layer to perform convolution processing to obtain a first multi-class score image, the number of channels of the first multi-class score image is the same as the number of preset classification classes, at least one classification class is used for representing defects, the self-attention module is used for enhancing context information of the fused feature images to obtain a fused feature image, and the classification head module is used for performing pixel-level classification prediction by using the fused feature image to obtain a defect segmentation result; The training module is used for training the defect segmentation model and comprises the following steps: The sample acquisition sub-module is used for acquiring a first sample image set, wherein the first sample image set comprises a plurality of first sample images and corresponding defect segmentation labels, and the defect segmentation labels are used for indicating defect areas in the first sample images; the super-parameter searching sub-module is used for training a defect segmentation model by using the first sample image set for each super-parameter combination in a plurality of preset super-parameter combinations, and evaluating the defect segmentation performance of the trained defect segmentation model; recording the defect segmentation performance of a defect segmentation model trained by adopting each super-parameter combination, wherein in the process of training the defect segmentation model by utilizing the first sample image set, if the defect segmentation error of the defect segmentation model for any one first sample image is smaller than a first error threshold value, the first sample image is abandoned, and the training of subsequent rounds is not participated; And the optimization training sub-module is used for selecting a super-parameter combination corresponding to the trained defect segmentation model with the best defect segmentation performance as a final super-parameter combination, and training the defect segmentation model by adopting the final super-parameter combination to obtain a trained defect segmentation model.
13. A computer program product comprising a computer program and/or instructions which, when executed by a processor, implements the defect detection method of any of claims 1 to 11.

Description

Defect detection method and device, and super-parameter optimization method and device Technical Field The invention relates to the technical field of machine vision, in particular to a defect detection method and device and a super-parameter optimization method and device. Background In some image processing tasks such as defect detection, deep learning techniques have been increasingly used. For example, in the floor deployment of industrial defect detection, the defect segmentation technology based on the neural network has many advantages of fine pixel-by-pixel positioning, strong scene adaptability, high precision and the like, and is widely adopted by the market, and is generally used as the first-choice technology of industrial detection. In training the deep learning model, super parameters such as batch number, learning rate, optimizer, regularization and data enhancement mode need to be set. Along with the continuous development of technology, the determination of proper or optimal super-parameter combination becomes a key factor for restricting the improvement of the performance of the deep learning model, and improper super-parameter setting can lead to long training time and poor detection effect of the model. To obtain the optimal superparameter combination, superparameter optimization is required, and a superparameter automatic search tool is generally used, including grid search, random search, parameter space search based on Bayesian optimization and the like. In practical use, grid search and random search are performed, because of higher resolution of industrial images, the calculation cost of a deep learning network is huge, the whole optimization process is excessively long, the Bayesian optimization utilizes historical data to continuously adjust the search space of the super parameters, and can find a proper super parameter combination with relatively fewer attempts, but Bayesian optimization uses agent models such as Gaussian process regression, in deep learning, a large number of super parameters usually need to be optimized, and under high-dimensional parameter space, accurate approximation of the agent model is difficult to find, so that performance is reduced, and meanwhile, for the deep learning model, an objective function is usually not convex, and the agent models such as Gaussian process regression are difficult to fit, and if the deep learning model is directly used, the time consumption is further increased. Disclosure of Invention The technical problem to be solved by the invention is how to reduce the time consumption of the super-parameter optimization process. According to a first aspect, in one embodiment, a defect detection method is provided, including: acquiring an image of a detected object; inputting the image of the detected object into a pre-trained defect segmentation model to obtain a defect area in the image of the detected object; Wherein the defect segmentation model is trained by: Acquiring a first sample image set, wherein the first sample image set comprises a plurality of first sample images and corresponding defect segmentation labels, and the defect segmentation labels are used for indicating defect areas in the first sample images; For each super-parameter combination in a plurality of preset super-parameter combinations, training a defect segmentation model by using the first sample image set by adopting the super-parameter combination, and evaluating defect segmentation performance of the trained defect segmentation model; recording the defect segmentation performance of a defect segmentation model trained by adopting each super-parameter combination, wherein in the process of training the defect segmentation model by utilizing the first sample image set, if the defect segmentation error of the defect segmentation model for any one first sample image is smaller than a first error threshold value, the first sample image is abandoned, and the training of subsequent rounds is not participated; selecting a super-parameter combination corresponding to the trained defect segmentation model with the best defect segmentation performance as a final super-parameter combination, and training the defect segmentation model by adopting the final super-parameter combination to obtain a trained defect segmentation model. According to a second aspect, in one embodiment, there is provided a super parameter optimization method, including: Acquiring a sample image set, wherein the sample image set comprises a plurality of sample images and corresponding labels; For each super-parameter combination in a plurality of preset super-parameter combinations, training a target task model by using the sample image set by adopting the super-parameter combination, and evaluating performance indexes of the trained target task model when executing a target task; recording performance indexes when the target task model trained by each super-parameter combination is used for execut