Search

CN-122023303-A - Zero sample fabric surface defect detection method and related device

CN122023303ACN 122023303 ACN122023303 ACN 122023303ACN-122023303-A

Abstract

The invention belongs to an image processing field, and discloses a zero sample fabric surface defect detection method and a related device, wherein the method comprises the steps of obtaining a fabric surface image to be detected and a first text prompt set; and calling a pre-trained zero-sample fabric surface defect detection model according to the fabric surface image to be detected and the first text prompt set to obtain a fabric surface defect detection result to be detected, and adapting to and detecting the surface defect detection of the unseen category. The zero sample fabric surface defect detection model can generate self-adaptive offset vectors of each defect text in the training process, ensure that the model can accurately identify abnormality under the condition of uneven distribution, improve the generalization capability of detection and enhance the adaptability of the model to discrete distribution samples. In addition, the amplitude of the motion vector is regulated in a grading manner, so that the generated defect synthesis characteristics can reflect the strength difference in the defect text description, and the adaptability and detection precision of the model to different defect intensities are improved.

Inventors

  • LV CHENGKAN
  • CHEN QIYU

Assignees

  • 中国科学院自动化研究所

Dates

Publication Date
20260512
Application Date
20260122

Claims (16)

  1. 1. A method for detecting surface defects of a zero sample fabric, comprising: Acquiring a fabric surface image to be detected and a first text prompt set; Invoking a pre-trained zero sample fabric surface defect detection model according to the fabric surface image to be detected and the first text prompt set to obtain a fabric surface defect detection result to be detected; the pre-trained zero-sample fabric surface defect detection model is obtained by the following steps: Constructing a zero sample fabric surface defect detection model, wherein the zero sample fabric surface defect detection model comprises a visual mapping layer and a text mapping layer; The method comprises the steps of obtaining a plurality of normal fabric surface images and a second text prompt set, carrying out iteration training until a preset termination condition is met, obtaining a pre-trained zero sample fabric surface defect detection model, wherein the training step comprises the steps of extracting image features of the normal fabric surface images through a visual mapping layer, mapping the image features to a shared feature space to obtain mapped image features, extracting text features of the normal texts and the defect texts in the second text prompt set through a text mapping layer, mapping the text features to the shared feature space to obtain mapped text features of the normal texts and the defect texts, obtaining and generating self-adaptive offset vectors of the defect texts by combining the mapped image features and the mapped text features of the defect texts based on discrete information of the mapped image features, obtaining and adjusting the intensity of the self-adaptive offset vectors according to defect description intensity levels of the defect texts, obtaining adjusted offset vectors of the defect texts, superposing the mapped image features to generate defect synthesis features corresponding to the defect texts, obtaining similarity of the mapped image features and the mapped text features of the normal texts and the defect texts, and optimizing the zero sample fabric surface defect detection model according to the similarity.
  2. 2. The method of claim 1, wherein the visual mapping layer comprises an image encoder and a first learnable linear layer connected in sequence, and the text mapping layer comprises a text encoder and a second learnable linear layer connected in sequence.
  3. 3. The method for detecting surface defects of a zero-sample fabric according to claim 1, wherein the discrete information of the mapped image features is obtained by: Acquiring a preset number of normal fabric surface images; Extracting image features of a preset number of normal fabric surface images through the visual mapping layer and mapping the image features to a shared feature space to obtain mapped image features of the preset number of normal fabric surface images; acquiring the average value of the mapping image features of the surface images of the normal fabric in a preset number to obtain a center feature; And acquiring covariance matrixes of mapping image features of a preset number of normal fabric surface images according to the central features, and taking the covariance matrixes as mapping image feature discrete information.
  4. 4. The method of claim 1, wherein the obtaining and generating an adaptive offset vector for each defect text based on the mapped image feature discrete information and in combination with the mapped image feature and the mapped text feature for each defect text comprises: obtaining a direction vector of mapping text features of the mapping image features pointing to each defect text; Obtaining the dispersion of each feature dimension of the mapping image features according to the mapping image feature dispersion information; according to the dispersion degree of each characteristic dimension of the mapping image characteristic, adjusting the weight coefficient of each characteristic dimension corresponding to the direction vector, wherein the dispersion degree is inversely related to the weight coefficient; And adjusting the direction vector according to the weight coefficient to obtain the self-adaptive offset vector of each defect text.
  5. 5. The method of claim 1, wherein the obtaining and adjusting the intensity of the adaptive offset vector according to the defect description intensity level of each defect text comprises: Acquiring defect description intensity levels of each defect text; Converting the defect description intensity level of each defect text into standard deviation parameters of Gaussian noise, wherein the higher the defect description intensity level is, the larger the standard deviation parameters of Gaussian noise are; And generating Gaussian distribution random noise vectors of the defect texts according to the standard deviation parameters of the Gaussian noise of the defect texts, and superposing the Gaussian distribution random noise vectors to the adaptive offset vectors of the defect texts.
  6. 6. The method for detecting surface defects of a zero-sample fabric according to claim 1, wherein the steps of obtaining similarities between the mapped image features and the defect synthesis features and the mapped text features of the normal text and each defect text, and optimizing the zero-sample fabric surface defect detection model according to the similarities comprise: Obtaining a first probability distribution of the mapping image features belonging to the normal text and each defect text according to the similarity of the mapping image features to the mapping text features of the normal text and each defect text; Obtaining second probability distribution of the defect synthesis characteristics belonging to the normal text and each defect text according to the similarity of the defect synthesis characteristics of the normal text and the mapping text characteristics of each defect text; And optimizing the zero-sample fabric surface defect detection model according to the first probability distribution and the second probability distribution and combining the cross entropy loss function.
  7. 7. The method for detecting surface defects of a fabric with zero sample according to claim 1, wherein the step of calling a pre-trained zero sample fabric surface defect detection model according to the fabric surface image to be detected and the first text prompt set to obtain a fabric surface defect detection result to be detected comprises the following steps: Invoking a pre-trained zero sample fabric surface defect detection model according to the fabric surface image to be detected and the first text prompt set to obtain the mapping image characteristics of the fabric surface image to be detected and the mapping text characteristics of the normal text and each defect text in the first text prompt set; Obtaining the similarity between the mapping image characteristics of the fabric surface image to be detected and the mapping text characteristics of the normal text and each defect text in the first text prompt set, and obtaining the text with the highest similarity; When the text with the highest similarity is a defect text, the surface defect detection result of the fabric to be detected is abnormal, and the abnormal type is the defect type corresponding to the current defect text.
  8. 8. A zero sample fabric surface defect detection system, comprising: the data acquisition module is used for acquiring the surface image of the fabric to be detected and a first text prompt set; The defect detection module is used for calling a pre-trained zero-sample fabric surface defect detection model according to the fabric surface image to be detected and the first text prompt set to obtain a fabric surface defect detection result to be detected; the pre-trained zero-sample fabric surface defect detection model is obtained by the following steps: Constructing a zero sample fabric surface defect detection model, wherein the zero sample fabric surface defect detection model comprises a visual mapping layer and a text mapping layer; The method comprises the steps of obtaining a plurality of normal fabric surface images and a second text prompt set, carrying out iteration training until a preset termination condition is met, obtaining a pre-trained zero sample fabric surface defect detection model, wherein the training step comprises the steps of extracting image features of the normal fabric surface images through a visual mapping layer, mapping the image features to a shared feature space to obtain mapped image features, extracting text features of the normal texts and the defect texts in the second text prompt set through a text mapping layer, mapping the text features to the shared feature space to obtain mapped text features of the normal texts and the defect texts, obtaining and generating self-adaptive offset vectors of the defect texts by combining the mapped image features and the mapped text features of the defect texts based on discrete information of the mapped image features, obtaining and adjusting the intensity of the self-adaptive offset vectors according to defect description intensity levels of the defect texts, obtaining adjusted offset vectors of the defect texts, superposing the mapped image features to generate defect synthesis features corresponding to the defect texts, obtaining similarity of the mapped image features and the mapped text features of the normal texts and the defect texts, and optimizing the zero sample fabric surface defect detection model according to the similarity.
  9. 9. The zero sample fabric surface defect detection system of claim 8, wherein the visual mapping layer comprises an image encoder and a first learnable linear layer connected in sequence, and wherein the text mapping layer comprises a text encoder and a second learnable linear layer connected in sequence.
  10. 10. The zero sample fabric surface defect detection system of claim 8, wherein the mapped image feature discrete information is obtained by: Acquiring a preset number of normal fabric surface images; Extracting image features of a preset number of normal fabric surface images through the visual mapping layer and mapping the image features to a shared feature space to obtain mapped image features of the preset number of normal fabric surface images; acquiring the average value of the mapping image features of the surface images of the normal fabric in a preset number to obtain a center feature; And acquiring covariance matrixes of mapping image features of a preset number of normal fabric surface images according to the central features, and taking the covariance matrixes as mapping image feature discrete information.
  11. 11. The zero sample fabric surface defect detection system of claim 8, wherein the obtaining and generating an adaptive offset vector for each defect text in combination with the mapped image features and the mapped text features for each defect text based on the mapped image feature discrete information comprises: obtaining a direction vector of mapping text features of the mapping image features pointing to each defect text; Obtaining the dispersion of each feature dimension of the mapping image features according to the mapping image feature dispersion information; according to the dispersion degree of each characteristic dimension of the mapping image characteristic, adjusting the weight coefficient of each characteristic dimension corresponding to the direction vector, wherein the dispersion degree is inversely related to the weight coefficient; And adjusting the direction vector according to the weight coefficient to obtain the self-adaptive offset vector of each defect text.
  12. 12. The zero sample fabric surface defect detection system of claim 8, wherein the acquiring and adjusting the intensity of the adaptive offset vector based on the defect description intensity level for each defect text comprises: Acquiring defect description intensity levels of each defect text; Converting the defect description intensity level of each defect text into standard deviation parameters of Gaussian noise, wherein the higher the defect description intensity level is, the larger the standard deviation parameters of Gaussian noise are; And generating Gaussian distribution random noise vectors of the defect texts according to the standard deviation parameters of the Gaussian noise of the defect texts, and superposing the Gaussian distribution random noise vectors to the adaptive offset vectors of the defect texts.
  13. 13. The zero sample fabric surface defect detection system of claim 8, wherein the obtaining the similarity of the mapped image features and the defect synthesis features to the mapped text features of the normal text and each of the defect texts, and optimizing the zero sample fabric surface defect detection model based on the similarity comprises: Obtaining a first probability distribution of the mapping image features belonging to the normal text and each defect text according to the similarity of the mapping image features to the mapping text features of the normal text and each defect text; Obtaining second probability distribution of the defect synthesis characteristics belonging to the normal text and each defect text according to the similarity of the defect synthesis characteristics of the normal text and the mapping text characteristics of each defect text; And optimizing the zero-sample fabric surface defect detection model according to the first probability distribution and the second probability distribution and combining the cross entropy loss function.
  14. 14. The zero sample fabric surface defect detection system of claim 8, wherein the invoking the pre-trained zero sample fabric surface defect detection model based on the fabric surface image to be detected and the first set of text cues to obtain the surface defect detection result for the fabric to be detected comprises: Invoking a pre-trained zero sample fabric surface defect detection model according to the fabric surface image to be detected and the first text prompt set to obtain the mapping image characteristics of the fabric surface image to be detected and the mapping text characteristics of the normal text and each defect text in the first text prompt set; Obtaining the similarity between the mapping image characteristics of the fabric surface image to be detected and the mapping text characteristics of the normal text and each defect text in the first text prompt set, and obtaining the text with the highest similarity; When the text with the highest similarity is a defect text, the surface defect detection result of the fabric to be detected is abnormal, and the abnormal type is the defect type corresponding to the current defect text.
  15. 15. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the zero sample fabric surface defect detection method according to any one of claims 1 to 7 when the computer program is executed.
  16. 16. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the steps of the zero sample fabric surface defect detection method according to any one of claims 1 to 7.

Description

Zero sample fabric surface defect detection method and related device Technical Field The invention belongs to the field of image processing, and relates to a zero sample fabric surface defect detection method and a related device. Background Under the current globalization market environment, the traditional production advantages gradually disappear, and higher requirements are put on technical innovation and production efficiency. The product quality directly determines the market competitiveness and economic benefit of enterprises, and especially in the traditional industries of textile manufacturing and the like, the detection of the surface defects of the fabrics is very important. These defects are often caused by mechanical failure, mishandling or mishandling of materials, etc., and are often tiny and difficult to identify with the naked eye. Fabric defect detection has long relied primarily on artificial vision. However, this method is not only inefficient, but also limited in accuracy by subjective judgment and fatigue of operators, severely affecting product quality and productivity. In recent years, deep learning-based defect detection technology is becoming a mainstream solution to this problem, especially in the field of textile surface defect detection. Although these techniques can significantly improve the efficiency and accuracy of detection, most methods still rely on supervised learning, requiring a large number of annotated defect images to construct the training set. Labeling is not only time-consuming and labor-consuming, but also requires high costs, especially in the face of rapidly changing and diversified fabric defects, existing training sets often fail to cover all possible defect types, resulting in limited model adaptability and generalization capabilities. In addition, along with diversification of market demands and continuous occurrence of novel fabric defects, the existing supervised learning method gradually exposes limitations, and particularly when the defects of unknown categories are faced, the existing supervised learning model is difficult to adapt quickly, so that defect recognition results are poor. Disclosure of Invention The invention aims to overcome the defects of the prior art and provide a zero-sample fabric surface defect detection method and a related device. In order to achieve the purpose, the invention is realized by adopting the following technical scheme: The invention provides a zero sample fabric surface defect detection method, which comprises the steps of obtaining a fabric surface image to be detected and a first text prompt set; the method comprises the steps of calling a pre-trained zero sample fabric surface defect detection model according to a fabric surface image to be detected and a first text prompt set to obtain a surface defect detection result of the fabric to be detected, wherein the pre-trained zero sample fabric surface defect detection model is obtained by constructing the zero sample fabric surface defect detection model, the zero sample fabric surface defect detection model comprises a visual mapping layer and a text mapping layer, acquiring a plurality of normal fabric surface images and a second text prompt set, iterating a training step until a preset termination condition is met to obtain the pre-trained zero sample fabric surface defect detection model, extracting image features of the normal fabric surface images through the visual mapping layer and mapping the image features to a shared feature space to obtain mapped image features, extracting text features of the normal texts and the defect texts in the second text prompt set through the text mapping layer and mapping the text features to the shared feature space to obtain mapped text features of the normal texts and the defect texts, acquiring and generating adaptive offset vectors of the defect texts based on the mapped image feature discrete information of the mapped image features and the mapping text features, acquiring and adjusting the adaptive offset vectors of the defect texts according to the defect intensity level description of the defect text to obtain the adaptive offset vectors of the defect text, obtaining the mapped text feature vectors and the mapped feature vectors and the corresponding mapped feature vectors and the mapped feature vectors, obtaining the mapped feature vectors and the corresponding feature vectors, and optimizing a zero sample fabric surface defect detection model according to the similarity. Optionally, the visual mapping layer comprises an image encoder and a first learnable linear layer which are sequentially connected, and the text mapping layer comprises a text encoder and a second learnable linear layer which are sequentially connected. The method comprises the steps of obtaining the characteristic discrete information of the mapping image, obtaining the characteristic discrete information of the mapping image by obtaining the norm