CN-121661430-B - Sewer pipe network defect detection method based on multi-label zero sample learning

CN121661430BCN 121661430 BCN121661430 BCN 121661430BCN-121661430-B

Abstract

The invention discloses a sewer pipe network defect detection method based on multi-label zero sample learning, which comprises the steps of generating defect descriptions corresponding to each pipeline defect type through a large language model, carrying out feature extraction and field adaptation on pipeline inner wall images and the defect descriptions by utilizing a representation guiding module to obtain global and local image features and defect detailed description text features, integrating the global and local image features of an image, respectively calculating global and local matching scores of the global and local image features and the detailed description text features, fusing the global and local image features to obtain initial prediction scores of defects, constructing a semantic relation adjacent matrix for displaying relations among different defect types, and correcting the initial prediction scores by utilizing the semantic relation adjacent matrix among the types to obtain final prediction scores of each defect type. The method provided by the invention realizes knowledge migration from known defects to unknown defects by constructing a guide-fusion-correction network, and effectively solves the problem of identifying the types of the unseen defects.

Inventors

CHEN ZHAOMIN
HUANG XINJIAN
GE YISU
CHEN LIYAN
ZHANG GUODAO
Guo Jialuo

Assignees

温州大学大数据与信息技术研究院
温州大学

Dates

Publication Date: 20260508
Application Date: 20260206

Claims (7)

1. A sewer pipe network defect detection method based on multi-label zero sample learning is characterized by comprising the following steps: Step S1, generating defect descriptions corresponding to each pipeline defect category through a large language model; s2, performing feature extraction and field adaptation on the pipeline inner wall image and the defect description by using a representation guiding module to acquire global and local image features and defect detailed description text features; S3, integrating global and local image features of the image, respectively calculating global and local matching scores of the global and local image features and detailed description text features, and fusing to obtain initial defect prediction scores; S4, constructing a semantic relation adjacency matrix for displaying the relation between different defect categories by using the generated defect brief phrase; S5, correcting the initial prediction scores by using semantic relation adjacency matrixes among the categories to obtain final prediction scores of each defect category; The step S3 specifically comprises the following steps: Step S31, calculating global matching score by global image feature and defect detail description text feature ; In the formula, Is a global image feature; describing text features in detail for the defect; is a transpose operator; step S32, calculating the affinity weight of the local image features and the detailed defect description text features, and obtaining local matching scores through score aggregation network aggregation ; In the formula, Is affinity weight; Is a normalization function; is a local image feature; t is a transpose operator; step S33, the global matching score and the local matching score are aggregated to obtain an initial prediction score; in the formula, Is the initial predictive score.
2. The method for detecting the defects of the sewer pipe network based on the multi-label zero sample learning according to claim 1, wherein the step S1 specifically comprises the following steps: step S11, generating defect detailed description corresponding to each pipeline defect category And brief phrases ; Step S12, using the text encoder to make a brief phrase Coding to obtain the text features of the brief phrases of all defect categories 。
3. The method for detecting the defects of the sewer pipe network based on the multi-label zero sample learning according to claim 1, wherein the step S2 specifically comprises the following steps: Step S21, imaging the inner wall of the pipeline Detailed description of defects in pipeline defect descriptions Respectively inputting an image encoder and a text encoder to respectively obtain original image characteristics and original text characteristics; Step S22, performing sewer pipeline field adaptation on the original image features and the original text features by using the representation guiding module to obtain adapted global image features Local image features And defect detailed description text feature 。
4. A method for detecting a defect in a sewer pipe network based on multi-label zero sample learning according to claim 3, wherein in step S22, the guiding module corrects the characteristics outputted by the transducer encoder by linear transformation, geLU activation function, dropout operation and residual connection, and the calculation is as follows: in the formula, Features derived by representing guidance; for the original image features obtained by the encoder, For a given input feature; Adapting image and text features of the pipeline scene; And Is a linear transformation layer; Is a nonlinear activation function; To discard operations.
5. The method for detecting the defects of the sewer pipe network based on the multi-label zero sample learning according to claim 1, wherein the step S4 specifically comprises the following steps: Step S41, using the brief phrase text features in the pipeline defect description Calculating cosine similarity matrixes among defect categories, filtering through a preset threshold value, and constructing an initial semantic relation adjacency matrix ; In the formula, Is a cosine similarity matrix; To brief phrase text features Solving for T is the transpose operator; For cosine similarity matrix Middle (f) Line 1 Similarity value of columns; A preset similarity threshold value is set; adjacency matrix for initial semantic relationship Middle (f) Line 1 Values of columns; , , Is the total number of defect categories; step S42, processing the initial semantic relation adjacency matrix, and calculating to obtain the final semantic relation adjacency matrix ; In the formula, Is a unit matrix; a semantic relation adjacency matrix for adding self-connection; is a diagonal matrix, the diagonal elements of which satisfy 。
6. The method for detecting defects of a sewer pipe network based on multi-label zero sample learning according to claim 1, wherein the step S5 specifically comprises the following steps: step S51, based on the initial prediction score and the semantic relation adjacency matrix, predicting and correcting the score by using the graph convolution network ; In the formula, Is a semantic relation adjacency matrix; Is a nonlinear activation function; Is an initial predictive score; And Is a weight that can be learned; step S52, the initial prediction score and the correction score are aggregated to obtain a final prediction score of each defect class; in the formula, Is the final predictive score.
7. The method for detecting the defects of the sewer pipe network based on the multi-label zero sample learning according to claim 1, wherein in the model training process, a loss function combining contrast learning and sorting constraint is used for model optimization, and the method comprises the following steps: step S61, calculating a multi-mode contrast loss function ; In the formula, An index set for all positive sample pairs within the same batch, ; Total number for all positive sample pairs; For the sample index of the positive sample pair, ; Is the number of samples; For the class index of the positive sample pair, ; Is the category number; For indicating the first Whether or not the sample contains the first A defect class; Is an exponential function; Is natural logarithm; Represent the first The first sample is at Final predictive score on individual defect categories; A temperature scaling factor for controlling the contrast learning intensity; Represent the first The first sample is at Final predictive score on individual defect categories; in order to traverse the sample indices of all samples, ; In order to traverse the category index for all categories, ; Step S62, calculating the sorting loss function ; In the formula, Index sets for all positive sample pairs in the same batch; total number for all positive sample pairs; for the sample On its positive label A predictive score on; for the sample The highest predictive score on all negative labels; minimum spacing between positive label and negative label of highest predictive score; Calculating the maximum value; For the index of the sample, ; Is a positive category index; step S63, comparing the multi-mode contrast loss function And a ranking loss function Weighted combination, forming a total loss function for model training Optimizing all the learnable parameters through a gradient descent algorithm; in the formula, Is a loss balance factor.

Description

Sewer pipe network defect detection method based on multi-label zero sample learning Technical Field The invention relates to the technical field of image processing, in particular to a sewer pipe network defect detection method based on multi-label zero sample learning. Background The accurate detection and identification of the pipeline defects are the basis for pipeline health assessment and repair decision. However, the current detection method based on supervised learning has two major bottlenecks, namely, firstly, the model is limited by a training data set, the identification capability of the model on unknown defect types which appear continuously in practice is poor, the detection omission risk exists, secondly, the model updating and maintenance cost is high, the data acquisition, the labeling and the training are required to be repeatedly carried out for covering new defects, and the operation and maintenance requirements of dynamic changes are difficult to adapt. There is therefore a need to propose a new solution to the above-mentioned problems. Disclosure of Invention The invention aims to overcome the defects of the prior art and provides a sewer pipe network defect detection method based on multi-label zero sample learning. In order to achieve the above purpose, the present invention adopts the following technical scheme: a sewer pipe network defect detection method based on multi-label zero sample learning comprises the following steps: Step S1, generating defect descriptions corresponding to each pipeline defect category through a large language model; s2, performing feature extraction and field adaptation on the pipeline inner wall image and the defect description by using a representation guiding module to acquire global and local image features and defect detailed description text features; S3, integrating global and local image features of the image, respectively calculating global and local matching scores of the global and local image features and detailed description text features, and fusing to obtain initial defect prediction scores; S4, constructing a semantic relation adjacency matrix for displaying the relation between different defect categories by using the generated defect brief phrase; and S5, correcting the initial prediction scores by using the semantic relation adjacency matrix among the categories to obtain the final prediction score of each defect category. Further, the step S1 specifically includes the following steps: step S11, generating defect detailed description corresponding to each pipeline defect category And brief phrases; Step S12, using the text encoder to make a brief phraseCoding to obtain the text features of the brief phrases of all defect categories。 Further, the step S2 specifically includes the following steps: Step S21, imaging the inner wall of the pipeline Detailed description of defects in pipeline defect descriptionsRespectively inputting an image encoder and a text encoder to respectively obtain original image characteristics and original text characteristics; Step S22, performing sewer pipeline field adaptation on the original image features and the original text features by using the representation guiding module to obtain adapted global image features Local image featuresAnd defect detailed description text feature。 Further, in step S22, the guiding module corrects the characteristics of the output of the transducer encoder by linear transformation, geLU activation functions, dropout operation and residual connection, and is calculated as follows: in the formula, Features derived by representing guidance; for the original image features obtained by the encoder, For a given input feature; Adapting image and text features of the pipeline scene; And Is a linear transformation layer; Is a nonlinear activation function; To discard operations. Further, the step S3 specifically includes the following steps: Step S31, calculating global matching score by global image feature and defect detail description text feature ; In the formula,Is a global image feature; describing text features in detail for the defect; is a transpose operator; step S32, calculating the affinity weight of the local image features and the detailed defect description text features, and obtaining local matching scores through score aggregation network aggregation ; In the formula,Is affinity weight; Is a normalization function; is a local image feature; t is a transpose operator; step S33, the global matching score and the local matching score are aggregated to obtain an initial prediction score; in the formula, Is the initial predictive score. Further, the step S4 specifically includes the following steps: Step S41, using the brief phrase text features in the pipeline defect description Calculating cosine similarity matrixes among defect categories, filtering through a preset threshold value, and constructing an initial semantic relation adjacency matrix; In the formula,Is