CN-116486070-B - Pavement crack detection method and device, electronic equipment and storage medium

CN116486070BCN 116486070 BCN116486070 BCN 116486070BCN-116486070-B

Abstract

The application provides a pavement crack detection method, a pavement crack detection device, a storage medium and electronic equipment, wherein the pavement crack detection method comprises the steps of constructing a segmentation model by using a preset first encoder and a preset second encoder, constructing an evaluation model by using a preset first crack image for training, alternately training the segmentation model and the evaluation model to obtain a trained segmentation model, pre-marking actual pixel-level crack marks on each first crack image, inputting a plurality of pavement images to be marked into the trained segmentation model, performing pixel-level image semantic segmentation on each pavement image to obtain a plurality of crack mark images containing pixel-level crack marks, and determining pavement cracks in the crack mark images.

Inventors

XU GUOSHENG
XU GUOAI
HU JUNFANG

Assignees

北京邮电大学

Dates

Publication Date: 20260505
Application Date: 20230307

Claims (10)

1. A pavement crack detection method, characterized by comprising: constructing a segmentation model by using a preset first encoder and a decoder, and constructing an evaluation model by using a preset second encoder; alternately training the segmentation model and the evaluation model by using a plurality of preset first crack images for training to obtain a trained segmentation model, wherein each first crack image is pre-marked with an actual pixel-level crack mark; Inputting a plurality of pavement images to be marked into the trained segmentation model, carrying out pixel-level image semantic segmentation on each pavement image to obtain a plurality of crack mark images containing pixel-level crack marks, and determining pavement cracks in the crack mark images; The method for training the segmentation model and the evaluation model by using a plurality of preset first crack images for training alternately trains the segmentation model and the evaluation model to obtain a trained segmentation model, which specifically comprises the following steps: The method comprises the steps that current first model parameters of a fixed segmentation model are unchanged, a plurality of input first crack images are predicted by the aid of the current segmentation model, and a prediction result of each first crack image is obtained, wherein the prediction result comprises a second crack image with pixel-level crack characteristics; the loss function is set to the form shown below: ; where L represents the feature loss, N represents the number of first crack images input into the segmentation model, Represents the average absolute error of the prediction results of the segmentation model, Representing the pixel distribution of the nth first crack image, Representing the marker distribution predicted by the segmentation model, Representing the actual marker distribution of the nth first fracture image, Then the multi-level features extracted by the evaluation model are represented; Wherein, the ; Wherein, the Representing the feature distribution extracted by the evaluation model from the ith layer in the predicted result, Representing real characteristic distribution preset in a training image; based on the calculated multi-scale feature loss, back-propagating the multi-scale feature loss, and adjusting the gradient of the evaluation model; according to the adjustment of the gradient, determining a second model parameter of the evaluation model when the multi-scale feature loss is maximized, and taking the second model parameter of the evaluation model when the multi-scale feature loss is maximized as a second target parameter; Fixing a second target parameter in the evaluation model, inputting the first crack image into the pavement crack detection model again, predicting the training image again by using the segmentation model, and obtaining a prediction result, wherein the prediction result is obtained based on the current first model parameter of the segmentation model; Based on the prediction result, the predicted multi-scale feature loss is determined by using the trained second target parameters and the loss function in the evaluation model, and is counter-propagated, so that a first model parameter when the multi-scale feature loss is minimized is determined, and the first model parameter of the multi-scale feature loss is minimized and used as the first target parameter.
2. The method of claim 1, wherein prior to alternately training the segmentation model and the evaluation model using the preset plurality of first fracture images for training, further comprising: Performing crack prediction on a plurality of preset training images by utilizing a pre-trained dense connection convolutional neural network to obtain a plurality of initial crack images containing block-level crack marks; carrying out histogram equalization, median filtering, image gray level logarithmic transformation and morphological open operation treatment on each initial crack image; And converting each processed initial crack image into a binary image according to a preset dynamic threshold value, and obtaining a plurality of first crack images containing pixel-level crack marks by using a pixel-level crack extraction algorithm based on edge detection.
3. The method of claim 1, wherein constructing the partition model using the preset first encoder and decoder comprises: Setting a step size of a convolution layer in the first encoder to 2 and up-sampling using transpose convolution in the decoder; A pilot filter is provided in both the first encoder and the decoder and an attention mechanism is added in the decoder.
4. The method of claim 1, wherein constructing an evaluation model using a preset second encoder comprises: setting a step size of a convolution layer in the second encoder to 2; And a loss function for evaluating the segmentation model and the evaluation model itself is set.
5. The method of claim 4, wherein the alternately training the segmentation model and the evaluation model comprises: Inputting the preset first crack images into the segmentation model and the evaluation model; Fixing the current first model parameters of the segmentation model, training the evaluation model, and determining second target parameters of the evaluation model, wherein the second target parameters are used for updating the current second model parameters of the evaluation model; And fixing the current second target parameters of the evaluation model, training the segmentation model, and determining first target parameters of the segmentation model, wherein the first target parameters are used for updating the first model parameters.
6. The method of claim 5, wherein the fixing the current first model parameters of the segmentation model, training the evaluation model, comprises: Fixing current first model parameters in the segmentation model, and predicting the plurality of first crack images by using the segmentation model to obtain a prediction result based on the current first model parameters; causing the evaluation model to evaluate a feature loss between the predicted result and the actual pixel-level crack marker using the loss function as shown below: wherein L represents the feature loss, N represents the number of first crack images input into the segmentation model, Representing the average absolute error of the prediction results of the segmentation model, Representing the pixel distribution of the nth first crack image, A marker distribution representing an image predicted by the segmentation model, A pixel-level marker distribution representing the actual of the nth first crack image; And back-propagating the characteristic loss, adjusting the second model parameter, and determining the second model parameter of the characteristic loss when the characteristic loss is maximized as a second target parameter of the evaluation model.
7. The method of claim 6, wherein the training the segmentation model with the second target parameters that fix the current evaluation model unchanged comprises: fixing the second target parameters in the evaluation model, and predicting the plurality of first crack images by using the segmentation model to obtain a prediction result based on the first model parameters; causing the evaluation model to evaluate a feature loss between the predicted result and the actual pixel-level crack marker using the loss function: back-propagating the feature loss and adjusting the first model parameter to determine the first model parameter at which the feature loss is minimized as a first target parameter of the segmentation model.
8. The pavement crack detection device is characterized by comprising a model construction module, a model training module and a model prediction module; The model construction module is configured to construct a segmentation model by using a preset first encoder and a decoder and construct an evaluation model by using a preset second encoder; The model training module is configured to alternately train the segmentation model and the evaluation model by using a plurality of preset first crack images for training to obtain a trained segmentation model, and each first crack image is pre-marked with an actual pixel-level crack mark; The model prediction module is configured to input a plurality of pavement images to be marked into the trained segmentation model, perform pixel-level image semantic segmentation on each pavement image to obtain a plurality of crack mark images containing pixel-level crack marks, and determine pavement cracks in the crack mark images; the model training module specifically comprises: The method comprises the steps that current first model parameters of a fixed segmentation model are unchanged, a plurality of input first crack images are predicted by the aid of the current segmentation model, and a prediction result of each first crack image is obtained, wherein the prediction result comprises a second crack image with pixel-level crack characteristics; the loss function is set to the form shown below: ; where L represents the feature loss, N represents the number of first crack images input into the segmentation model, Represents the average absolute error of the prediction results of the segmentation model, Representing the pixel distribution of the nth first crack image, Representing the marker distribution predicted by the segmentation model, Representing the actual marker distribution of the nth first fracture image, Then the multi-level features extracted by the evaluation model are represented; Wherein, the ; Wherein, the Representing the feature distribution extracted by the evaluation model from the ith layer in the predicted result, Representing real characteristic distribution preset in a training image; based on the calculated multi-scale feature loss, back-propagating the multi-scale feature loss, and adjusting the gradient of the evaluation model; according to the adjustment of the gradient, determining a second model parameter of the evaluation model when the multi-scale feature loss is maximized, and taking the second model parameter of the evaluation model when the multi-scale feature loss is maximized as a second target parameter; Fixing a second target parameter in the evaluation model, inputting the first crack image into the pavement crack detection model again, predicting the training image again by using the segmentation model, and obtaining a prediction result, wherein the prediction result is obtained based on the current first model parameter of the segmentation model; Based on the prediction result, the predicted multi-scale feature loss is determined by using the trained second target parameters and the loss function in the evaluation model, and is counter-propagated, so that a first model parameter when the multi-scale feature loss is minimized is determined, and the first model parameter of the multi-scale feature loss is minimized and used as the first target parameter.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable by the processor, wherein the processor implements the method of any one of claims 1 to 7 when executing the computer program.
10. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method of any one of claims 1 to 7.

Description

Pavement crack detection method and device, electronic equipment and storage medium Technical Field The embodiment of the application relates to the technical field of image recognition, in particular to a pavement crack detection method, a pavement crack detection device, electronic equipment and a storage medium. Background On one hand, due to the restriction of labor cost, time cost and the like, the related pavement crack detection mode basically uses the identified crack images based on block-level marks, and the granularity of the block-level marks is too coarse, so that the improvement of the automatic pavement crack detection precision is fundamentally limited. On the other hand, the road surface condition is influenced by climate, traffic flow and the like, cracks are various, such as shallow cracks, crazes and network cracks, and interference factors are various, such as lane lines, stains and illumination are uneven, so that the difficulty of crack detection is greatly improved, the recognition effect of a road surface crack detection model obtained based on data training in an actual application scene on fine cracks is poor, and the omission recognition rate is high. Based on this, a solution capable of achieving finer granularity of pavement crack detection is required. Disclosure of Invention In view of the above, an object of the present application is to provide a method, an apparatus, an electronic device, and a storage medium for detecting a road surface crack. Based on the above object, the present application provides a pavement crack detection method, comprising: constructing a segmentation model by using a preset first encoder and a decoder, and constructing an evaluation model by using a preset second encoder; alternately training the segmentation model and the evaluation model by using a plurality of preset first crack images for training to obtain a trained segmentation model, wherein each first crack image is pre-marked with an actual pixel-level crack mark; Inputting a plurality of pavement images to be marked into the trained segmentation model, carrying out pixel-level image semantic segmentation on each pavement image to obtain a plurality of crack mark images containing pixel-level crack marks, and determining pavement cracks in the crack mark images. Further, before the segmentation model and the evaluation model are alternately trained by using a plurality of preset first fracture images for training, the method further includes: Performing crack prediction on a plurality of preset training images by utilizing a pre-trained dense connection convolutional neural network to obtain a plurality of initial crack images containing block-level crack marks; carrying out histogram equalization, median filtering, image gray level logarithmic transformation and morphological open operation treatment on each initial crack image; And converting each processed initial crack image into a binary image according to a preset dynamic threshold value, and obtaining a plurality of first crack images containing pixel-level crack marks by using a pixel-level crack extraction algorithm based on edge detection. Further, constructing a segmentation model using a preset first encoder and decoder, comprising: Setting a step size of a convolution layer in the first encoder to 2 and up-sampling using transpose convolution in the decoder; a pilot filter is added in both the first encoder and the decoder and an attention mechanism is set in the decoder. Further, constructing an evaluation model using a preset second encoder, comprising: setting a step size of a convolution layer in the second encoder to 2; And a loss function for evaluating the segmentation model and the evaluation model itself is set. Further, alternately training the segmentation model and the evaluation model includes: Inputting the preset first crack images into the segmentation model and the evaluation model; Fixing the current first model parameters of the segmentation model, training the evaluation model, and determining second target parameters of the evaluation model, wherein the second target parameters are used for updating the current second model parameters of the evaluation model; And fixing the current second target parameters of the evaluation model, training the segmentation model, and determining first target parameters of the segmentation model, wherein the first target parameters are used for updating the first model parameters. Further, fixing the current first model parameter of the segmentation model, training the evaluation model, including: Fixing current first model parameters in the segmentation model, and predicting the plurality of first crack images by using the segmentation model to obtain a prediction result based on the current first model parameters; causing the evaluation model to evaluate a feature loss between the predicted result and the actual pixel-level crack marker using the loss function as shown