CN-122024040-A - Pavement disease image generation and identification method based on residual depth generation countermeasure network

CN122024040ACN 122024040 ACN122024040 ACN 122024040ACN-122024040-A

Abstract

The invention discloses a pavement defect image generation and identification method based on a residual depth generation countermeasure network, which comprises the steps of constructing a residual depth generation countermeasure network, training by using a hole defect image in an original pavement defect image as training data, outputting a pavement hole defect generation image based on RESWPGANET after training, seamlessly embedding the pavement hole defect generation image into a target background image by using a Poisson fusion algorithm to obtain a pavement hole defect synthetic image which is consistent with illumination of the pavement background image and has true texture, wherein RESWPGANET comprises a generator and a discriminator, the generator comprises a plurality of cascaded residual upsampling modules ReBlockG, and a plurality of cascaded residual upsampling modules ReBlockG are used for expanding resolution of an initial feature image step by step.

Inventors

HE GUOTAO
FAN JIFEI
LIU XIAONA
JIN XIAONIU

Assignees

陕西高速电子工程有限公司

Dates

Publication Date: 20260512
Application Date: 20251231

Claims (10)

1. The pavement disease image generation method based on the residual depth generation countermeasure network is characterized by comprising the following steps of: constructing a residual depth generation countermeasure network RESWPGANET, and training RESWPGANET by using a pothole disease image in an original pavement disease image as training data; Based on RESWPGANET after training, taking the noise vector as input, and outputting a pavement pothole disease generated image; Based on a road background image as a target background image and a road surface pothole disease generation image as a source image, seamlessly embedding the road surface pothole disease generation image into the target background image by using a Poisson fusion algorithm to obtain a road surface pothole disease synthesis image which is consistent with illumination of the road background image and has real textures; The RESWPGANET comprises a generator and a discriminator, wherein the generator sequentially comprises a full connection layer, a plurality of cascade residual upsampling modules ReBlockG, a standard convolution layer and a Tanh activation function layer, the full connection layer is used for mapping and remodelling input noise into an initial characteristic diagram, the plurality of cascade residual upsampling modules ReBlockG are used for expanding the initial characteristic diagram step by step in resolution, and each residual upsampling module ReBlockG expands the characteristic diagram of the input module into an initial characteristic diagram Double resolution, n concatenated ReBlockG expands the input 1 st ReBlockG feature map to Outputting after doubling the resolution, wherein the discriminator sequentially comprises a plurality of cascaded residual downsampling modules ReBlockD and a full-connection layer, and each residual downsampling module ReBlockD reduces the characteristic diagram input into the module to Double resolution, n concatenated ReBlockG reduce the input 1 st ReBlockD feature map to And outputting after the resolution is multiplied.
2. The method for generating a pavement damage image based on a residual depth generation countermeasure network of claim 1, wherein the residual upsampling module ReBlockG includes a first main branch and a first shortcut branch in parallel, the first main branch sequentially includes a batch normalized BN, a ReLU, a 3 x 3 transpose convolution, a batch normalized BN, a ReLU, a 3 x 3 standard convolution, the first shortcut branch sequentially includes a batch normalized BN, a ReLU, a 1x 1 transpose convolution, and an output splice fusion of the first main branch and the first shortcut branch is used as an output of the residual upsampling module ReBlockG.
3. The method for generating a pavement damage image based on a residual depth generation countermeasure network of claim 1, wherein the residual downsampling module ReBlockD includes a parallel second main branch and a second shortcut branch with the same input end, the second main branch sequentially includes a3×3 standard convolution, a batch normalized BN, a LeakyReLU, a3×3 standard convolution, and a batch normalized BN, the second shortcut branch sequentially includes a1×1 standard convolution, a batch normalized BN, and a LeakyReLU, and outputs of the second main branch and the second shortcut branch are spliced and fused and then activated by LeakyReLU to obtain an output of the residual downsampling module ReBlockD.
4. The method for generating a pavement damage image based on a residual depth generation countermeasure network according to claim 1, wherein each residual upsampling module ReBlockG enlarges a feature map inputted to the present module Each residual downsampling module ReBlockD reduces the feature map input to the module The number of residual upsampling modules ReBlockG and the number of residual downsampling modules ReBlockD are 5; in the generator, the full-connection layer is used for mapping and reshaping the input noise into an initial feature map of 4×4×512, and the 5 residual upsampling modules ReBlockG step-up the initial feature map to 128×128; in the arbiter, 5 residual downsampling modules ReBlockD scale down the 128×128 resolution feature map to 4×4.
5. The method for generating a pavement slab image based on a residual depth generation countermeasure network according to claim 1, wherein the step of the poisson fusion algorithm comprises: Based on source image Selecting a region to obtain a mask region, wherein the mask region comprises a pavement damage region, and is based on a target background image Performing region selection and determining a target background region; acquiring gradients of a source image Gradient of target background image ; According to a preset gradient selection strategy, a fusion area is formed between the mask area and the target background area The gradient selection strategy comprises any one or combination of ①, ②, ③, wherein the gradient of the source image is selected in a mask area, the gradient of the target background image is selected in a non-mask area, the gradient of the source image is compared with the gradient of the target background image in strength, and the gradient with larger gradient amplitude is selected as the target gradient; The method comprises the steps of obtaining a divergence of a target gradient, constructing a poisson equation, and obtaining a fusion processing result image by solving the poisson equation, wherein the poisson equation is as follows: Wherein, the Is a fusion region The reconstructed image of the interior of the container, Is the image pixel value of the fused region, In order to fuse the regions of the material, As a boundary of the fused region, Is the target gradient.
6. The method for generating a pavement damage image based on a residual depth generation countermeasure network according to claim 5, wherein the step of comparing the gradient of the source image with the gradient of the target background image in intensity and selecting the gradient with a larger gradient amplitude as the target gradient comprises the steps of: ; Gradient amplitude using The norm represents: 。
7. a pavement defect recognition method, characterized by comprising: Obtaining an original pavement defect image, and obtaining a pavement pit defect synthetic image with a true texture consistent with the illumination of a road background image by adopting the method as set forth in any one of claims 1-6; Constructing a pavement defect detection enhancement data set containing expanded pothole samples based on the combination of the pavement pothole defect synthesized image and the original pavement defect image; training a pavement disease recognition detection network based on the pavement disease detection enhancement data set; the road image to be detected and identified is input into a trained road surface disease identification and detection network, and a road surface disease identification result is obtained, wherein the road surface disease identification result comprises road surface disease types and positions, and the road surface disease types comprise longitudinal cracks, transverse cracks, cracks and pits.
8. A pavement slab image generation apparatus for generating a countermeasure network based on a residual depth, comprising: the generation model construction unit is used for constructing a residual depth generation countermeasure network RESWPGANET and training RESWPGANET by using a pothole disease image in the original pavement disease image as training data; the generating unit is used for outputting a pavement pothole disease generating image by taking the noise vector as input based on RESWPGANET after training; The image synthesis unit is used for seamlessly embedding the pavement pit disease generation image into the target background image by using the Poisson fusion algorithm based on the pavement background image serving as the target background image and taking the pavement pit disease generation image as the source image, so as to obtain a pavement pit disease synthesis image which is consistent with the illumination of the pavement background image and has real texture; The RESWPGANET comprises a generator and a discriminator, wherein the generator sequentially comprises a full connection layer, a plurality of cascade residual upsampling modules ReBlockG, a standard convolution layer and a Tanh activation function layer, the full connection layer is used for mapping and remodelling input noise into an initial characteristic diagram, the plurality of cascade residual upsampling modules ReBlockG are used for expanding the initial characteristic diagram step by step in resolution, and each residual upsampling module ReBlockG expands the characteristic diagram of the input module into an initial characteristic diagram Double resolution, n concatenated ReBlockG expands the input 1 st ReBlockG feature map to Outputting after doubling the resolution, wherein the discriminator sequentially comprises a plurality of cascaded residual downsampling modules ReBlockD and a full-connection layer, and each residual downsampling module ReBlockD reduces the characteristic diagram input into the module to Double resolution, n concatenated ReBlockG narrow down the input 1 st ReBlockD feature map And outputting after the resolution is multiplied.
9. An electronic device, the electronic device comprising: A processor; A memory for storing processor-executable instructions; Wherein the processor is configured to implement the method of any one of claims 1-6 or the method of claim 7 by executing the executable instructions.
10. A computer readable storage medium having stored thereon computer instructions, which when executed by a processor, implement the method of any of claims 1-6 or the steps of the method of claim 7.

Description

Pavement disease image generation and identification method based on residual depth generation countermeasure network Technical Field The invention relates to the technical field of image processing, in particular to a pavement disease image generation and identification method based on a residual depth generation countermeasure network. Background Along with the continuous expansion of the mileage scale of the highway, the center of gravity of highway construction is gradually shifted from construction to maintenance and operation, and the requirements of road safety and intelligent maintenance are increasingly outstanding. In actual operation, under the combined action of traffic load, climate environment, water damage and other factors, the road is easy to generate diseases such as longitudinal cracks, transverse cracks, crazes, pits and the like, and if the road cannot be timely and accurately identified, the service life of the road surface is shortened and the running safety is threatened. The traditional road disease detection mainly comprises manual inspection, has the problems of high labor intensity, low efficiency, strong subjectivity, potential safety hazard and the like, and is difficult to adapt to the inspection requirement of a large-scale expressway network. Although the automatic detection method based on the traditional image processing can improve the automation level to a certain extent, the automatic detection method is easily influenced by illumination change, shadow shielding and road texture interference, and has limited capability of distinguishing disease boundaries under complex background. With the development of deep learning, convolutional neural networks are widely used in road surface fault detection tasks. The two-stage detection method such as R-CNN series has higher detection accuracy but lower reasoning speed, and the single-stage method such as YOLO, SSD and the like has higher detection efficiency and is more suitable for real-time detection scenes. However, the deep learning method generally depends on a large-scale, high-quality and class-balanced labeling data set, and when disease samples are scarce and class is seriously unbalanced, the model is easy to be fitted, and the detection performance of low-frequency diseases, particularly pothole diseases, is obviously reduced. In the public road damage data set, the number of diseases such as longitudinal cracks, transverse cracks and crazes is large, the natural formation probability of pothole diseases is low, and the pothole diseases are often repaired rapidly once occurring, so that the number of pothole samples is limited, the effects of shooting angles, illumination conditions and road grades are large, and obvious category imbalance and scene difference are shown. The traditional data enhancement methods such as geometric transformation, illumination disturbance and the like still essentially belong to low-order transformation of an original image, new structural lesion information is difficult to introduce, the diversity of generated data in texture and morphology is limited, complex changes of the lesion appearance under a real road environment are difficult to cover, and the improvement effect is limited. The depth generation model provides a new thought for relieving data scarcity if a countermeasure network (GenerativeAdversarialNetwork, GAN) is generated, and disease images with rich textures and various forms can be automatically synthesized by learning potential distribution of real disease images. However, in the road surface disease scene, the problems of unstable training, mode collapse, texture distortion, edge artifact, unnatural fusion with the background and the like still exist in the traditional GAN and the improved model thereof, and the synthesized image and the real road surface scene have differences in illumination and texture continuity, so that popularization of the synthesized image in engineering application is limited, and therefore, a technical scheme capable of generating a pothole disease image with high quality, naturally fusing with the real road surface background and remarkably improving road pothole detection precision and robustness is needed. Disclosure of Invention Aiming at the problems in the prior art, the invention provides a method for generating and identifying road surface disease images based on a residual depth generation countermeasure network, which effectively improves the detection precision of different types of road surface diseases and small-scale diseases. In a first aspect, a method for generating a pavement damage image based on a residual depth generation countermeasure network is provided, including the steps of: constructing a residual depth generation countermeasure network RESWPGANET, and training RESWPGANET by using a pothole disease image in an original pavement disease image as training data; Based on RESWPGANET after training, taking the noise vec