Search

CN-122024096-A - Expressway disease inspection system and method based on intelligent unmanned aerial vehicle

CN122024096ACN 122024096 ACN122024096 ACN 122024096ACN-122024096-A

Abstract

The invention belongs to the technical field of intelligent traffic and unmanned aerial vehicles, and particularly discloses a highway disease inspection system and method based on an intelligent unmanned aerial vehicle with a body, wherein the system comprises the steps of collecting multidimensional data of a highway and pose information of the unmanned aerial vehicle through the unmanned aerial vehicle with a multi-source sensor; the method comprises the steps of adjusting a follow-up flight path of an unmanned aerial vehicle and multi-mode disease identification by utilizing an edge calculation module, realizing local real-time identification and decision of highway disease inspection, outputting highway disease identification results, receiving the highway disease identification results by utilizing a communication subsystem and transmitting the highway disease identification results to a rear-end intelligent management platform, receiving the highway disease identification results by utilizing the rear-end intelligent management platform, and carrying out visual display, maintenance decision support and data storage. The invention solves the problems of insufficient automation level, low multi-source data fusion efficiency and weak real-time response capability in the prior art.

Inventors

  • LI YAN
  • LI YILIN

Assignees

  • 西北工业大学

Dates

Publication Date
20260512
Application Date
20260122

Claims (10)

  1. 1. The expressway disease inspection system based on the intelligent unmanned aerial vehicle with the body is characterized by comprising an unmanned aerial vehicle subsystem, a communication subsystem and a rear-end intelligent management platform, wherein the unmanned aerial vehicle subsystem and the rear-end intelligent management platform are connected through the communication subsystem; the unmanned aerial vehicle is provided with a multi-source sensor and is used for collecting multi-dimensional data of the expressway and pose information of the unmanned aerial vehicle; The edge calculation module is used for adjusting the subsequent flight path of the unmanned aerial vehicle and identifying multi-mode diseases based on the multi-dimensional data and the pose information of the unmanned aerial vehicle, realizing the local real-time identification and decision of highway disease inspection, and outputting a highway disease identification result; the communication subsystem is used for receiving the highway disease recognition result and transmitting the highway disease recognition result to the rear-end intelligent management platform, integrates the 5G module and the LoRa double link, and ensures the communication redundancy in high-speed data transmission and weak signal environments; the rear-end intelligent management platform is used for receiving the expressway disease identification result, and carrying out visual display, maintenance decision support and data storage.
  2. 2. The highway disease inspection system based on the intelligent unmanned aerial vehicle with body according to claim 1, wherein the back-end intelligent management platform comprises: The visual billboard is used for displaying disease distribution thermodynamic diagrams, historical trend analysis and real-time alarm information based on the GIS map; the decision support unit is used for recommending maintenance priority and repair schemes based on the multidimensional data and the traffic flow data of the expressway; and the data archiving unit is used for realizing the encryption storage and blockchain certification of the expressway disease identification result and the time stamp.
  3. 3. A highway disease inspection method based on the highway disease inspection system of any one of claims 1-2, characterized by comprising the following steps: collecting multidimensional data of a highway pavement through an unmanned aerial vehicle carrying a multi-active sensor; the multi-mode data comprises visible light images, infrared thermal imaging data and laser radar point cloud data; inputting the multidimensional data to a modality specific encoder to extract visual features, infrared features and spatial geometric features; Inputting the visual features, the infrared features and the space geometric features into a cross-modal alignment unit, and carrying out space-time registration on the modal features by taking the expressway lane line structure as a reference to obtain space-time consistent multi-modal features; inputting the multi-modal features with consistent time and space into a layering time and space fusion transducer network, and performing cross-modal feature alignment and fusion on the multi-modal features with consistent time and space by using dynamic weights to obtain multi-scale cross-modal features; inputting the multi-scale cross-modal characteristics to a task specific decoder for identification and classification, and outputting to obtain the expressway pavement disease category, position coordinates and severity evaluation result; and uploading the evaluation results of the road surface disease category, the position coordinates and the severity of the expressway to a rear-end intelligent management platform in real time, performing visual display and disease statistical analysis, and generating maintenance decision support information.
  4. 4. A highway disease inspection method according to claim 3 wherein the mode specific encoder comprises a visible light encoder, an infrared encoder and a LiDAR encoder in parallel; the visible light encoder is used for extracting visual characteristics, wherein the visual characteristics comprise cracks and pits; The infrared encoder is used for detecting a temperature abnormal region to obtain temperature characteristics; and the LiDAR encoder is used for extracting space geometric features, wherein the space geometric features comprise three-dimensional deformation and sedimentation.
  5. 5. The highway disease inspection method according to claim 3, wherein the hierarchical space-time fusion transducer comprises a local feature fusion layer, a region association layer and a global reasoning layer which are sequentially connected; the local feature fusion layer is used for enabling the multi-mode features with consistent time and space to perform internal interaction, strengthening local features, and further associating the local features of the corresponding positions among the modes to obtain local details and cross-mode associated local features; The regional association layer is used for carrying out characteristic aggregation on suspected disease regions based on the enhanced local detail and the cross-modal associated local characteristics, and establishing relationships of different parts in the same region and relationships among different regions to obtain a disease diffusion relationship; and the global reasoning layer is used for carrying out smooth verification on the disease spread relation, integrating the environmental information and the road surface type of the current road section, and further carrying out global scene reasoning to obtain multi-scale cross-modal characteristics, namely a comprehensive risk assessment result.
  6. 6. A highway disease inspection method according to claim 3, wherein the task specific decoder comprises a backbone network, a neck network, a detection head unit and a post-processing unit connected in sequence; The backbone network adopts an improved CSPDARKNET for receiving multi-scale cross-modal characteristics, extracting and encoding semantic characteristics again and outputting a multi-scale characteristic diagram; the neck network is a self-adaptive feature pyramid and is used for receiving the multi-scale feature images, and fusing and reprocessing the multi-scale feature images to obtain three-level enhanced feature images; The detection head unit is a multi-task detection head and is used for receiving the three-level enhanced feature images and outputting a detection frame, a segmentation mask and an evaluation level of the disease in parallel; The post-processing unit is a special post-processing unit for the expressway, and is used for receiving a detection frame, a segmentation mask and an evaluation grade of the diseases and integrating and outputting the evaluation results of the types, the position coordinates and the severity of the expressway road surface diseases.
  7. 7. The highway disease inspection method according to claim 6, wherein the modified CSPDARKNET comprises a dedicated input processing layer, N stacked modified cross-stage partial blocks, and a spatial pyramid pooling layer, the input of the first modified cross-stage partial block being connected to the output of the dedicated input processing layer, the input of the nth modified cross-stage partial block being connected to the output of the nth-1 modified cross-stage partial block, the output of the nth modified cross-stage partial block being connected to the input of the spatial pyramid pooling layer; The improved cross-stage partial block comprises a main branch and a residual branch which are parallel, wherein the main branch comprises a first environment self-adaptive layer, a first one-dimensional convolution unit, a three-dimensional depth separable convolution unit, a disease perception fusion unit, a high way-CBAM attention unit and a DropPath regularization unit which are sequentially connected, the residual branch comprises a second environment self-adaptive layer and a second one-dimensional convolution unit which are sequentially connected, the first convolution unit and the second convolution unit have the same structure and comprise a first one-dimensional convolution layer, a first normalization layer and a first activation function layer which are sequentially connected, and the three-dimensional depth separable convolution unit comprises a three-dimensional convolution layer, a second normalization layer and a second activation function layer which are sequentially connected; the special input processing layer is used for carrying out preliminary space downsampling and channel adjustment on multi-scale cross-mode characteristics; Each improved cross-stage part block is used for carrying out depth feature extraction and fusion on the multi-scale cross-mode features after spatial downsampling and channel adjustment; the spatial pyramid pooling layer is used for carrying out parallel pooling on the multi-scale cross-modal characteristics after depth characteristic extraction and fusion to obtain a multi-scale characteristic map; The first environment self-adaptive layer and the second environment self-adaptive layer are used for dynamically adjusting parameters of the first normalization layer and the second normalization layer according to environment parameters acquired by the unmanned aerial vehicle in real time; the disease sensing fusion unit is used for respectively extracting linear texture characteristics of the cracks and block morphological characteristics of the pits through parallel branches and carrying out self-adaptive weighted fusion; the Highway-CBAM attention unit is used for focusing attention weight on a road surface area of a traffic lane by taking the geometric information of the traffic lane as space guidance; DropPath regularization unit for preventing overfitting.
  8. 8. The highway disease inspection method according to claim 6, wherein the multi-scale feature map comprises a low-level feature C3, a middle-level feature C4 and a high-level feature C5, and the neck network fuses and reprocesses the multi-scale feature map through a top-down and bottom-up bidirectional path to obtain three-level enhancement feature maps; The top-down path specifically comprises the following steps: Carrying out one-dimensional convolution on the high-level characteristic C5 to reduce the channel dimension, inputting the high-level characteristic C5 with the reduced channel dimension into an environment self-adaptive unit and a disease specificity enhancing module to carry out self-adaptive adjustment and characteristic enhancement, and obtaining a first enhanced characteristic; upsampling the first enhanced feature to the same spatial resolution as the middle-layer feature C4 to obtain a feature M5, and splicing the feature M5 and the middle-layer feature C4 along the channel dimension to obtain a first spliced feature; carrying out one-dimensional convolution on the first spliced feature to reduce the channel dimension to obtain an intermediate feature P4_td, and carrying out up-sampling on the intermediate feature P4_td to obtain a feature M4; splicing the feature M4 with the low-layer feature C3 to obtain a second spliced feature, and further reducing the channel dimension of the second spliced feature through one-dimensional convolution to obtain a 128×80×80 enhancement feature map P3_out; the bottom-up path specifically comprises the following steps: Performing three-dimensional convolution downsampling on the enhanced feature map P3_out to obtain a feature P3_ds, and splicing the feature P3_ds with the intermediate feature P4_td to obtain a third spliced feature; carrying out one-dimensional convolution on the third splicing characteristic to reduce the channel dimension, and inputting the third splicing characteristic into an environment self-adaptive unit and a disease specificity enhancing module to carry out self-adaptive adjustment and characteristic enhancement to obtain a second enhanced characteristic, namely an enhanced characteristic map P4_out of 256 multiplied by 40; Performing three-dimensional convolution downsampling on the enhanced feature map P4_out to obtain a feature P4_ds, and splicing the feature P4_ds with a feature M5 to obtain a fourth spliced feature; And carrying out self-adaptive adjustment and characteristic enhancement on the fourth splicing characteristic through an environment self-adaptive unit and a disease specificity enhancement module to obtain a third enhancement characteristic, namely an enhancement characteristic diagram P5_out of 512 multiplied by 20.
  9. 9. The highway disease inspection method according to claim 8, wherein the disease specificity enhancing module comprises three parallel crack characteristic branches, pit characteristic branches and general disease branches, and an adaptive characteristic fusion unit, and an output end of the crack characteristic branches, an output end of the pit characteristic branches and an output end of the general disease branches are connected with an input end of the adaptive characteristic fusion unit; The crack characteristic branch is used for capturing the linear trend, the continuity and the fine edge characteristics of cracks in the multi-scale characteristic map by adopting a directional convolution group and a cavity convolution sequence, and outputting the crack characteristic map by inhibiting texture noise through a local thinning unit; the pit feature branch is used for carrying out shadow gradient analysis, texture mutation analysis and block context information analysis on the multi-scale feature map so as to explicitly model the concave form and boundary outline of the pit and output the pit feature map; The general disease branch is used for reserving original semantics and global context information of the multi-scale feature map through 1X 1 convolution and outputting a general feature map as a supplementary substrate; And the self-adaptive feature fusion unit is used for dynamically calculating the weight coefficient, carrying out element-by-element weighted summation on the crack feature map, the pit feature map and the general feature map according to the weight coefficient, and outputting the strengthening feature.
  10. 10. The highway disease inspection method according to claim 3, further comprising the steps of collecting pose information of the unmanned aerial vehicle, adjusting flight control instructions of the unmanned aerial vehicle in real time based on the multidimensional data and the pose information of the unmanned aerial vehicle, and realizing autonomous navigation of the unmanned aerial vehicle, wherein the autonomous navigation of the unmanned aerial vehicle comprises environment perception, dynamic path planning and obstacle avoidance control and comprises the following steps: Constructing a road three-dimensional map and detecting obstacles through point cloud data of a laser radar and a visual SLAM algorithm to realize environment perception; Generating a global path and a local path based on a road three-dimensional map by utilizing an A-algorithm and a dynamic window method, and dynamically adjusting the flying height and the speed by combining a road topological structure; based on the obstacle detection result, the unmanned aerial vehicle is automatically prevented from obstacle by utilizing a deep reinforcement learning strategy; The state space of the deep reinforcement learning strategy comprises the pose of the unmanned aerial vehicle, an obstacle detection result and environmental parameters, and the action space comprises a motion control instruction and a sensor control instruction of the unmanned aerial vehicle; obtaining an optimal patrol strategy through maximum reward function training of the deep reinforcement learning strategy, and obtaining the reward function The method comprises the following steps: 。

Description

Expressway disease inspection system and method based on intelligent unmanned aerial vehicle Technical Field The invention belongs to the technical field of intelligent traffic and unmanned aerial vehicles, and particularly relates to an expressway disease inspection system and method based on an intelligent unmanned aerial vehicle. Background The expressway is an important component of national infrastructure, and timely discovery and accurate assessment of pavement diseases (such as cracks, pits, subsidence and the like) of the expressway are of great significance in guaranteeing traffic safety and prolonging the service life of roads. At present, inspection of expressway diseases mainly depends on manual or traditional camera inspection vehicle modes, and the following technical problems exist: 1) The manual inspection efficiency is low, the risk is high, the precision is poor, the personnel can inspect on site on the expressway, the potential safety hazard is large, the manual visual inspection is greatly influenced by illumination and weather, and the efficiency is difficult to meet the requirement of rapidly-developed road maintenance. 2) The traditional vehicle-mounted equipment has poor flexibility, is limited by factors such as terrain, lane conditions and the like, and is difficult to cover nonstandard pavement areas such as side slopes, isolation belts and the like. 3) The existing system mainly uses image acquisition, later identification depends on manual or rule setting, and the system lacks self-adaptive learning capability and has limited intelligent degree. Although the unmanned aerial vehicle inspection has partially replaced manual work, the intelligent level still stays in the stage of 'data acquisition-manual analysis': 1) The traditional unmanned plane flies according to a fixed path and cannot adapt to autonomous obstacle avoidance and path planning of complex road environments (such as tunnels, bridges and curves) or adaptively adjust the view angle (such as shooting the bottom surface of the bridge by a specific elevation angle) depending on a preset route. 2) And after disease identification, the image data needs to be transmitted back to the rear end for manual interpretation, the time from inspection to report generation is more than or equal to 24 hours, and the emergency disease (such as a burst pit) is difficult to respond in time. 3) Multimode data splitting, namely that a single sensor (such as an optical camera) is difficult to consider different disease types, and the multi-sensor data lacks fusion analysis capability. 4) The system has low integration level, and the hardware (sensor and communication subsystem) and the software (navigation and recognition algorithm) have poor cooperativity and lack of a unified management platform. Technical bottlenecks to be solved in the prior art include: 1) The adaptability of the dynamic environment is insufficient, and the traditional unmanned plane lacks the capability of real-time sensing and path re-planning of complex road conditions (such as abrupt tunnel light and shadow changes and curve airflow disturbance in mountainous areas). 2) The multi-modal disease feature learning is insufficient, the existing algorithm is mostly based on single optical image training, and the cross-modal feature (optical, infrared and LiDAR) fusion capability of the multi-modal disease such as cracks, pits, greasy dirt and the like is insufficient. 3) And the autonomous operation flow is fragmented, the inspection task needs more manual intervention links (such as route adjustment and sensor switching), and a full-flow automatic closed loop is not formed. Disclosure of Invention The invention aims to solve the problems of insufficient automation level, low multi-source data fusion efficiency and weak real-time response capability in the prior art, and provides an expressway disease inspection system and method based on an intelligent unmanned aerial vehicle. The expressway disease inspection system based on the intelligent unmanned aerial vehicle comprises an unmanned aerial vehicle subsystem, a communication subsystem and a rear-end intelligent management platform, wherein the unmanned aerial vehicle subsystem and the rear-end intelligent management platform are connected through the communication subsystem; the unmanned aerial vehicle is provided with a multi-source sensor and is used for collecting multi-dimensional data of the expressway and pose information of the unmanned aerial vehicle; The edge calculation module is used for adjusting the subsequent flight path of the unmanned aerial vehicle and identifying multi-mode diseases based on the multi-dimensional data and the pose information of the unmanned aerial vehicle, realizing the local real-time identification and decision of highway disease inspection, and outputting a highway disease identification result; the communication subsystem is used for receiving the highway disease recognition