CN-121979190-A - Multi-sensor fusion and data intelligent transmission method for inland waterway inspection

CN121979190ACN 121979190 ACN121979190 ACN 121979190ACN-121979190-A

Abstract

The invention discloses a multisensor fusion and data intelligent transmission method for inland waterway inspection, which uses an unmanned ship as a core execution carrier to cover three inspection scenes of inland bank protection diseases, water floats and damage of a bridge pier column structure of a river, and designs inland waterway inspection tasks of the unmanned ship around navigation of the unmanned ship, target data acquisition of the multiple scenes, transmission and analysis of target data in a whole process, and adapts to inland waterway inspection requirements under different inspection scenes, wherein the inland waterway inspection tasks comprise an inspection task initialization step, an unmanned ship autonomous inspection task execution step, inspection task termination and subsequent processing steps. The method effectively solves the problems of low efficiency, poor environmental adaptability, lag data processing, insufficient fusion precision and the like of the traditional inspection and the conventional unmanned ship multi-scene inspection, and provides reliable technical support for safe operation and maintenance of the inland river.

Inventors

WAN JIAN
YANG FANGYI
WU HAO
ZHAO HAIFENG
WEN JING
ZHU YUTONG
HUANG KAI
YAN DONG
LIU HUAYU
ZENG YUE
HONG LEI
DING JINFENG
HUANG YAMIN
XIAO CHANGSHI
ZHU WANNING
ZHU ZHOU

Assignees

金陵科技学院

Dates

Publication Date: 20260505
Application Date: 20251209

Claims (10)

1. A multi-sensor fusion and data intelligent transmission method for a inland waterway inspection is characterized in that an unmanned ship is used as a core to execute a carrier, three inspection scenes of inland bank protection diseases, floating objects on water and damage of a bridge pier column structure of a river are covered, a whole-flow design unmanned ship's inland waterway inspection task is designed around the navigation of the unmanned ship, the target data acquisition of the multiple scenes, the transmission and analysis of the target data, and the inland waterway inspection task of the unmanned ship is adapted to the inland waterway inspection requirements under different inspection scenes, wherein the inland waterway inspection task comprises an inspection task initialization step, an unmanned ship autonomous inspection task execution step, inspection task termination and subsequent processing steps; The method comprises the following steps of initializing the patrol task, planning the patrol path of the unmanned ship according to the actual trend of a inland waterway and the actual distribution of scene targets, presetting synchronous working parameters of an optical camera, a laser radar and an auxiliary sensor according to three patrol scenes, and presetting an abnormality judgment threshold value by combining safety operation and maintenance standards under different patrol scenes; The unmanned ship autonomous patrol task execution step comprises the steps that the unmanned ship navigates according to patrol paths, the multi-sensor triggers multi-scene fusion monitoring of synchronization-layering processing-cross verification to collect target data and locate patrol targets, the target data is intelligently transmitted and analyzed according to dynamic transmission and analysis logic of multi-dimensional signal evaluation-data layering scheduling-load linkage adaptation, and scene exception processing is locally executed according to exception judgment thresholds under different patrol scenes; And the inspection task is terminated and the subsequent processing steps are carried out, the unmanned ship plans a return route according to return conditions and executes return, after the return destination position is reached, the inspection record generated by the current inland channel inspection task is transmitted to a shore-based database, and unmanned ship maintenance is executed.
2. The multi-sensor fusion and data intelligent transmission method for inland waterway inspection according to claim 1, wherein the inspection task initializing step is performed as follows: s11, defining longitude and latitude electronic fences covering the inspection area according to the actual trend of the inland waterways and the actual distribution of scene targets, so that in the process of navigation of an unmanned ship along an inspection path, a plurality of sensors can cover all scene targets, and the inland waterway inspection requirements under different inspection scenes are met; s12, respectively designating scene targets for three inspection scenes, and presetting synchronous working parameters of an optical camera, a laser radar and an auxiliary sensor by combining navigation working conditions including unmanned ship speed and target distance; S13, presetting an abnormality judgment threshold value by combining safety operation and maintenance standards in different inspection scenes, and defining the abnormality judgment standards in different inspection scenes to provide judgment basis for subsequent scene abnormality processing; S14, automatically executing equipment linkage self-checking after the unmanned ship is started, ensuring that a system comprising a propeller, an optical camera, a laser radar, an auxiliary sensor and a communication module is adapted to inland waterway patrol requirements under different patrol scenes, and synchronously downloading geographic data of a patrol area and historical patrol records under different patrol scenes by an edge calculation module, and assisting dynamic transmission and processing logic and scene exception processing.
3. The multi-sensor fusion and data intelligent transmission method for inland waterway inspection according to claim 1, wherein the unmanned ship autonomous inspection task executing step comprises the following steps: S21, loading a pure tracking algorithm in a control system of the unmanned ship, inputting longitude and latitude coordinates of preset waypoints divided according to different inspection scenes, setting planning time intervals of an inspection path, and ensuring that the unmanned ship can navigate according to the inspection path to cover all scene targets; S22, starting a positioning module, presetting a positioning data sampling frequency, and automatically triggering a routing inspection path correction instruction when the positioning deviation reaches a set threshold value, so as to ensure the positioning accuracy of a scene target; S23, starting a ship body attitude sensor and an air speed sensor, and connecting the original target data acquired by the optical camera, the laser radar and the auxiliary sensor to an edge calculation module to calculate the navigation environment and the navigation attitude of the unmanned ship in real time so as to obtain the navigation working condition of the unmanned ship; S24, setting a scene linkage control rule, namely ①, outputting a propeller power lifting instruction by an edge calculation module when the water flow speed reaches a set threshold value, ②, outputting a cradle head angle down adjustment instruction by the edge calculation module when the wind speed reaches the set threshold value, ③, outputting a propeller speed reduction instruction by the edge calculation module when the target distance between the unmanned ship and the bridge pier reaches the set threshold value, and synchronizing the control instruction to a shore-based database by the edge calculation module.
4. The multi-sensor fusion and data intelligent transmission method for inland waterway inspection according to claim 1, wherein the multi-sensor triggers synchronous-layering processing-cross-verification multi-scene fusion monitoring to collect target data and position inspection targets, ensures accurate association of the target data collected by an optical camera, a laser radar and an auxiliary sensor in space-time dimension, focuses on identification and positioning of scene targets in three inspection scenes, and comprises the following implementation processes: (1) Triggering synchronization, namely taking an optical camera carried by an unmanned ship as a trigger signal, which can theoretically fully cover and shoot a inland river revetment, and synchronously working in linkage with a laser radar, wherein an edge calculation module uniformly time-feeds by taking the trigger signal as a time-feed reference, sets a time stamp error threshold value of each sensor, and ensures sensor data synchronization of a scene target; (2) The layering processing step comprises the steps that after the edge computing module receives original target data acquired by each sensor, firstly, an abnormal scene target is primarily identified through a lightweight target identification mechanism, then, the original target data is graded through a data value grade layering mechanism to obtain target data of different grades, then, a fuzzy optical image and invalid laser radar point cloud data are removed through validity screening, then, data compression is carried out on the target data, then, the target data is smoothed through a Kalman filtering method, and finally, a two-dimensional abnormal identification-three-dimensional coordinate positioning layering fusion algorithm is executed: ① A two-dimensional anomaly identification layer for identifying scene targets in the optical image by YOLOv algorithm aiming at different inspection scenes and positioning the scene targets in the two-dimensional scene; ② The three-dimensional coordinate positioning layer is used for calling laser radar point cloud data, dividing a scene target point cloud by adopting a RANSAC algorithm, constructing a three-dimensional grid model, mapping a scene target in a two-dimensional scene to the three-dimensional grid model and obtaining three-dimensional space coordinates; (3) The cross verification step is to establish a scene multi-sensor cross verification mechanism to ensure reliable layered fusion results: ① If the sensor data are not synchronous, the edge calculation module automatically triggers time service again; ② If the scene target identification accuracy in the optical image is lower than a set threshold value, calling laser radar point cloud data to supplement and identify a scene target; ③ And automatically marking the abnormal scene target, and interpolating and supplementing the original target data acquired by the same sensor at adjacent moments to ensure the data continuity of the scene target.
5. The multi-sensor fusion and data intelligent transmission method for inland waterway inspection according to claim 1, wherein the method is characterized in that target data is intelligently transmitted and analyzed according to dynamic transmission and analysis logic of multi-dimensional signal evaluation-data layered scheduling-load linkage adaptation, intelligent switching is realized based on multi-dimensional signal quality indexes, data value grades and edge calculation load states, high-efficiency transmission of the target data and stable operation of equipment are ensured, and the method is implemented as follows: (1) The edge computing module builds a multi-dimensional signal quality assessment system, acquires key indexes of the communication module in real time, computes multi-dimensional signal quality indexes, and divides four large signal scenes, namely a high-quality signal area, a good signal area, a general signal area and a difference signal area; (2) Establishing a data value level layering mechanism, and dividing original target data into three levels according to importance, namely primary data formed by core data, secondary data formed by important data and tertiary data formed by conventional data; (3) The load state of the edge computing module is monitored in real time, and the load state of the edge computing module is divided into two types, namely a high load state and a low load state based on a load threshold value; (4) And integrating the signal scene, the target data grade and the load state, constructing a linkage adaptation rule of a transmission mode and processing capacity, and intelligently transmitting and analyzing the target data.
6. The multi-sensor fusion and data intelligent transmission method for inland waterway inspection according to claim 5, wherein when a multi-dimensional signal quality evaluation system is constructed, key indexes of a communication module are collected in real time, including reference signal received power RSRP, signal to noise ratio SNR, transmission delay and packet loss rate, a multi-dimensional signal quality comprehensive score index SQI= (RSRP standardized value×40+SNR standardized value×30+transmission delay reverse standardized value×15+packet loss rate reverse standardized value×15)/100 is calculated through a weighting algorithm, four large signal scenes are divided according to the multi-dimensional signal quality comprehensive score index SQI, ① is a good signal area if SQI is greater than or equal to 80, ② is a good signal area if SQI is less than or equal to 60 and equal to 80, ③ is a general signal area if SQI is less than or equal to 40 and ④ is a bad signal area if SQI is less than 40.
7. The multi-sensor fusion and data intelligent transmission method for inland waterway inspection according to claim 5, wherein a data value grade layering mechanism is established, target data are divided into three grades according to importance, and transmission priority of the target data of each grade is defined: ① The first-level data is core data, and comprises seriously abnormal scene target data with highest priority, wherein the scene target data comprises an optical image/laser radar point cloud, space coordinates and an abnormal grade mark; ② The secondary data are important data, including scene target data of common abnormality, periodic inspection reports and key operation parameters of equipment, and the priority level is inferior; ③ The three-level data are conventional data, and comprise scene target data without abnormality, and the priority is lowest.
8. The multi-sensor fusion and data intelligent transmission method for inland waterway inspection of claim 5, wherein constructing a linkage adaptation rule of a transmission mode and processing capacity comprises the following steps: ① The original target data acquisition, namely continuously acquiring original target data by an optical camera, a laser radar and an auxiliary sensor; ② The edge computing module receives all original target data indiscriminately to finish preliminary caching; ③ The method comprises the steps of grading target data, namely primarily identifying scene targets in original target data by an edge computing module through a lightweight target identification mechanism, and dividing the abnormal degree of the scene target division into three categories of serious abnormality, common abnormality and no abnormality; ④ The method comprises the steps of screening the effectiveness, namely, eliminating blurred and shielded optical images and invalid laser radar point cloud data for all primary data and secondary data, eliminating optical images which cannot identify inspection scenes or scene targets for the tertiary data, and eliminating invalid, damaged and unresolved laser radar point cloud data; ⑤ Data compression, namely the compression degree of the three-level data is higher than that of the two-level data, and the compression layer degree of the two-level data is higher than that of the one-level data; ⑥ Kalman filtering, namely smoothing the compressed primary data and secondary data by a Kalman filtering method with a core iteration step, and smoothing the compressed tertiary data by a Kalman filtering method without a core iteration step; ⑦ Two-dimensional anomaly identification, namely identifying a scene target in an optical image by YOLOv algorithm for primary data and secondary data, and positioning the scene target in a two-dimensional scene; ⑧ Three-dimensional coordinate positioning, namely, for primary data, segmenting a scene target point cloud by adopting a RANSAC algorithm, constructing a three-dimensional grid model, mapping a scene in a two-dimensional scene to the three-dimensional grid model, and obtaining three-dimensional space coordinates; ⑨ The repair grade mark is used for subdividing an abnormal scene target into an emergency repair grade and a conventional maintenance grade for the primary data and the secondary data; ⑩ Data encapsulation, namely standardized naming primary data, secondary data and tertiary data according to a specified format; ⑪ Uploading or caching data, namely uploading primary data, secondary data and tertiary data in a high-quality signal area, uploading the primary data and the secondary data in a good signal area, uploading the primary data in a general signal area, and caching the primary data, the secondary data and the tertiary data in a difference signal area; ⑫ The collaborative review comprises the steps that an edge calculation module collaborates with a target data analysis result uploaded by a shore-based control center, only collaboratively reviews first-level data when the edge calculation module is in a high-load state in a high-quality signal area, a good signal area and a general signal area, and collaboratively reviews second-level data after the collaborative review of the first-level data is completed when the edge calculation module is in a low-load state in the high-quality signal area, the good signal area and the general signal area.
9. The multi-sensor fusion and data intelligent transmission method for inland waterway inspection according to claim 1 is characterized in that the method locally executes scene exception handling and generates a periodic inspection report according to exception judging thresholds under different inspection scenes, the executing process comprises the steps that an edge computing module generates the periodic inspection report once every 5 minutes, the content comprises the current position of an unmanned ship, the length of an inspected inland river bank protector, the identified scene target and the running state of equipment, when the severely exceptional scene target is detected, the unmanned ship pauses navigation, adjusts the angle of a cradle head to focus the scene target, supplements a special image of the acquired scene target, feeds back to a shore-based control center, and the shore-based control center issues a control instruction, and responds to the control instruction.
10. The multi-sensor fusion and data intelligent transmission method for inland waterway inspection according to claim 1, wherein the inspection task is terminated and the following processing steps are performed as follows: S31, when the unmanned ship completes the navigation channel inspection task, the navigation instruction received to the shore-based control center or the electric quantity of the lithium battery is reduced to a set threshold value, automatically starting the navigation, planning the shortest navigation route by combining the geographic data of the inspection area by the edge calculation module, and reporting the navigation progress to the shore-based control center at regular time; s32, after the unmanned ship reaches a return terminal position, transmitting a patrol record generated by the current inland channel patrol task to a shore-based database through a gigabit wired Ethernet, wherein the patrol record is named and stored according to a format of date-channel number-shore protection section-equipment number-signal area type, ZIP compression is adopted before transmission, MD5 verification is adopted in transmission, and verification fails to be automatically retransmitted; S33, maintenance is carried out on the unmanned ship by an operator, wherein the maintenance comprises cleaning a ship body by a high-pressure water gun, wiping an optical camera lens and a laser radar emission window by dust-free cloth, checking a propeller and charging a lithium battery pack, and a shore-based control center automatically generates a patrol task and disc-multiplexing report, wherein the content comprises coverage range of the patrol task of the current inland waterway, the number and processing conditions of scene target discovery and equipment running state, comparison with historical patrol records, analysis of change development rules of the scene target and support operation and maintenance decision.

Description

Multi-sensor fusion and data intelligent transmission method for inland waterway inspection Technical Field The invention relates to the technical field of inland waterway operation and maintenance, in particular to a multi-sensor fusion and data intelligent transmission method for inland waterway inspection, which is characterized in that the core focusing unmanned ship is used for accurately controlling three scenes of inland shore protection, water floats and bridge pier columns of a river, and intelligent dynamic switching transmission processing of scene multi-sensor data depth fusion and multidimensional signal evaluation, data layered scheduling and load linkage adaptation is performed, and the efficient and accurate inspection is developed aiming at core inspection targets of all scenes. Background The inland river is used as an important component of the amphibious transportation hub, and the safe operation and maintenance of the inland river directly relate to the safe passage of a channel and the ecological stability of a coastal area. Firstly, the bank protection is used as a basic barrier for river flood control and impact prevention, is influenced by water flow scouring, environmental erosion and the like for a long time, is easy to generate structural diseases such as cracks, collapse, flaking and the like, and can cause the potential safety hazards such as instability of the bank protection, channel siltation and the like if the processing is not found in time; secondly, the water surface is used as a shipping core channel, along with the acceleration and industrialization development of the urban process, the problems of water surface floaters formed by domestic garbage, industrial waste, agricultural non-point source pollution, natural floaters and the like are increasingly serious, multiple threats are formed on river ecology, shipping safety and drinking water safety, thirdly, the river-crossing bridge pier column is used as a bridge bearing core component, and is subjected to long-term traffic load, natural environment erosion and natural disaster impact, concrete cracking, peeling, reinforcement corrosion, surface attachment accumulation and other damages are easily generated, and if the damage is not found in time, the bearing capacity of the pier column is possibly reduced, and even bridge collapse accidents are caused. With the development of unmanned technology, the existing part of unmanned ships are applied to inland river inspection, but a plurality of technical defects still exist: 1. The problems of low precision and poor synchronism of multi-sensor fusion are solved by adopting a mode of single-function sensors or simply overlapping a plurality of single-function sensors to acquire data in the prior art, and the unified time service and cross calibration mechanism is absent among different sensors, so that the differential requirements of target abnormality under different scenes are difficult to accurately identify. 2. The data transmission and processing suitability is poor, the prior art is mostly dependent on single 4G/5G communication, the data transmission is unstable and easy to lose in a complex channel environment, and the continuity of the data of the revetment disease/floater target/pier stud structure damage cannot be ensured. 3. And the abnormal response is insufficient, namely a quick response mechanism aiming at scene target abnormality is lacking, and the abnormal information acquisition is incomplete. Therefore, there is a need for a inland multi-scene inspection technology integrating high-precision multi-sensor fusion, dynamic intelligent switching transmission and real-time processing of 4G/5G-local SSD storage and dynamic abnormal response by taking an unmanned ship as a core, which is specially aimed at multi-scene core scene targets, solves the pain point of inspection of the existing unmanned ship, and improves inspection efficiency and precision. Disclosure of Invention The invention aims to solve the problems of low inspection efficiency, poor data transmission adaptability, delayed abnormal response and the like of the existing inland unmanned ship, and provides the inland multi-scene inspection method which has high automation degree, accurate data processing and stable and reliable transmission, and the accurate identification, positioning and real-time feedback of the core targets of three large inland scenes are realized. The technical scheme adopted by the invention is as follows: A multi-sensor fusion and data intelligent transmission method for inland waterway inspection is characterized in that an unmanned ship is used as a core to execute a carrier, three inspection scenes including inland bank protection diseases, floating objects on water and damage of a bridge pier column structure of a river are covered, a whole-flow design unmanned ship's inland waterway inspection task is designed around navigation of the unmanned ship, multi-scene ta