CN-122017867-A - Inland ship tracking system based on laser radar and video fusion

CN122017867ACN 122017867 ACN122017867 ACN 122017867ACN-122017867-A

Abstract

The invention belongs to the technical field of intelligent monitoring of a inland waterway and ship traffic management, and mainly relates to a inland ship tracking system based on laser radar and video fusion, which performs time synchronization and space calibration through a space-time calibration algorithm to eliminate deviation between the laser radar and video monitoring, realizes depth coordination of the laser radar and the video monitoring, and ensures accurate matching of data; the ship monitoring system based on the track prediction model can track ships in all weather stably by adopting an AI target locking algorithm and a multi-distance level tracking optimization technology, predict the future position of the ships in advance based on the track prediction model, provide sufficient avoidance time for collision early warning, effectively solve the monitoring error caused by a floating platform through a hardware and software dual anti-shake compensation mechanism, improve the stability and accuracy of ship monitoring, and simultaneously maintain higher recognition rate under severe weather conditions.

Inventors

TANG ZHENGTAO
CHEN XIAOHE
ZHU ZIWEI
GU XIEQIN
LI DONGSHENG
SUN AIFENG
ZHANG SHAOKUN
SHI LONG
ZHANG XIAONING
XIONG RONGJUN
OuYang Jinze

Assignees

长航检测科技(武汉)有限公司

Dates

Publication Date: 20260512
Application Date: 20260109

Claims (10)

1. Inland ship tracking system based on laser radar and video fusion, characterized by comprising: the hardware cooperative deployment module takes a floating navigation mark as a platform and integrates a laser radar, a video monitoring, an anti-shake bracket, an edge computing unit and auxiliary equipment in a standardized manner; The space-time calibration algorithm module is used for eliminating space-time deviation between the laser radar and the video monitoring through time synchronization and space calibration, establishing a unified coordinate system, and eliminating errors existing in the laser radar through distance calibration: the linkage tracking strategy module is used for linking the plurality of modules, designing a linkage flow of laser radar triggering-video automatic tracking-data fusion output, automatically tracking and monitoring the ship and outputting fusion data; the dynamic risk pre-judging and tracking optimizing module is used for realizing the dynamic adjustment of the future position and tracking strategy of the predicted ship through a track prediction model and a multi-distance-level tracking optimizing method based on the fusion data; And the floating platform anti-shake compensation module is used for designing a double compensation mechanism comprising hardware compensation and software compensation and reducing monitoring errors caused by navigation mark shake.
2. The inland vessel tracking system of claim 1, wherein the hardware co-deployment module comprises: The hardware collaborative deployment module takes a floating navigation mark as a platform, and integrates a laser radar, a video monitoring, an anti-shake bracket, an edge computing unit and auxiliary equipment in a standardized manner; The laser radar has the core parameters of ranging range of 0.1-50m, precision +/-0.1 m and scanning frequency of 10Hz, and the deployment requirement that the laser radar is arranged at the center position of the top of a navigation mark, the laser emission direction is consistent with the central axis of the navigation mark, the horizontal scanning angle is 360 degrees, and the vertical angle is-15 degrees to +15 degrees; The video monitoring has the core parameters of image resolution of not less than 400 ten thousand pixels and optical zooming capability of not less than 30 times, wherein the deployment requirement is that the video monitoring is installed at the position of 0.5m on the right side of the laser radar, the initial cradle head angle is consistent with the reference direction of the laser radar, and horizontal 360-degree rotation and vertical-30-90-degree rotation are supported; The deployment requirement is that the laser radar and the video monitoring share the same anti-shake bracket, the bottom of the bracket is rigidly connected with the navigation mark cabinet body, and the navigation mark shake is compensated in real time through the gyroscope; The edge computing unit has the core parameter requirements of calculating energy Se9 and supporting GPU acceleration, and the deployment requirements of being installed in a navigation mark power distribution cabinet and connecting a laser radar with video monitoring through an RJ45 gigabit network to realize real-time data processing.
3. The inland vessel tracking system of claim 1, wherein the space-time calibration algorithm module comprises: including functions of time synchronization, spatial calibration, and distance calibration: The time synchronization is that a PPS signal of a Beidou positioning system is used as a time reference, the sampling frequency of a locking laser radar is consistent with that of video monitoring, the error is reduced, and the problem of data dislocation is solved; The space calibration comprises azimuth calibration and distance calibration, wherein the azimuth calibration comprises the steps of setting 3 calibration points with known coordinates in an avigraph range, and measuring the azimuth angle of the calibration points by a laser radar 、、 Video monitoring rotates to a calibration point and records the angle of the cradle head 、、 Calculating deviation compensation by a least square method, wherein the formula is as follows: ; Wherein, the In order to calibrate the direction angle of the point, Is the angle of the cradle head and is used for adjusting the angle of the cradle head, Is a fixed compensation value; By using Performing fixed compensation on a measurement result of the laser radar; The distance calibration is to measure the distance D of the ship by the laser radar, combine the pixel width W of the ship in the video picture, reversely verify the distance measurement precision of the laser radar by a pre-trained distance-pixel mapping model, compare the distance measurement result of the laser radar with the output result of the model, automatically correct if the deviation is more than 0.3m, and utilize the deviation of the distance-pixel mapping model to calculate the formula: ; Wherein, the In order to have a distance value that needs to be corrected, Is a predicted value obtained from a distance-to-pixel mapping model, Is the distance difference value to be corrected; The calculation formula of (2) is as follows: ; Wherein, the For the slope in the distance-to-pixel mapping model, Obtaining a pixel width deviation value for calculation; using the obtained distance value to be corrected And correcting the ranging result of the laser radar.
4. The inland vessel tracking system of claim 3, wherein the distance-to-pixel mapping model building process comprises: The pre-training stage comprises constructing training set, selecting a group of typical ships with known real ship widths as calibration samples in the navigation mark monitoring water area, and making the calibration ships at a plurality of different and known distance points A, sailing upwards, synchronously collecting a pair of data at each distance point, wherein the accurate distance value D measured by a laser radar is used as a reference true value, and the pixel width W corresponding to the ship in a video picture is automatically extracted by an image analysis algorithm; Based on the constructed training set, fitting a preset model by adopting a regression algorithm least square method, wherein the model formula is as follows: ; Wherein k is the slope and b is the intercept; the goal of this algorithm is to find an optimal set of model parameters k and b such that the model calculates the theoretical pixel width Pixel width actually measured in data set The overall error between them is minimal; through fitting to the training set, a set of determined, optimal parameters k and b are obtained, this formula containing the specific k and b And obtaining a pre-trained distance-pixel mapping model.
5. The inland vessel tracking system of claim 1, wherein the linkage tracking strategy module comprises: designing a linkage flow of laser radar triggering, video automatic tracking and data fusion output; The target triggering stage comprises the steps of scanning a laser radar in real time, and acquiring a ship distance D, an azimuth angle alpha, a speed V and a ship point cloud characteristic when the ship is detected to enter a 50-meter monitoring range; Video automatic tracking stage, namely after video monitoring receives the instruction, based on a space calibration formula The method comprises the steps of controlling a cradle head to rotate to a target azimuth angle within 0.5 seconds, adopting an AI target locking algorithm to automatically identify the outline of a ship in a video picture, dynamically adjusting the focal length and the angle through the cradle head to ensure that the ship is always in the center of the picture, modifying a tracking strategy according to a multi-distance level tracking optimization method of a dynamic risk pre-judging and tracking optimization module, starting a ship name identification function based on an OCR algorithm when the ship distance is less than 30 meters, extracting identity information such as ship names, MMSI codes and the like, correlating with distance and speed data of a laser radar, and generating a ship identity-dynamic parameter unified file; And in the fused data output stage, the edge calculation unit fuses the obtained data to form fused data, namely the distance/speed of the laser radar and the ship name/image of the video, the fused data is pushed to the shore-based platform in real time through an MQTT protocol and stored in an aviation specimen, the storage frequency is 1 time/second, and 30-day historical data is reserved.
6. The inland vessel tracking system of claim 5, wherein the extracting the point cloud features of the vessel comprises: preprocessing the collected original point cloud, and effectively eliminating environmental noise such as water surface reflection, spray, rainwater and the like by using a filtering algorithm to obtain pure point cloud data; dividing the point clouds adjacent to each other in space into independent clusters by using Euclidean clustering algorithm so as to separate potential target objects; calculating the centroid position of the obtained ship cluster, and further obtaining the accurate distance D and azimuth angle of the ship relative to the navigation mark ; And finally, combining multi-frame detection results of continuous tracking of the same ship, correlating the current detection result with a historical frame, and accurately estimating the real-time speed V based on the position change of the current detection result.
7. The inland vessel tracking system of claim 5, wherein the AI targeting algorithm comprises: The AI target locking algorithm adopts a video target tracking technology, selects the algorithm according to the difference of deployment environments, and adopts a high-efficiency tracker represented by discrimination related filtering aiming at open water scenes; aiming at complex background scenes, a discriminant tracker based on deep learning is started, high-level semantic features are extracted through a deep convolutional neural network, and target ships are better distinguished from visually similar background interferents.
8. The inland vessel tracking system of claim 5, wherein the ship name recognition function specifically comprises: meanwhile, an AIS receiver continuously receives signals, and an edge computing unit correlates and computes MMSI codes and bound official ship names in AIS targets closest to the target azimuth according to the target azimuth provided by the laser radar; Performing ship name area detection and image enhancement on the captured video frame, identifying text content in the image by using an OCR (optical character recognition) algorithm, and outputting the identified text content and the confidence coefficient thereof; comparing the text content extracted by OCR with AIS ship name, outputting ship name as the ship name when ship names are consistent, using AIS ship name as the recognized ship name when the ship name extracted by OCR is abnormal, and sending information to appointed background personnel to request manual intervention when problems occur in OCR extraction and AIS reception.
9. The inland vessel tracking system of claim 1, wherein the dynamic risk prediction and tracking optimization module comprises: based on the fusion data, the future position of the predicted ship and the tracking strategy are dynamically adjusted through a track prediction model and a multi-distance-level tracking optimization method; the track prediction model refers to a continuous 3 sampling points based on a laser radar by adopting a 3-point extension line and course angle correction algorithm Position of 、、 Calculating course angle of ship Prediction of Position after seconds If the predicted position enters the collision risk area of 30 meters of the navigation mark, triggering early warning 5 seconds in advance; the multi-distance level tracking optimization method is characterized in that working parameters of a laser radar and a video are dynamically adjusted according to the distance D between a ship and a navigation mark, and the balance accuracy and the energy consumption are divided into the following three distance intervals: When D is 50-30 m, the laser radar scans according to 10Hz, the video monitoring is switched to the medium multiplying power, and the ship name recognition is started; When D is less than 30 meters, the laser radar scanning frequency is increased to 15Hz, the ship is locked by adopting high multiplying power in video monitoring, and the synchronous linkage audible and visual alarm plays early warning voice.
10. The inland vessel tracking system of claim 1, wherein the floating platform anti-shake compensation module comprises: the dual compensation mechanism comprising hardware compensation and software compensation is designed, so that the monitoring error caused by the shaking of the navigation mark is reduced; The hardware compensation means that a gyroscope arranged in the anti-shake support collects the shake angle of a navigation mark in real time, and the shake amplitude of the laser radar and the video is controlled within +/-0.5 degrees by driving the support to reversely rotate through a motor; The software compensation refers to the correction formula of the horizontal shaking angle as follows: ; where x is the abscissa axis coordinate before correction, y is the ordinate axis coordinate before correction, Is a horizontal shaking angle; The correction formula of the vertical shaking angle is as follows: ; where x is the abscissa axis coordinate before correction, y is the ordinate axis coordinate before correction, Is a vertical shaking angle; And ensuring that the point cloud data is consistent with the actual ship position through correction.

Description

Inland ship tracking system based on laser radar and video fusion Technical Field The invention belongs to the technical field of intelligent channel management, and mainly relates to a inland ship tracking system based on laser radar and video fusion. Background The inland waterway (especially the Yangtze river trunk line) has large ship flow, complex navigation environment (turbulent water flow, more foggy days and dense barriers), and the ship tracking and monitoring are key links for guaranteeing navigation safety. The existing ship tracking technology is mostly dependent on a single sensor or simple data superposition; The invention provides a ship tracking method and a ship tracking device based on vision feature optimization radar-AIS, which relate to the technical field of target tracking, and are used for solving the problem that the ship tracking method and the ship tracking device based on vision feature optimization radar-AIS are available in China application number CN 202510429693.4. The method comprises the steps of guiding a photoelectric turntable to turn to the azimuth of a target ship according to target information of the target ship acquired by a radar-AIS, identifying current images acquired by the photoelectric turntable to obtain a plurality of current ships, tracking the current ships, acquiring visual characteristic information and track information of each current ship, and determining the target ship from the current ships according to the target information, the visual characteristic information and the track information to track. The scheme realizes accurate identification and tracking of the ship target and provides stable and reliable tracking effect. The scheme has provided a perfect ship tracking method, but has some core defects that firstly, a laser radar can only provide distance and azimuth data, visual characteristics such as appearance, name and the like of a ship cannot be obtained, target identity confirmation is difficult to realize, secondly, video monitoring is greatly influenced by environment, recognition rate is low under low visibility, tracking loss is easy to occur when the ship moves rapidly, and the laser radar and the video monitoring lack of cooperative work to cause data to be not time-space calibrated and positioning deviation to occur, meanwhile, a system cannot dynamically pre-judge based on a historical track, collision early warning triggering delay is reserved, the avoidance time for a crew is insufficient, in addition, the prior art has poor adaptability to a floating platform, water flow shaking causes monitoring precision to be reduced, and the prior art cannot effectively solve four core problems of accurate recognition, low-delay cooperative tracking and dynamic risk pre-judging and floating platform adaptation in general. In order to solve the problems, the invention provides a inland ship tracking system based on laser radar and video fusion, which establishes a unified space-time coordinate system through a multi-sensor depth cooperative framework to realize millisecond cascade dynamic tracking of the laser radar and the video, realizes all-weather stable tracking and identity recognition through an AI target locking algorithm and multi-distance level tracking optimization, pre-warns risks in advance based on a track prediction model, overcomes platform shaking through a hardware-software dual anti-shake compensation mechanism, and finally systematically solves four technical problems of all-weather accurate recognition, low-delay cooperative tracking, dynamic risk prediction and floating platform adaptation. Disclosure of Invention The invention provides a inland ship tracking system based on laser radar and video fusion, and aims to solve the problems of single sensing capability, lack of a cooperative mechanism, insufficient dynamic pre-judging capability and poor suitability of a floating platform of the laser radar and video monitoring in the prior art. In order to solve the problems, the invention is realized by adopting the following technology: provided is a inland ship tracking system based on laser radar and video fusion, comprising: the hardware cooperative deployment module takes a floating navigation mark as a platform and integrates a laser radar, a video monitoring, an anti-shake bracket, an edge computing unit and auxiliary equipment in a standardized manner; The space-time calibration algorithm module is used for eliminating space-time deviation between the laser radar and the video monitoring through time synchronization and space calibration, establishing a unified coordinate system, and eliminating errors existing in the laser radar through distance calibration: the linkage tracking strategy module is used for linking the plurality of modules, designing a linkage flow of laser radar triggering-video automatic tracking-data fusion output, automatically tracking and monitoring the ship and outputting fusion data; the dyn