CN-121982888-A - Holographic intersection data fusion system and method based on multi-source perception

CN121982888ACN 121982888 ACN121982888 ACN 121982888ACN-121982888-A

Abstract

The invention relates to the technical field of intelligent transportation, in particular to a holographic intersection data fusion system and method based on multi-source perception, which synchronously acquire camera visual data and millimeter wave Lei Dadian cloud data at a road side and realize the purpose of ensuring that the data are distributed in multi-access edge computing equipment at the road side, and carrying out real-time close coupling fusion processing on the multisource perception data by means of unified space-time alignment, self-adaptive feature level fusion, collaborative track estimation and event discrimination modules, generating structured holographic intersection traffic data comprising target tracks, states and traffic events, and distributing the structured holographic intersection traffic data to vehicles and cloud platforms through a road side communication unit and a high-speed backhaul network. The method realizes high-efficiency and accurate multi-mode data fusion at the edge side, remarkably improves the holographic sensing precision and instantaneity of the intersection, and effectively solves the problem that the requirements of low time delay and high reliability of vehicle-road cooperation cannot be met due to the fact that fusion processing is lagged and precision is insufficient in the prior art.

Inventors

ZHANG ZHONGWEI
YANG JIAMING
SONG TIANCHENG
YUAN XIU
YAN CHUN
ZHOU YUXIN

Assignees

绵阳新投实业有限责任公司

Dates

Publication Date: 20260505
Application Date: 20260122

Claims (10)

1. The holographic intersection data fusion method based on multi-source perception is characterized by comprising the following steps of: Synchronously collecting and preprocessing multisource perception data at a road side; In multi-access edge computing equipment deployed at a road side, performing tight coupling data fusion processing on the preprocessed multi-source perception data to generate structured holographic intersection traffic data; and distributing the holographic intersection traffic data.
2. The holographic intersection data fusion method based on multi-source perception according to claim 1, wherein the synchronous acquisition and preprocessing of the multi-source perception data are performed at the road side, and the method specifically comprises the following steps: The multi-source perception data comprise visual data collected by a high-definition camera deployed at an intersection and radar point cloud data collected by a millimeter wave radar deployed at the intersection; The preprocessing comprises the steps of decoding the visual data, detecting and tracking targets to generate visual target features with time stamps, and filtering, clustering and tracking the radar point cloud data to generate radar target features with time stamps.
3. The holographic intersection data fusion method based on multi-source perception according to claim 2, wherein in a multi-access edge computing device deployed on a road side, the pre-processed multi-source perception data is subjected to tight coupling data fusion processing, and structured holographic intersection traffic data is generated, and the method specifically comprises the following steps: mapping the visual target features and the radar target features to the same space-time coordinate system based on the unified high-precision time service signals and the pre-calibrated space transformation parameters; inputting the mapped visual target characteristics and radar target characteristics into a trained multi-mode fusion neural network model, wherein the model outputs an enhanced fusion target characteristic vector; Based on the fusion target feature vector, cross-modal data association is executed, motion state estimation is restrained by utilizing the measured value of the radar target feature in a unified state estimation framework, and simultaneously, iteration updating and optimization of a target track are carried out by combining the semantic recognition result of the visual target feature, and a track list containing target identity, type, centimeter-level position and speed information is output.
4. The holographic intersection data fusion method based on multi-source perception according to claim 3, wherein the mapped visual target features and radar target features are input into a trained multi-modal fusion neural network model, and the model outputs enhanced fusion target feature vectors, specifically comprising: the multimodal fusion neural network model is configured to adaptively adjust fusion weights of visual features and radar features according to scene context of input data.
5. The multi-source perception based holographic intersection data fusion method of claim 3, wherein the pre-processed multi-source perception data is subjected to tight coupling data fusion processing in multi-access edge computing equipment deployed at a road side to generate structured holographic intersection traffic data, and the method further comprises: based on the output track list, a preconfigured rule or a lightweight event judging model is applied, and traffic event description information is detected and generated in real time; The holographic intersection traffic data comprises the track list and the traffic event description information.
6. The multi-source perception based holographic intersection data fusion method of claim 1, wherein the holographic intersection traffic data is distributed, in particular comprising: The first road distribution, through disposing the communication unit of road side of the identical road side, encapsulate the traffic data of the said holographic crossing into the vehicle road and cooperate with the message format, and broadcast or unicast mode send to the networking vehicle of the periphery with low delay; And distributing the second path, and uploading the holographic intersection traffic data to a cloud traffic management platform through a high-speed backhaul network.
7. The multi-source perception based holographic intersection data fusion method of claim 6, wherein the multi-source perception based holographic intersection data fusion method further comprises: And receiving an optimization instruction or a control strategy from the cloud traffic management platform, and adjusting data fusion parameters or executing local control actions in the multi-access edge computing equipment.
8. The multi-source perception based holographic intersection data fusion method of claim 1, further comprising, prior to the synchronously collecting and preprocessing the multi-source perception data at the road side: And configuring deployment position and angle parameters for the high-definition camera and the millimeter wave radar according to the physical layout and the perception requirements of the intersection, and completing joint space-time calibration among multiple sensors.
9. The holographic intersection data fusion method based on multi-source perception according to claim 1, wherein in a multi-access edge computing device deployed on a road side, the pre-processed multi-source perception data is subjected to tight coupling data fusion processing, and structured holographic intersection traffic data is generated, and the method specifically comprises the following steps: and executing under the configuration of the edge computing power resources meeting the delay requirement required by the vehicle-road cooperative security application.
10. The holographic intersection data fusion system based on multi-source perception is used for realizing the holographic intersection data fusion method based on multi-source perception as claimed in claim 1, and is characterized in that, The system comprises a perception execution layer, an edge fusion layer and a network service layer; The perception execution layer comprises a high-definition camera and a millimeter wave radar which are arranged at an intersection according to a preset layout and is used for synchronously acquiring original perception data and performing front-end preprocessing; The edge fusion layer comprises a multi-access edge computing device, and a space-time alignment module, a self-adaptive feature level fusion module, a collaborative track and state estimation module and an event discrimination module are arranged in the multi-access edge computing device, and are used for carrying out tight coupling fusion processing on the preprocessing data from the perception execution layer to generate holographic intersection traffic data; the network service layer comprises a road side communication unit which is in communication connection with the multi-access edge computing device and is used for distributing the holographic intersection traffic data to vehicles, and a high-speed return interface which is used for uploading the holographic intersection traffic data to a cloud platform; the high-definition camera, the millimeter wave radar, the multi-access edge computing equipment and the road side communication unit are interconnected with a communication network through power supply of an intersection.

Description

Holographic intersection data fusion system and method based on multi-source perception Technical Field The invention relates to the technical field of intelligent transportation, in particular to a holographic intersection data fusion system and method based on multi-source perception. Background With the rapid development of intelligent network-connected automobiles and intelligent traffic systems, extremely high requirements are put on the digital and intelligent perception of a complex scene of a road intersection. The method realizes the holographic perception of the intersection, namely, the state information and traffic events of all traffic participants (vehicles, pedestrians, non-motor vehicles and the like) are accurately and comprehensively acquired in real time, and the method is a key basis for supporting the safe passing of high-level automatic driving vehicles and improving the traffic management efficiency. Traditional traffic monitoring mainly relies on a single type of sensor (such as a camera) for video acquisition, and has inherent limitations in all-weather conditions, accurate speed and distance measurement and sensing capability in complex occlusion scenes. In order to overcome the defect of a single sensor, a scheme for deploying multiple types of sensors (such as a visual camera, a millimeter wave radar, a laser radar and the like) at an intersection is proposed in the prior art. For example, some schemes attempt to process data independently by simply arranging a camera and a radar side by side, and then upload the result to the cloud or the vehicle end for post-processing. The method utilizes the characteristic complementation of different sensors to a certain extent, and improves the target existence detection probability under bad weather or illumination. Another common idea is to adopt a centralized processing architecture, and transmit all the original perception data of each intersection back to a data center for unified fusion calculation. However, in the prior art, in the data fusion aspect, most schemes only perform loose data superposition or asynchronous association at the rear end, and the capability of performing real-time and tightly coupled collaborative sensing and fusion processing on multi-source heterogeneous data at the edge side is lacking, so that the judgment precision on the target track, speed and type is insufficient, and the harsh requirement of vehicle-road collaborative application on extremely low time delay is difficult to meet. Disclosure of Invention The invention aims to provide a holographic intersection data fusion system and method based on multi-source perception, which solve the problems that the holographic perception precision of an intersection is insufficient and the time delay is too high and the harsh requirements of vehicle-road cooperative application are difficult to meet due to the lack of real-time and tight coupling cooperative fusion processing of multi-source heterogeneous data on the edge side in the prior art. In order to achieve the above purpose, the invention provides a holographic intersection data fusion method based on multi-source perception, which comprises the following steps: Synchronously collecting and preprocessing multisource perception data at a road side; In multi-access edge computing equipment deployed at a road side, performing tight coupling data fusion processing on the preprocessed multi-source perception data to generate structured holographic intersection traffic data; and distributing the holographic intersection traffic data. The method for synchronously collecting and preprocessing the multisource perception data at the road side specifically comprises the following steps: The multi-source perception data comprise visual data collected by a high-definition camera deployed at an intersection and radar point cloud data collected by a millimeter wave radar deployed at the intersection; The preprocessing comprises the steps of decoding the visual data, detecting and tracking targets to generate visual target features with time stamps, and filtering, clustering and tracking the radar point cloud data to generate radar target features with time stamps. In the multi-access edge computing device deployed on the road side, the pre-processed multi-source perception data is subjected to tight coupling data fusion processing to generate structured holographic intersection traffic data, which specifically comprises the following steps: mapping the visual target features and the radar target features to the same space-time coordinate system based on the unified high-precision time service signals and the pre-calibrated space transformation parameters; inputting the mapped visual target characteristics and radar target characteristics into a trained multi-mode fusion neural network model, wherein the model outputs an enhanced fusion target characteristic vector; Based on the fusion target feature vector, cross-modal