CN-121982880-A - Data analysis method and system based on cross-modal semantic communication

CN121982880ACN 121982880 ACN121982880 ACN 121982880ACN-121982880-A

Abstract

A data analysis method and system based on cross-modal semantic communication comprises the steps of obtaining multi-source data of an intersection to be analyzed, wherein the multi-source data comprise vehicle tracks, video images, meteorological information and pedestrian mobile equipment data, conducting feature extraction on the multi-source data by means of a semantic extraction technology to obtain key semantic features, compressing the key semantic features of the data by means of a reinforcement learning optimization compression technology, encrypting the compressed data by means of a lightweight password hash algorithm fusion blockchain technology, conducting real-time interaction on the encrypted data between a basic facility and the vehicle by means of a low-delay transmission protocol to obtain a complete data set, extracting the multi-modal features from the complete data set, integrating the multi-modal features by means of a data fusion technology to generate a preliminary relation graph of dynamic conflict of the intersection, and identifying potential conflict points. The multi-mode data of the traffic intersection is fully utilized, the coordination capability of the vehicles and the roads is strong, the behavior data is considered, the occurrence rate of traffic accidents is reduced, and the low-delay transmission efficiency is realized.

Inventors

ZHANG XIAOYAN
SONG YUE
ZHOU WEI
HE ZHENGRUI

Assignees

北京工业职业技术学院

Dates

Publication Date: 20260505
Application Date: 20251231

Claims (10)

1. A data analysis method based on cross-modal semantic communication, comprising: Acquiring multi-source data of an intersection to be analyzed, wherein the multi-source data comprise vehicle tracks, video images, weather information and pedestrian mobile equipment data; Extracting features of the multi-source data by adopting a semantic extraction technology to obtain key semantic features, and compressing the data key semantic features by adopting a compression technology of reinforcement learning optimization to obtain a compressed data set; Encrypting the compressed data based on a lightweight password hash algorithm fusion blockchain technology, and carrying out real-time interaction on the encrypted data between a base facility and a vehicle by utilizing a low-delay transmission protocol to obtain a complete data set; the multi-modal features are extracted from the complete data set, the multi-modal features are integrated by adopting a data fusion technology, a preliminary relationship diagram of dynamic conflict of the intersection is generated, and potential conflict points are identified.
2. The method of claim 1, wherein the feature extraction of the multi-source data using semantic extraction techniques to obtain key semantic features comprises: Substituting the multi-source data into a pre-constructed multi-dimensional weight scoring system to obtain priority scores of the data streams; Performing primary screening based on priority scores of data flows in combination with a set threshold value to obtain a first data set; Extracting key features by adopting a pre-trained ViT model to obtain key semantic features; the multi-dimensional weight scoring system is constructed by taking priority levels as first-level indexes, setting factors influencing the first-level indexes as second-level indexes, and setting weights for the second-level indexes; the trained ViT model is obtained by training the ViT model based on the marked vehicle track data, weather data and pedestrian mobile device data.
3. The method of claim 1, wherein the compressing the data key semantic features using reinforcement learning optimized compression techniques results in a compressed data set comprising: Defining a state space based on the current bandwidth utilization, network delay, data priority, and historical compression quality score, and taking the state space as an input of the DQN model; defining an action space based on the compression selection, determining decision options of the DQN model from the action space; And designing a reward function based on the PSNR value, the transmission delay and the bandwidth saving rate, guiding the DQN model to learn and balance compression quality, instantaneity and efficiency by the reward function, and finally obtaining a compressed data set.
4. The method of claim 1, wherein the fusing blockchain technique based on the lightweight cryptographic hash algorithm encrypts the compressed data, comprising: calculating the compressed data packet by adopting a lightweight password hash algorithm to generate a hash value with a fixed length; the compressed data, the corresponding hash value, the corresponding time stamp, the corresponding data serial number and the corresponding data packet type identifier are packaged together to form a complete transmission message; And submitting the transmission message to a alliance chain network constructed based on HYPERLEDGER FABRIC for certification through an independent low-priority thread or channel.
5. The method of claim 1, wherein the real-time interaction of the encrypted data with the vehicle using the low latency transport protocol to obtain the complete data set comprises: after the hash value is calculated, immediately sending a transmission message to a target vehicle-mounted OBU through a low-delay transmission protocol; and searching a certification record based on a block chain verification thread, judging that the data packet is complete and reliable when the local hash value and the metadata are consistent with the certification record, and then decrypting or decoding the transmission message to form a complete data set.
6. The method of claim 1, wherein the extracting the multi-modal features from the complete dataset, integrating the multi-modal features using a data fusion technique, generating a preliminary relationship graph of dynamic conflicts at the intersection, identifying potential conflict points, comprises: extracting multi-modal features from the complete data set, integrating the multi-modal features by adopting a data fusion technology, and constructing a three-dimensional semantic map; and carrying out graph structure analysis by adopting a conflict detection algorithm based on the three-dimensional semantic graph, determining potential conflict points, and marking the potential conflict points in the three-dimensional semantic graph.
7. The method of claim 6, wherein extracting the multi-modal features from the complete dataset, integrating the multi-modal features using a data fusion technique, and constructing the three-dimensional semantic graph comprises: Taking a car or a person as an entity; Carrying out Kalman filtering processing on the multi-mode characteristics of each entity to obtain track prediction data; calculating cross attention weights of the entity and other entities and environments based on the environment information in the entity and other entities and multi-modal features; All cross attention weights of the same entity are weighted and aggregated to obtain a context sensing feature vector of each entity; and taking each entity as a node, taking the context perception feature vector of the entity as an attribute vector of the node in the map, and taking the relationship between the entity and other entities and the environment as an edge to construct the three-dimensional semantic map.
8. A data analysis system based on cross-modal semantic communication, comprising: the parameter acquisition module is used for acquiring multi-source data of the intersection to be analyzed, wherein the multi-source data comprise vehicle tracks, video images, weather information and pedestrian mobile equipment data; the feature extraction module is used for extracting features of the multi-source data by adopting a semantic extraction technology to obtain key semantic features, and compressing the key semantic features of the data by adopting a compression technology of reinforcement learning optimization to obtain a compressed data set; The data updating module is used for encrypting the compressed data based on a lightweight password hash algorithm fusion blockchain technology, and carrying out real-time interaction on the encrypted data between a base facility and a vehicle by utilizing a low-delay transmission protocol to obtain a complete data set; The data analysis module is used for extracting multi-mode features from the complete data set, integrating the multi-mode features by adopting a data fusion technology, generating a preliminary relationship diagram of dynamic conflict of the intersection, and identifying potential conflict points.
9. The electronic equipment is characterized by comprising at least one processor and a memory, wherein the memory and the processor are connected through a bus; the memory is used for storing one or more programs; a data analysis method based on cross-modal semantic communication as claimed in any one of claims 1 to 7 when the one or more programs are executed by the at least one processor.
10. A readable storage medium having stored thereon an execution program which, when executed, implements a data analysis method based on cross-modal semantic communication according to any one of claims 1 to 7.

Description

Data analysis method and system based on cross-modal semantic communication Technical Field The invention relates to the technical field of intelligent transportation, in particular to a data analysis method and system based on cross-modal semantic communication. Background Along with the acceleration of the urban process, abnormal events such as traffic intersection congestion and accidents are frequent, and the traditional traffic management system has the problems of response lag, insufficient data fusion and the like. In the prior art, although traffic efficiency can be improved through a fixed timing signal lamp (document 1), vehicle speed-based self-adaptive control (document 2) or digital twin optimization (document 3), the problems of real-time semantic fusion and vehicle-road coordination of multi-mode data (such as video, V2X track and weather) are not solved. Document 4 discloses an anomaly detection method of end-cloud cooperation, but does not integrate V2X communication, and cannot realize deep interaction between a vehicle and an infrastructure, so that blind area detection is insufficient and response delay is high. Therefore, a method capable of integrating V2X and cross-mode semantic communication is needed, and accuracy and instantaneity of intelligent analysis of the intersection are improved. Disclosure of Invention In order to solve the problems of insufficient utilization of multi-mode data at a traffic intersection, weak vehicle-road coordination capability and high response delay in the prior art, the invention provides a data analysis method based on cross-mode semantic communication, which comprises the following steps: Acquiring multi-source data of an intersection to be analyzed, wherein the multi-source data comprise vehicle tracks, video images, weather information and pedestrian mobile equipment data; Extracting features of the multi-source data by adopting a semantic extraction technology to obtain key semantic features, and compressing the data key semantic features by adopting a compression technology of reinforcement learning optimization to obtain a compressed data set; Encrypting the compressed data based on a lightweight password hash algorithm fusion blockchain technology, and carrying out real-time interaction on the encrypted data between a base facility and a vehicle by utilizing a low-delay transmission protocol to obtain a complete data set; the multi-modal features are extracted from the complete data set, the multi-modal features are integrated by adopting a data fusion technology, a preliminary relationship diagram of dynamic conflict of the intersection is generated, and potential conflict points are identified. Preferably, the feature extraction of the multi-source data by using a semantic extraction technology to obtain key semantic features includes: Substituting the multi-source data into a pre-constructed multi-dimensional weight scoring system to obtain priority scores of the data streams; Performing primary screening based on priority scores of data flows in combination with a set threshold value to obtain a first data set; Extracting key features by adopting a pre-trained ViT model to obtain key semantic features; the multi-dimensional weight scoring system is constructed by taking priority levels as first-level indexes, setting factors influencing the first-level indexes as second-level indexes, and setting weights for the second-level indexes; the trained ViT model is obtained by training the ViT model based on the marked vehicle track data, weather data and pedestrian mobile device data. Preferably, the compressing technology for reinforcement learning optimization compresses the key semantic features of the data to obtain a compressed data set, including: Defining a state space based on the current bandwidth utilization, network delay, data priority, and historical compression quality score, and taking the state space as an input of the DQN model; defining an action space based on the compression selection, determining decision options of the DQN model from the action space; And designing a reward function based on the PSNR value, the transmission delay and the bandwidth saving rate, guiding the DQN model to learn and balance compression quality, instantaneity and efficiency by the reward function, and finally obtaining a compressed data set. Preferably, training parameters are set, and the DQN model outputs the optimal compression rate according to the network state acquired once at regular intervals. Preferably, the encrypting the compressed data based on the lightweight cryptographic hash algorithm fusion blockchain technology comprises the following steps: calculating the compressed data packet by adopting a lightweight password hash algorithm to generate a hash value with a fixed length; the compressed data, the corresponding hash value, the corresponding time stamp, the corresponding data serial number and the corresponding data packet type