Search

CN-121999578-A - Regional disaster intelligent early warning and emergency response linkage method integrating video stream and sensor data

CN121999578ACN 121999578 ACN121999578 ACN 121999578ACN-121999578-A

Abstract

The invention relates to the technical field of disaster monitoring and early warning, in particular to an intelligent early warning and emergency response linkage method for regional disasters, which integrates video streams and sensor data, and comprises the following steps of 1, data acquisition; the method comprises the steps of step 2 of preprocessing data, step 3 of extracting features, step 4 of fusing video features and sensor features to generate fused features, dynamically adjusting weights of the video features and the sensor features by adopting an attention mechanism, calculating the weights based on confidence coefficient of the sensor data, step 5 of training a disaster early warning model, step 6 of real-time early warning, step 7 of emergency response linkage and step 8 of feedback optimization. The video stream and the sensor data are fused in depth, the data weight is dynamically adjusted by using an attention mechanism, the accuracy and the instantaneity of disaster early warning are improved, and the manual intervention and the response delay are reduced by automatic emergency response linkage.

Inventors

  • ZHANG JINHUA
  • WANG YUNYAN

Assignees

  • 北京潮耘科技有限公司

Dates

Publication Date
20260508
Application Date
20260127

Claims (10)

  1. 1. The regional disaster intelligent early warning and emergency response linkage method integrating the video stream and the sensor data is characterized by comprising the following steps of: The method comprises the steps of 1, collecting video stream data and sensor data in an area, wherein the video stream data are from a plurality of cameras deployed in the area, the sensor data comprise seismic sensor data, water level sensor data and meteorological sensor data, the video stream data and the sensor data are synchronously collected, and time stamp alignment is realized through a network time protocol; Step 2, data preprocessing, namely denoising, enhancing and stabilizing the acquired video stream data, and performing data cleaning, outlier detection and normalization on the acquired sensor data; Step 3, extracting video features from the preprocessed video stream data, extracting video features by using a convolutional neural network model, extracting sensor features from the preprocessed sensor data, and calculating time domain features; Step 4, a data fusion step, in which video features and sensor features are fused to generate fusion features, the weights of the video features and the sensor features are dynamically adjusted by adopting an attention mechanism, and the weights are calculated based on the confidence coefficient of the sensor data; Training a disaster early-warning model by using historical data, wherein the historical data comprises video stream data, sensor data and a disaster label, and the disaster early-warning model adopts a deep neural network; Step 6, a real-time early warning step, namely inputting real-time fusion characteristics into a trained disaster early warning model to generate early warning information, and triggering early warning if the disaster probability exceeds a preset threshold value, wherein the early warning information comprises disaster types, disaster positions, expected influence ranges and severity; step 7, an emergency response linkage step, namely automatically triggering emergency response according to the early warning information, mapping disaster types to response actions through a response rule base, wherein the response actions comprise evacuation instructions and resource allocation; and 8, feeding back an optimization step, optimizing a disaster early warning model according to the early warning result and the actual disaster situation, and updating the model by using an online learning algorithm.
  2. 2. The method for intelligent early warning and emergency response linkage of regional disasters by integrating video streams and sensor data according to claim 1, wherein in step 1, the video stream data come from a plurality of cameras deployed in a region, and the cameras collect the video stream at a fixed frame number per second; The sensor data comprise seismic sensor data acquisition vibration acceleration, water level sensor data acquisition water level height, weather sensor data acquisition rainfall and wind speed; The sensor nodes transmit data through a wireless network, wherein the data comprises a time stamp, sensor position information and measured values; The video stream data and the sensor data are synchronously collected, the time stamp alignment is realized through a network time protocol, and the network time protocol is synchronized through a global positioning system clock source; the key points in the coverage area of the deployment position of the camera comprise the river along the line, the dense building area and the mountain slope, and the deployment density of the sensor nodes is adjusted according to the regional topography and the disaster history data.
  3. 3. The method for intelligent early warning and emergency response linkage of regional disasters by integrating video streams with sensor data according to claim 1, wherein in step 2, preprocessing of the video stream data comprises denoising, enhancing and stabilizing: Denoising the video frame by using Gaussian filtering, wherein the kernel size of the Gaussian filtering is adaptively adjusted according to the image noise level, and the image noise level is obtained by calculating the gray value variance of the video frame; Enhancement processing uses histogram equalization to enhance the video frame; the stabilization process uses motion estimation and compensation for video stabilization; Preprocessing of sensor data includes data cleaning, outlier detection and normalization: Detecting abnormal values of sensor data by using a sliding window method in data cleaning, wherein the size of the sliding window is determined according to the sampling frequency of the sensor, the abnormal values are identified by calculating standard deviation of the data in the window, and if the data point deviation average exceeds three times of the standard deviation, the data point deviation average is marked as abnormal; The missing values are filled using linear interpolation; the normalization process scales the sensor data to zero mean and unit variance, and the normalization parameters are calculated from the historical data.
  4. 4. The method for intelligent early warning and emergency response linkage of regional disasters by integrating video streams and sensor data according to claim 1, wherein in step 3, a convolutional neural network model is used for video feature extraction: Inputting the preprocessed video frames into a pretrained convolutional neural network, wherein the convolutional neural network comprises a plurality of convolutional layers and a pooling layer, the convolutional layers use rectifying linear unit activation functions, the pooling layer uses maximum pooling operation, and the convolutional neural network outputs high-level characteristic representations comprising edge characteristics, texture characteristics and motion characteristics; Sensor feature extraction time domain features are calculated from the preprocessed sensor data by segmenting the sensor data using sliding windows, window size is determined based on data sampling frequency and disaster type, mean, variance, trend coefficients and peak features are calculated in each window, trend coefficients are obtained by fitting the slope of the data sequence in the window by linear regression, and peak features are obtained by identifying local maxima in the window and calculating their magnitudes.
  5. 5. The method for intelligent early warning and emergency response linkage of regional disasters by fusing video streams and sensor data according to claim 1, wherein in step 4, dynamic fusion is realized by adopting an attention mechanism in data fusion: Designing an attention network, wherein the attention network comprises a full connection layer and a softmax function, inputting sensor characteristics and video characteristics, and outputting weight vectors; The weight vector is calculated based on the confidence coefficient of the sensor data, the confidence coefficient is obtained through the signal-to-noise ratio of the sensor data, the signal-to-noise ratio is obtained through calculating the ratio of the signal power to the noise power of the sensor, and the noise power is extracted from the abnormal value detection result of the preprocessing step; The fusion process comprises the steps of splicing video features and sensor features into joint feature vectors, and adjusting weights of all components in the joint feature vectors through an attention network to generate weighted fusion feature vectors; training of the attention network uses historical fusion data to optimize network parameters by minimizing reconstruction errors.
  6. 6. The regional disaster intelligent early warning and emergency response linkage method integrating video streams and sensor data according to claim 1, wherein in step 5, a disaster early warning model trains use history data, the history data comprises video stream data, sensor data and corresponding disaster labels, and the disaster labels comprise flood, earthquake and no disaster; the disaster early warning model adopts a deep neural network, the deep neural network comprises an input layer, a plurality of hidden layers and an output layer, the input layer receives the fusion feature vector, and the output layer outputs disaster probability by using a softmax function; The training process uses a cross entropy loss function and a gradient descent optimization algorithm, the gradient descent optimization algorithm updates network parameters through back propagation, and the training iteration number is determined according to the historical data amount until the loss function converges; The historical data is divided into a training set and a verification set, and the verification set is used for adjusting the model super parameters.
  7. 7. The method for intelligent early warning and emergency response linkage of regional disasters by integrating video streams and sensor data according to claim 1, wherein in step 6, the real-time early warning step comprises: inputting the real-time fusion characteristics into a trained disaster early warning model, and outputting disaster probability including disaster type probability and severity probability by the model; the preset threshold is adjusted according to the false alarm rate and the missing alarm rate of the historical data, and if the disaster probability exceeds the preset threshold, the early warning is triggered; The early warning information comprises a disaster type, a disaster position, an expected influence range and a severity degree, wherein the disaster position is obtained by fusing a sensor position and a video analysis result, and the expected influence range is calculated by combining geographic information system data with the disaster type; the early warning information is published through a variety of channels including short messages, broadcast and mobile application push.
  8. 8. The method for intelligent early warning and emergency response linkage of regional disasters by integrating video streams and sensor data according to claim 1, wherein in step 7, the step of emergency response linkage comprises: Designing a response rule base, and mapping disaster types to response actions by the response rule base; The linkage mechanism sends instructions to an emergency department through a communication network, wherein the instructions comprise action plans and resource allocation; The execution state of the response action is monitored and fed back to the system, and the response effect is verified in real time by monitoring the sensor data and the video stream data; The emergency response linkage is integrated with an external system, including a traffic control system and a medical rescue system.
  9. 9. The method for intelligent early warning and emergency response linkage of regional disasters by integrating video streams and sensor data according to claim 1, wherein in step 8, the feedback optimization step comprises: Collecting feedback data, wherein the feedback data comprises early warning accuracy data and emergency response effect data, the early warning accuracy data is obtained by comparing an early warning result with an actual disaster record, and the emergency response effect data is obtained by calculating response time and loss; Updating a disaster early warning model by using an online learning algorithm, wherein the online learning algorithm adjusts model parameters through incremental data; the feedback data is also used to optimize the attention network in the data fusion step and the parameters in the preprocessing step.
  10. 10. The method for intelligent early warning and emergency response linkage of regional disasters by combining video streams with sensor data according to any one of claims 1 to 9, wherein in step 4, the data combining step further comprises multi-modal feature alignment: before fusion, performing time alignment and space alignment on the video features and the sensor features, wherein the time alignment adjusts the time resolution of the feature sequence through an interpolation method, and the space alignment associates the video features and the sensor features to the same geographic position through geographic coordinate mapping; In the step 7, the emergency response linkage step further comprises the step of dynamically adjusting response rules, wherein a response rule base is automatically updated according to feedback data, response effects are evaluated through a rule engine, and the mapping relation between disaster types and response actions is optimized; in step 8, the feedback optimization step further includes model interpretation analysis, wherein the decision process of the disaster early warning model is analyzed by using the visualization tool, so that the early warning result can be interpreted and used for improving the feature extraction and data fusion strategies.

Description

Regional disaster intelligent early warning and emergency response linkage method integrating video stream and sensor data Technical Field The invention relates to the technical field of disaster monitoring and early warning, in particular to an intelligent early warning and emergency response linkage method for regional disasters by integrating video streams and sensor data. Background Regional disaster early warning systems are key technologies for reducing natural disaster loss, and the prior art mainly relies on a single sensor network or a video monitoring system. The sensor network, including seismic sensors, water level sensors, weather sensors, etc., can provide real-time physical parameter data such as vibration acceleration, water level height, and rainfall. These sensors are typically deployed at fixed points, transmitting data over a wireless network, enabling point-like monitoring. However, the coverage area of the sensor network is limited, the disaster situation of a large area cannot be comprehensively reflected, and sensor data is easily subjected to environmental interference, such as electromagnetic interference or physical damage, so that false alarm or missing alarm is caused. For example, the flood early warning is taken as an example, the water level sensor can only monitor the water level change of a specific point, but cannot capture the overall flooding trend or the tributary influence of the river, so that the early warning accuracy is lower in complex terrains. The video monitoring system collects visual information through a camera, and performs motion detection or object identification by using an image processing algorithm to identify disaster signs such as abnormal water flow or building collapse. Video data can provide intuitive spatial information, but conventional video analysis techniques are limited by environmental conditions such as light changes, rainy and foggy weather, or camera occlusion, resulting in poor recognition robustness. In addition, the video data volume is big, and processing delay is high, is difficult to satisfy the real-time requirement of calamity early warning. There are some methods in the prior art that attempt to fuse sensor data with video data, for example by a weighted average or decision-level fusion strategy, to simply combine the sensor output with the video analysis results. However, these methods lack an adaptive mechanism, cannot dynamically adjust the data weight, and the fusion effect is significantly reduced when the sensor fails or the video quality is reduced. Another problem is insufficient linkage of the early warning system with the emergency response. Existing systems typically rely on manual judgment to initiate an emergency response after generating the warning information, which increases response time and may lead to disaster expansion in emergency situations. For example, in earthquake early warning, even if the sensor detects vibration, the evacuation instruction can be triggered only after manual confirmation, and the golden rescue time is delayed. In addition, the existing data processing method mostly adopts traditional machine learning algorithms, such as a support vector machine or a decision tree, which require a large amount of labeling data training, have poor generalization capability in a new environment, and cannot adapt to a region-specific disaster mode. In summary, the prior art has the following problems that the fusion of sensor data and video data is insufficient, high-precision early warning cannot be realized, the linkage of an early warning system and emergency response is insufficient, the response delay is high, the data processing method is simple, and self-adaption and intellectualization are not available. Therefore, the invention provides an intelligent early warning and emergency response linkage method for regional disasters, which integrates video streams and sensor data, and solves the problems through a depth data fusion and automatic response mechanism. Disclosure of Invention Based on the above purpose, the invention provides an intelligent early warning and emergency response linkage method for regional disasters by fusing video streams and sensor data, which comprises the following steps: The method comprises the steps of 1, collecting video stream data and sensor data in an area, wherein the video stream data are from a plurality of cameras deployed in the area, the sensor data comprise seismic sensor data, water level sensor data and meteorological sensor data, the video stream data and the sensor data are synchronously collected, and time stamp alignment is realized through a network time protocol; Step 2, data preprocessing, namely denoising, enhancing and stabilizing the acquired video stream data, and performing data cleaning, outlier detection and normalization on the acquired sensor data; Step 3, extracting video features from the preprocessed video stream data, extracting video f