CN-121982459-A - Security situation awareness method and system based on three-dimensional GIS and panoramic video

CN121982459ACN 121982459 ACN121982459 ACN 121982459ACN-121982459-A

Abstract

The invention provides a security situation sensing method and a security situation sensing system based on a three-dimensional GIS and panoramic video, wherein a three-dimensional scene model of a monitoring area is built on a three-dimensional GIS platform, a plurality of cameras and an environment sensor are connected to achieve space-time alignment and coordinate unification of video streams and multi-source sensing data, a panoramic video picture is generated by projecting video frames onto a three-dimensional scene based on internal parameters and external parameters of the cameras, a target detection model is adopted to detect pedestrians and vehicles and track the vehicles across the cameras to obtain a target three-dimensional track, a time sequence prediction model is utilized to predict target position and local density change, abnormal conditions including crowding, detention and retrograde are judged and early warning is sent out, and a camera deployment and collaborative pursuit is planned through a point location optimization algorithm to reduce monitoring blind areas.

Inventors

AN RAN
JIAN XIA
MA JIANMIN
ZHOU YONG
CHEN ZHIYING

Assignees

中铁第一勘察设计院集团有限公司

Dates

Publication Date: 20260505
Application Date: 20251125

Claims (10)

1. The security situation awareness method based on the three-dimensional GIS and the panoramic video is characterized by comprising the following steps: Data acquisition and modeling, namely importing basic geographic information data of a monitoring area into a three-dimensional GIS platform, constructing a three-dimensional scene model comprising a building, a road, a topography and a monitoring equipment installation position, arranging multiple cameras and sensors such as an inertial measurement unit, wi-Fi sensing equipment and the like, and acquiring multiple paths of video streams and environment sensing data; The time-space alignment and the coordinates are unified, time synchronization is carried out on the multipath video streams and the sensor data, and the position information of the video camera and the sensor is unified under the coordinate system of the three-dimensional GIS scene model, so that the time-space alignment of the video data and the three-dimensional space data is realized; video projection and panorama fusion are carried out, video frames of all cameras are projected onto the surface of the three-dimensional GIS scene or the virtual screen based on the internal and external parameters of the cameras and the unified coordinate information, distortion and brightness are corrected and spliced, and a three-dimensional panorama video picture covering a monitoring area is generated; Target detection and tracking, wherein a target detection model is operated on the panoramic video picture or each camera video frame, targets including pedestrians and vehicles are detected, target categories and two-dimensional boundary frames are output, and the targets are continuously tracked by combining cross-frame data association and cross-camera identity association to obtain a three-dimensional track and a behavior feature sequence of the targets; The time sequence prediction and the abnormality judgment are carried out, the behavior feature sequence is input into a time sequence prediction model, the prediction results of the future position, the speed and the local density of the target are obtained, the abnormal behaviors including congestion, detention and retrograde are judged according to preset rules, and the security situation assessment result and the early warning information are generated; The camera point location optimization and collaborative tracking are carried out, the deployment position and the visual angle parameters of the cameras are calculated through a point location optimization algorithm based on the three-dimensional GIS model of the monitoring area and the candidate camera mounting points, and the cloud platforms and the focal lengths of the cameras are automatically controlled according to the predicted track in the target movement process, so that the collaborative tracking of the target is realized; and displaying the situation and outputting the early warning, superposing and displaying a three-dimensional panoramic video picture, a target track, group density visual information and an early warning mark in the three-dimensional GIS scene, and pushing the early warning information to a security manager through a terminal interface, an acousto-optic warning or a network message.
2. The security situation awareness method based on three-dimensional GIS and panoramic video according to claim 1, wherein in the target detection and tracking step, the target detection model adopts YOLO algorithm, divides an input image into s×s grids, predicts B bounding boxes and confidence levels and category probabilities thereof for each grid, and outputs a tensor with dimensions s×s×b× (5+C), wherein 5 represents a central position x, y, a width w, a height h of the bounding box and a target existence confidence level, and C represents a target category number, and the comprehensive confidence level of a target is calculated according to the following formula: Wherein x and y are normalized coordinates of the center of the bounding box relative to the grid, w and h are normalized values of the width and the height of the bounding box, P (c) is the probability that the target belongs to the class c, and the detected bounding box is used as the input of the follow-up tracking and behavior analysis after non-maximum suppression processing.
3. The security situation awareness method based on three-dimensional GIS and panoramic video according to claim 1, wherein in the time sequence prediction and anomaly determination step, the time sequence prediction model is a long-short-term memory network LSTM, and forgets to gate Input door Candidate cell state State of cell Output door Hidden state The calculation is carried out according to the following formula: Wherein, the The characteristic vector of the target position, speed, acceleration and local density is included for the current moment, 、 Respectively the hidden states of the adjacent moments, 、、、 And 、、、 For the network weight matrix and the bias term, sigma is a Sigmoid function, and tanh is a hyperbolic tangent function, long-term memory of target historical behaviors is realized through the gating mechanism, and a future behavior prediction result is output.
4. The security situation awareness method based on the three-dimensional GIS and the panoramic video according to claim 1, wherein in the point location optimization and collaborative tracking step, a particle swarm optimization algorithm PSO is adopted by the point location optimization algorithm, and iterative updating is performed on a particle location vector formed by camera location and view angle parameters, and the speeds vi (t+1) and the positions xi (t+1) satisfy: Wherein, the For the velocity of the particle i at the instant t, For the position of particle i at time t, w is the inertial weight, 、 In order for the learning factor to be a function of, 、 Is a random number between zero and one, For the historic optimal position of the particle i, And the fitness function is comprehensively defined according to the monitoring coverage rate, the dead zone area and the equipment number for the global optimal position of the particle population, and aims to maximize the monitoring coverage and minimize the dead zone and redundant equipment.
5. The security situation sensing method based on the three-dimensional GIS and the panoramic video, which is disclosed in claim 1, is characterized in that in the step of space-time alignment and coordinate unification, time synchronization between each camera and a server is realized through a PTP precision time protocol or an NTP network time protocol, a unified ground or city coordinate system is adopted on a three-dimensional GIS platform to represent a scene of a monitoring area, and pixel coordinates and three-dimensional GIS coordinates are in one-to-one correspondence according to an external reference matrix and an internal reference matrix of the camera, so that spatial registration of video frames and the three-dimensional scene is realized.
6. The security situation awareness method based on three-dimensional GIS and panoramic video according to claim 1, wherein in the time sequence prediction and anomaly determination step, a crowd density change trend is calculated according to a target track prediction result and a local density prediction result output by an LSTM, when the prediction density exceeds a congestion threshold or the stay time of a target in the same space region exceeds a stay threshold, or the target movement direction is opposite to a prescribed direction, the security situation awareness method is determined as a congestion, stay or retrograde anomaly event, and in the situation display and early warning output step, the anomaly region is highlighted and classified early warning is performed.
7. The security situation awareness method based on the three-dimensional GIS and the panoramic video according to claim 1, wherein in the camera point location optimization and collaborative tracking step, a camera to be relayed is selected in advance and a pan-tilt angle and a zoom multiple thereof are adjusted according to a target future position predicted by an LSTM model in an operation stage, so that a target at the edge of a field of view of a current camera is located in a preset attention area in a next field of view of the camera, and continuous seamless cross-lens tracking is realized in a multi-camera scene.
8. Safety situation sensing system based on three-dimensional GIS and panoramic video, its characterized in that includes: The data acquisition module is used for acquiring basic geographic information data of a monitoring area, constructing a three-dimensional GIS scene model and acquiring multiple paths of sensor data comprising a video stream of a camera, an inertial measurement unit and Wi-Fi sensing equipment; The space-time alignment and coordinate unification module is used for carrying out time synchronization on the video stream and the sensor data and unifying the position information of the camera and the sensor into a coordinate system of the three-dimensional GIS scene model; The three-dimensional mapping and fusing module is used for projecting the multipath video frames to the surface of the three-dimensional GIS scene or the virtual screen based on the internal and external parameters of the camera, and carrying out distortion correction and brightness adjustment to generate a three-dimensional panoramic video picture; the target detection and tracking module is used for executing target detection, frame-crossing tracking and camera-crossing identity association on the panoramic video picture or each camera video frame and outputting a three-dimensional track and a behavior characteristic sequence of a target; The situation analysis and anomaly prediction module is used for inputting the behavior characteristic sequence into a time sequence prediction model to obtain a target future behavior and population density prediction result, and judging the anomaly behavior and evaluating the safety situation according to the result; The point location planning and collaborative tracking module is used for calculating camera deployment positions and visual angle parameters by adopting a point location optimization algorithm based on a three-dimensional GIS scene and candidate camera installation points, and controlling cloud platforms and focal lengths of a plurality of cameras according to target prediction tracks in a monitoring process so as to realize collaborative tracking; The situation display and interaction module is used for displaying panoramic video pictures, target tracks, density visualization and early warning information in a three-dimensional GIS scene in a superposition mode, and providing interaction interfaces of early warning viewing, history playback and parameter configuration for safety management staff.
9. An electronic device, comprising: And at least one memory communicatively coupled to the processor, wherein the memory stores program instructions executable by the processor, the processor invoking the program instructions capable of performing the three-dimensional GIS and panoramic video based security posture awareness method of any one of claims 1 to 7.
10. A storage medium comprising a stored program which when executed by a processor implements the security posture awareness method based on three-dimensional GIS and panoramic video according to any one of claims 1 to 7.

Description

Security situation awareness method and system based on three-dimensional GIS and panoramic video Technical Field The invention belongs to the technical field of intelligent video monitoring and security situation awareness, and particularly relates to a security situation awareness method and system based on a three-dimensional GIS and panoramic video. Background The invention belongs to the technical field of intelligent video monitoring and security situation awareness, and particularly relates to a security situation awareness method and system based on a three-dimensional GIS and panoramic video. With the increase in the number of urban public spaces, transportation hubs, parks and key places, video surveillance systems have become the infrastructure for security protection. The traditional system relies on single or a small number of fixed cameras to monitor the planar video, and the monitoring center mainly acquires the field information through two-dimensional picture playback or manual round robin. In a scene with a large range, a complex structure or dense people flow, the system has the problems of limited visual angle coverage, multiple blind areas, easy shielding of targets, difficult visual reflection of the overall spatial relationship of the environment and the like, and is difficult for monitoring personnel to understand the scene situation timely and accurately. In order to improve the space expression capability, part of the system starts to introduce GIS or three-dimensional scene modeling technology into video monitoring, maps the information of camera positions, buildings, roads and the like into the three-dimensional scene to realize the associated display of video pictures and space models, and also has the system to try to combine panoramic video or multi-camera spliced video with the three-dimensional model for improving the perception range of a large-scale scene. However, the existing three-dimensional GIS and video fusion application stays in the visual level, the time synchronization, coordinate unification and accurate projection mapping support of multi-source data are limited, the accurate association of target positions, motion tracks and environmental objects is difficult to be carried out under a unified coordinate system, and the detailed description of group behaviors and local density changes in complex scenes is lacking. On the other hand, the security situation awareness system gradually introduces algorithms such as target detection, behavior recognition and the like, and is used for automatically extracting activity information of objects such as people, vehicles and the like from the video. However, the real-time performance and reliability of the existing system in a large-scale and multi-camera scene are still insufficient, firstly, target detection and tracking are based on single-camera visual angles, cross-camera identity correlation and seamless relay capability are limited, secondly, modeling of historical tracks and behavior sequences is simpler, effective prediction of crowd behaviors such as crowding, detention and retrograde is difficult, thirdly, a system optimization method combining a three-dimensional scene is lacking in the aspect of camera layout, camera points and visual angles are determined by manual experience, and coverage rate imbalance and blind areas are caused. In addition, the data from different cameras and various sensors have differences in time reference, space coordinates and data formats, and the existing multi-source data fusion technology is difficult to realize high-precision space-time alignment and unified management in a complex environment, so that continuous and integral security situation analysis on a large-scale scene is limited. In summary, the prior art still has the defects in the aspects of depth fusion of three-dimensional GIS and video data, continuous tracking of cross-camera targets, group behavior prediction, optimization of camera point positions and the like, and a method and a system for realizing intelligent perception and early warning of complex scene security situations by fusing panoramic video and multi-source perception data under a unified three-dimensional space frame are needed. Disclosure of Invention The invention provides a security situation sensing method and system based on a three-dimensional GIS and a panoramic video, which are used for solving the problems of insufficient fusion of a three-dimensional scene and video data, difficult space-time alignment of multisource sensing data, insufficient intelligent early warning capability of continuous tracking and group abnormal situations across cameras and the like in the prior art. In order to achieve the above purpose, the technical scheme of the invention is as follows: in a first aspect, the invention provides a security situation awareness method based on a three-dimensional GIS and a panoramic video, which comprises the following steps: Da