CN-121999434-A - Intelligent decision system and method based on space-time diagram neural network

CN121999434ACN 121999434 ACN121999434 ACN 121999434ACN-121999434-A

Abstract

The invention relates to the technical field of security monitoring, in particular to an intelligent decision system and method based on a space-time diagram neural network. The method comprises the steps of extracting coordinate information of each node from a monitoring video, constructing an initial graph sequence containing time dimension, obtaining space-time characteristic representation through a space-time graph neural network, extracting abnormal propagation characteristics reflecting joint abnormal incidence relation, forming time sequence dependent data describing an abnormal evolution process, generating time dependent characteristics, determining gesture change trend vectors, screening to obtain an abnormal candidate sequence, further carrying out characteristic enhancement on the abnormal candidate sequence, obtaining enhanced characteristic representation, determining the occurrence position and time of abnormal behaviors according to the enhanced characteristic representation, and making intervention decisions. The invention solves the problems that the abnormal behavior is difficult to accurately identify, locate and intelligently intervene in decision making under the complex monitoring scene, and realizes the efficient detection, accurate space-time location and intelligent intervention decision making of the abnormal behavior.

Inventors

MA WEIPING
Ma Rongyang

Assignees

深圳市千里马安防软件工程有限公司

Dates

Publication Date: 20260508
Application Date: 20260123

Claims (10)

1. An intelligent decision making system based on a space-time diagram neural network, the system comprising: The image structure constructing unit is used for acquiring a continuous video frame sequence from the monitoring video, extracting the coordinate information of each joint point of each human body in each video frame, and constructing an image structure according to the coordinate information of each joint point; The abnormal propagation unit is used for inputting a space-time diagram neural network to the initial diagram sequence, acquiring space-time characteristic representation, extracting abnormal propagation characteristics reflecting joint abnormal association relation based on the space-time characteristic representation, arranging the abnormal propagation characteristics according to the video frame time sequence, and constructing time sequence dependent data for describing an abnormal evolution process; The trend judging unit is used for generating time-dependent features and determining gesture change trend vectors based on the time-series dependent data; The abnormal behavior locating unit is used for carrying out feature enhancement on the abnormal candidate sequence, obtaining enhanced feature representation and extracting corresponding frame sequence fragments, extracting descriptors from the enhanced feature representation, and determining and making an intervention decision based on the occurrence position and time of the abnormal behavior.
2. The system according to claim 1, wherein the graph structure constructing unit for constructing an initial graph sequence containing time dimension information includes: extracting a continuous video frame sequence from a monitoring video, processing the video frame sequence through a preset gesture estimation model to obtain human body joint point coordinate data in each frame, constructing a graph structure by taking joint points as graph nodes according to the human body joint point coordinate data, generating an adjacent matrix through bone edge connection, and arranging the graph structure according to time sequence to form an initial graph sequence.
3. The system of claim 1, wherein the anomaly propagation unit to obtain the spatio-temporal feature representation comprises: The method comprises the steps of inputting an initial graph sequence into a space-time graph neural network, capturing dynamic relations among joint nodes by adopting space-time convolution operation, extracting a high-level space mode through graph convolution layer stacking, obtaining feature information in a space dimension, stacking multi-frame sequences along the time dimension, converging adjacent matrix information by a message transfer mechanism, combining connection relations among the joint nodes with time change through the message transfer mechanism to generate comprehensive features, and forming space-time feature representation according to the comprehensive features.
4. A system as claimed in claim 3, characterized by an anomaly propagation unit for constructing time-series dependent data for describing the evolution process of the anomaly behaviour, comprising: Extracting abnormal propagation characteristics from the space-time characteristic representation through a message transmission mechanism, capturing abnormal association among joint nodes, arranging the abnormal propagation characteristics in time sequence, converting the abnormal propagation characteristics into time sequence dependent data, recording joint node representation changes of each time step aiming at the time sequence dependent data, constructing a joint node representation sequence through the time sequence dependent data, and retaining the time correlation of the abnormal propagation characteristics according to the joint node representation sequence.
5. The system of claim 1, wherein the trend determination unit for generating the time-dependent feature and determining the posture change trend vector comprises: The method comprises the steps of inputting time sequence dependent data into a long-period memory network to update hidden states, retaining history evolution information, controlling forgetting degree of corresponding gating memory states through a forgetting gating mechanism, screening key history information related to current gesture change, adjusting the updating proportion of current time step new gesture information parameters and gating memory states based on the key history information through an input gating mechanism, updating the gating memory states, generating time dependent features according to the updated gating memory states, capturing change rules of human gestures in time dimension, and determining gesture change trend vectors through the time dependent features and the change rules.
6. The system of claim 5, wherein the trend determining unit for obtaining the anomaly candidate sequence comprises: Acquiring the movement direction and amplitude data of the joint according to the gesture change trend vector, predicting gesture change amplitude through an output gating mechanism if the amplitude data corresponding to the movement direction of the joint exceeds a preset threshold, marking an abnormal trend according to the gesture change amplitude and recording a corresponding time position index, and generating an abnormal candidate sequence through the time position index.
7. The system of claim 1, wherein the abnormal behavior localization unit for acquiring the enhanced feature representation and extracting the corresponding frame sequence segments comprises: Focusing key joint node representation by adopting an attention mechanism aiming at the abnormal candidate sequence, highlighting abnormal related characteristics, aggregating enhanced space-time characteristic representation through graph node characteristics, generating enhanced characteristic representation, extracting frame sequence fragments corresponding to abnormal trends according to the enhanced characteristic representation, recording characteristic change information of the key joint node aiming at the frame sequence fragments, and reserving space-time detail characteristics of abnormal behaviors through the enhanced characteristic representation.
8. The system of claim 7, wherein the abnormal behavior localization unit to determine and make an intervention decision based on the abnormal behavior occurrence location and time comprises: Extracting a lightweight descriptor from the enhanced feature representation to represent a feature mode of abnormal behavior, classifying the lightweight descriptor through a classifier to confirm an abnormal type, acquiring an original video frame number corresponding to each frame of joint node representation if the abnormal type is confirmed, recording a timestamp of occurrence of the abnormal behavior according to the original video frame number, determining a specific occurrence position and time of the abnormal behavior through the timestamp and the frame number, and generating an abnormal intervention decision aiming at the occurrence position and time of the abnormal behavior.
9. A space-time diagram neural network based intelligent decision method implemented based on the system according to any one of claims 1-8, characterized in that the method comprises: Step S1, acquiring a continuous video frame sequence from a monitoring video, extracting coordinate information of each joint point of each human body in each video frame, and constructing a graph structure according to the coordinate information of each joint point; S2, inputting a space-time diagram neural network to the initial diagram sequence, acquiring space-time characteristic representation, extracting abnormal propagation characteristics reflecting joint abnormal association relation based on the space-time characteristic representation, arranging the abnormal propagation characteristics according to a video frame time sequence, and constructing time sequence dependent data for describing an abnormal evolution process; Step S3, generating time-dependent features and determining gesture change trend vectors based on the time-series dependent data; And S4, carrying out feature enhancement on the abnormal candidate sequence, obtaining enhanced feature representation and extracting corresponding frame sequence fragments, extracting descriptors from the enhanced feature representation, and determining and making an intervention decision based on the occurrence position and time of the abnormal behavior.
10. A computer readable storage medium having instructions stored thereon, which when executed by a processor, implement the method of claim 9.

Description

Intelligent decision system and method based on space-time diagram neural network Technical Field The invention relates to the technical field of security monitoring, in particular to an intelligent decision system and method based on a space-time diagram neural network. Background Along with the wide application of the video monitoring system in the fields of public safety, urban management, industrial production and the like, how to accurately identify abnormal behaviors from a monitoring video and provide reliable basis for subsequent alarming and treatment becomes a key technical problem in an intelligent decision-making system. In the prior art, modeling is mostly carried out based on single-frame image features or simple time sequence features in the abnormal behavior analysis process, and the inherent association relation of human behaviors in the spatial structure and time evolution level is difficult to fully describe. Under a complex monitoring scene, the topological relation between human joints and the dynamic characteristics of the topological relation between human joints along with the change of the dynamic characteristics are not effectively utilized, so that the recognition accuracy of abnormal behaviors is low, and the time and the space positions of the abnormal behaviors are difficult to accurately position. In addition, the abnormal behavior generally has obvious time-space evolution characteristics, the formation process of the abnormal behavior relates to gradual propagation of abnormal association between human body nodes, and the abnormal trend is difficult to model stably only by depending on static or local time sequence characteristics, so that the practical value of an abnormal detection result in an intelligent decision scene is limited. Based on the problems, the invention provides the method, which analyzes the propagation process and the development trend of the abnormal behavior by constructing the dynamic graph structure of the human body joint point and combining the modeling space topological relation and the time evolution characteristic, and outputs the specific time and the space information of the occurrence of the abnormality so as to improve the accuracy and the reliability of intelligent decision in a complex monitoring scene. Disclosure of Invention The invention provides an intelligent decision system and method based on a space-time diagram neural network, which are used for solving the problem that the traditional monitoring system is difficult to accurately identify and position abnormal behaviors, and not only can the abnormal behaviors be detected in real time but also accurate intervention decisions can be automatically generated through the guidance of intelligent decisions, so that the system can perform efficient and intelligent response under a complex monitoring scene, and the practicability and safety of the monitoring system are greatly improved. In a first aspect, the present invention provides an intelligent decision system based on a space-time diagram neural network, the system comprising: The image structure constructing unit is used for acquiring a continuous video frame sequence from the monitoring video, extracting the coordinate information of each joint point of each human body in each video frame, and constructing an image structure according to the coordinate information of each joint point; The abnormal propagation unit is used for inputting a space-time diagram neural network to the initial diagram sequence, acquiring space-time characteristic representation, extracting abnormal propagation characteristics reflecting joint abnormal association relation based on the space-time characteristic representation, arranging the abnormal propagation characteristics according to the video frame time sequence, and constructing time sequence dependent data for describing an abnormal evolution process; The trend judging unit is used for generating time-dependent features and determining gesture change trend vectors based on the time-series dependent data; The abnormal behavior locating unit is used for carrying out feature enhancement on the abnormal candidate sequence, obtaining enhanced feature representation and extracting corresponding frame sequence fragments, extracting descriptors from the enhanced feature representation, and determining and making an intervention decision based on the occurrence position and time of the abnormal behavior. As a preferred embodiment of the present invention, a graph structure construction unit, configured to construct an initial graph sequence including time dimension information, includes: extracting a continuous video frame sequence from a monitoring video, processing the video frame sequence through a preset gesture estimation model to obtain human body joint point coordinate data in each frame, constructing a graph structure by taking joint points as graph nodes according to the human body joint point coo