CN-122019896-A - Space-time synchronization fusion playback method and device for multisource monitoring data

CN122019896ACN 122019896 ACN122019896 ACN 122019896ACN-122019896-A

Abstract

The application relates to the field of Internet of things and data visualization, in particular to a space-time synchronous fusion playback method and device for multisource monitoring data, wherein the method comprises the steps of receiving a data packet uploaded from a mobile monitoring terminal; the method comprises the steps of analyzing a data packet, associating a time stamp, a position stamp, an event stamp and a storage index of media data under the same equipment identifier to construct a space-time fusion data association relationship, responding to a playback request from a client, searching a target data set based on the space-time fusion data association relationship according to the playback request, wherein the target data set comprises a continuous position track sequence in a target time period, all event markers and the media data storage index associated with each event marker, packaging the target data set into a fusion data packet, and transmitting the fusion data packet to the client for integrated playback by the client. The application is helpful to improve the time-space relevance and the backtracking efficiency.

Inventors

LIANG GUANGCAI
MAO JIAFU
CHENG XUWEI
YANG YONG

Assignees

深圳市途强物联科技有限公司

Dates

Publication Date: 20260512
Application Date: 20260128

Claims (10)

1. A spatio-temporal synchronization fusion playback method of multisource monitoring data, comprising: Receiving a data packet uploaded from a mobile monitoring terminal, wherein the data packet comprises media data subjected to space-time tagging by a time stamp and a position stamp, event marking data subjected to space-time tagging by the time stamp, the position stamp and the time stamp, and the media data comprises video data and audio data; Analyzing the data packet, and extracting a device identifier, a time stamp, a position stamp, an event stamp and a storage index of the media data in the data packet; Correlating the time stamp, the position stamp, the event stamp and the storage index of the media data under the same equipment identifier to construct a space-time fusion data correlation; Responding to a playback request from a client, and according to the playback request, searching a target data set based on the space-time fusion data association relation, wherein the target data set comprises a continuous position track sequence in a target time period, all event marks and media data storage indexes associated with the event marks; And packaging the target data set into a fusion data packet, and issuing the fusion data packet to the client for the client to perform integrated playback.
2. The method for spatio-temporal synchronization fusion playback of multisource monitoring data according to claim 1, wherein the associating the time stamp, the location stamp, the event stamp and the storage index of the media data under the same device identifier, and constructing the spatio-temporal fusion data association relationship comprises: establishing a main index table taking a device identifier as a main key and a time stamp as an ordering field, wherein each record in the main index table is associated with a corresponding position stamp and a media data storage index; generating a time and position mapping relation table based on the main index table, wherein a time stamp is used as a key and a position stamp is used as a value in the mapping relation table; Constructing an event correlation sub-table, wherein an event stamp, a corresponding event trigger time stamp, an event trigger position stamp and a media data storage index are recorded in the event correlation sub-table; and carrying out association binding on the main index table, the time and position mapping relation table and the event association sub-table through the equipment identification and the timestamp field to form a space-time fusion data association relation.
3. The method for spatio-temporal synchronous fusion playback of multisource monitored data according to claim 2, further comprising: If the position stamp corresponding to the continuous time stamp is missing, calculating the coordinate of the missing position stamp by adopting a linear interpolation method based on the coordinates and time intervals of the front and back adjacent effective position stamps; If the media data storage index corresponding to the event stamp is missing, marking the corresponding event as an unassociated media data state, and recording a missing log.
4. The method of claim 1, wherein the responding to the playback request from the client and retrieving the target data set based on the spatiotemporal fusion data association relationship according to the playback request comprises: Responding to a playback request from a client, wherein the playback request of the client at least comprises a device identifier and a time range; positioning a main index table, a time and position mapping relation table and an event association sub-table corresponding to the target monitoring equipment based on the space-time fusion data association relation by taking the equipment identification as a primary retrieval condition; Based on a main index table, a time and position mapping relation table and an event correlation sub-table which correspond to the target monitoring equipment, acquiring all position stamps in the time range to form a continuous position track sequence, and acquiring all event stamps and event stamp correlation information in the time range; and integrating all acquired information to form a target data set.
5. The method for spatiotemporal synchronous fusion playback of multisource monitored data according to claim 1, wherein the client comprises a user interaction interface, and the user interaction interface comprises a first display area and a second display area; The first display area is used for rendering a timeline control based on an absolute time axis, and the second display area is used for rendering an electronic map; dynamically drawing or replaying a device moving track on the electronic map according to the continuous position track point sequence in the fusion data packet; and visually marking event marks on the timeline control and the corresponding positions of the tracks of the electronic map.
6. The method of claim 5, wherein the user interface further comprises a third display area, the third display area being configured to play the media data; When a user drags the time slider of the time line control, the track highlight position of the electronic map is synchronously updated, and the third display area jumps to the media data of the corresponding time point and is played; When a user clicks any event mark on the electronic map, the time sliders of the time line control are synchronously positioned to corresponding time points, and the third display area plays media data with preset time lengths before and after the corresponding time points.
7. The method for time-space synchronous fusion playback of multi-source monitoring data according to claim 1, wherein the mobile monitoring terminal synchronously collects GPS position information at the current moment when collecting the media data, packages the GPS position information as a position stamp, packages the collection time as a time stamp, and binds the data packet with corresponding media data; When the mobile monitoring terminal monitors that the preset event trigger is generated, triggering event mark generation flow, synchronously collecting GPS position information and time information at the moment of event trigger, respectively packaging the GPS position information and the time information into a position stamp and a time stamp corresponding to the event, simultaneously packaging the type information of the triggering event into an event stamp, and associating and binding the event stamp, the position stamp, the time stamp and media data in a corresponding time period, and storing the event stamp, the position stamp, the time stamp and the media data in the data packet together.
8. A spatiotemporal synchronous fusion playback device of multisource monitored data, comprising: the data packet receiving module is used for receiving a data packet uploaded from the mobile monitoring terminal, wherein the data packet comprises media data subjected to space-time tagging through a time stamp and a position stamp and event marking data subjected to space-time tagging through the time stamp, the position stamp and the time stamp, and the media data comprises video data and audio data; The data packet analysis module is used for analyzing the data packet and extracting the equipment identifier, the time stamp, the position stamp, the event stamp and the storage index of the media data in the data packet; The data association module is used for associating the time stamp, the position stamp, the event stamp and the storage index of the media data under the same equipment identifier to construct a space-time fusion data association relationship; The target data retrieval module is used for responding to a playback request from a client and retrieving a target data set based on the space-time fusion data association relation according to the playback request, wherein the target data set comprises a continuous position track sequence in a target time period, all event marks and media data storage indexes associated with the event marks; And the data sending module is used for packaging the target data set into a fusion data packet and sending the fusion data packet to the client for the client to perform integrated playback.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 7 when the computer program is executed.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 7.

Description

Space-time synchronization fusion playback method and device for multisource monitoring data Technical Field The application relates to the technical field of Internet of things and data visualization, in particular to a space-time synchronization fusion playback method and device for multisource monitoring data. Background In the application scenario of a mobile intelligent monitoring device (e.g., an on-board camera, a portable tracker) with GPS and communication functions, the device can continuously or event-triggered generate various heterogeneous data streams, including continuous video streams, continuous audio streams, discrete or timed GPS location point sequences, and discrete event markers triggered by sensors (e.g., vibration, voice control). In the current mainstream solution, video, audio and track data are usually stored and managed separately, a user cannot intuitively know the specific geographic position during recording when viewing video recordings, a field picture and sound of a corresponding position point cannot be directly called when viewing a driving track, association retrieval is difficult, when an abnormal event (such as collision) occurs, the user needs to manually search data in a video recording list, a GPS track log and an event alarm record for the same time period respectively, manual comparison and association are performed, the process is tedious and error-prone, playback experience is split, and the user cannot continuously and contextually trace the whole event process in one view through operating a time axis or map track points. Therefore, a technical scheme capable of automatically fusing multi-source heterogeneous monitoring data and providing a space-time unified view angle to realize intelligent retrieval and playback is needed, so as to solve the problems of poor space-time correlation and low retrieval and backtracking efficiency in the prior art. Disclosure of Invention Based on this, it is necessary to provide a method and a device for playing back multi-source monitoring data by time-space synchronization fusion, which can improve time-space correlation and backtracking efficiency. In a first aspect, the present application provides a method for spatio-temporal synchronous fusion playback of multisource monitoring data, the method comprising: Receiving a data packet uploaded from a mobile monitoring terminal, wherein the data packet comprises media data subjected to space-time tagging by a time stamp and a position stamp, event marking data subjected to space-time tagging by the time stamp, the position stamp and the time stamp, and the media data comprises video data and audio data; Analyzing the data packet, and extracting a device identifier, a time stamp, a position stamp, an event stamp and a storage index of the media data in the data packet; Correlating the time stamp, the position stamp, the event stamp and the storage index of the media data under the same equipment identifier to construct a space-time fusion data correlation; Responding to a playback request from a client, and according to the playback request, searching a target data set based on the space-time fusion data association relation, wherein the target data set comprises a continuous position track sequence in a target time period, all event marks and media data storage indexes associated with the event marks; And packaging the target data set into a fusion data packet, and issuing the fusion data packet to the client for the client to perform integrated playback. In one embodiment, the associating the time stamp, the location stamp, the event stamp and the storage index of the media data under the same device identifier, and constructing the spatio-temporal fusion data association relationship includes: establishing a main index table taking a device identifier as a main key and a time stamp as an ordering field, wherein each record in the main index table is associated with a corresponding position stamp and a media data storage index; generating a time and position mapping relation table based on the main index table, wherein a time stamp is used as a key and a position stamp is used as a value in the mapping relation table; Constructing an event correlation sub-table, wherein an event stamp, a corresponding event trigger time stamp, an event trigger position stamp and a media data storage index are recorded in the event correlation sub-table; and carrying out association binding on the main index table, the time and position mapping relation table and the event association sub-table through the equipment identification and the timestamp field to form a space-time fusion data association relation. In one embodiment, the spatiotemporal synchronous fusion playback method of multi-source monitoring data further comprises: If the position stamp corresponding to the continuous time stamp is missing, calculating the coordinate of the missing position stamp by adopting a linear interpolatio