CN-115203241-B - Sky-ground multi-source heterogeneous data management method

CN115203241BCN 115203241 BCN115203241 BCN 115203241BCN-115203241-B

Abstract

According to the sky-ground multi-source heterogeneous data management method, device and storage medium, multi-source heterogeneous data are obtained, corresponding preprocessing operation is carried out on different source data in the multi-source heterogeneous data to obtain corresponding preprocessed data, the preprocessed data are subjected to unified space-time reference, data quality verification and data attribute extraction to obtain bottom characteristic data, attribute information extraction is carried out on the bottom characteristic data to obtain metadata corresponding to the bottom characteristic data, an automatic multi-source data association strategy is utilized based on the metadata to obtain association relations among the multi-source heterogeneous data, and a knowledge graph is generated according to the association relations. According to the application, the bottom characteristic data in a unified format corresponding to the multi-source heterogeneous data can be obtained, and the bottom characteristic data is extracted and the corresponding knowledge graph is generated by utilizing the multi-source data automatic association strategy, so that the input target object can be conveniently and rapidly searched and analyzed.

Inventors

Lv Shouye
ZHAO WEIWEI
WANG YAN
ZENG XIAOLI
HOU SHUQIANG
LIU PENG
LI QIAN

Assignees

北京市遥感信息研究所
北京市遥感信息研究所

Dates

Publication Date: 20260421
Application Date: 20220510
Priority Date: 20220510

Claims (4)

1. A method for managing sky-ground multi-source heterogeneous data, the method comprising: The method comprises the steps of obtaining multi-source heterogeneous data, wherein the multi-source heterogeneous data comprise satellite remote sensing data, aerial remote sensing data, ground video data, network information data, geographic information data and business information data, satellite loading of the satellite remote sensing data comprises full color, multispectral, hyperspectral, infrared and SAR, the aerial remote sensing data comprise visible light, infrared and SAR type aerial remote sensing image data, visible light and infrared type video data, target tracks, flight tracks and text message telemetry data, the geographic information data comprise basic support data and urban traffic data, and the business information data comprise personnel, vehicles, mechanisms and case information; Corresponding preprocessing operations are carried out on different source data in the multi-source heterogeneous data to obtain corresponding preprocessed data, wherein the preprocessing operations are carried out on the satellite remote sensing data, the aerial remote sensing data, the ground video data and the network information data respectively to obtain corresponding preprocessed data; Extracting the preprocessed data through unified space-time reference, data quality check and data attribute to obtain bottom characteristic data corresponding to the multi-source heterogeneous data; Extracting attribute information from the bottom characteristic data to obtain metadata corresponding to the bottom characteristic data, wherein the metadata comprises attribute metadata, structure metadata, semantic metadata, target metadata and event metadata; Based on the metadata, utilizing a multi-source data automatic association strategy to obtain an association relation between the multi-source heterogeneous data; generating a knowledge graph according to the association relation so as to be used for rapidly searching and analyzing the input target object; The preprocessing operation for the satellite remote sensing data comprises the following steps: Processing the obtained satellite remote sensing image by using a standard scene product to obtain a corresponding standard satellite remote sensing image, wherein the processing of the standard scene product comprises full-color data analysis, auxiliary data separation, view field splicing, radiation correction and system-level geometric correction; normalizing the standard satellite remote sensing image to obtain preprocessed data; the preprocessing operation for the aerial remote sensing data comprises the following steps: Performing space three encryption on the satellite remote sensing original photo to obtain an external azimuth element of each photo; Forming a stereopair of each photo by the external azimuth element of each photo; Obtaining corresponding DEM data through a digital elevation model DEM according to the stereopair of each photo; According to the stereopair of each photo, a corresponding orthographic image DOM is manufactured; preprocessing ground video monitoring data, including: Extracting characteristic information and carrying out structural description processing on the ground video monitoring data to obtain first preprocessing data; automatically converging and standardizing the first preprocessing data to obtain second preprocessing data; Carrying out data analysis on the second preprocessing data, and if blacklist data exists, calling metadata and data elements related to the blacklist data so as to obtain more detailed information of the blacklist; preprocessing network information data, including: performing sensitive information decryption and characteristic information extraction on the network information data to obtain third preprocessing data; performing security scanning, security audit and intrusion detection operation on the third preprocessing data to obtain fourth preprocessing data; and carrying out data analysis on the fourth preprocessing data, and if blacklist data exists, calling metadata and data elements related to the blacklist data so as to obtain more detailed information of the blacklist.
2. The method as recited in claim 1, wherein said method further comprises: And performing corresponding preprocessing operation on different source data in the multi-source heterogeneous data to obtain corresponding preprocessed data, wherein the corresponding preprocessed data comprises the geographic information data and the business information data, and the preprocessing operation is performed to obtain corresponding preprocessed data.
3. A device for managing sky-ground multi-source heterogeneous data, wherein the device is configured to implement the method for managing sky-ground multi-source heterogeneous data according to claim 1, the device includes the following modules: the acquisition module is used for acquiring multi-source heterogeneous data; The preprocessing module is used for carrying out corresponding preprocessing operation on different source data in the multi-source heterogeneous data to obtain corresponding preprocessed data; The processing module is used for extracting the preprocessed data through unified space-time reference, data quality check and data attribute to obtain bottom characteristic data corresponding to the multi-source heterogeneous data; The extraction module is used for extracting attribute information of the bottom layer characteristic data to obtain metadata corresponding to the bottom layer characteristic data; the mining module is used for utilizing a multi-source data automatic association strategy based on the metadata to obtain association relations among the multi-source heterogeneous data; And the generation module is used for generating a knowledge graph according to the association relation so as to be used for quickly searching and analyzing the input target object.
4. A non-transitory computer readable storage medium, having stored thereon a computer program, which when executed by a processor, implements the method according to any of claims 1-2.

Description

Sky-ground multi-source heterogeneous data management method Technical Field The application relates to the technical field of big data processing, in particular to a method and a device for managing sky-ground multi-source heterogeneous data and a storage medium. Background With the continuous development of sky-ground data integration, people can acquire remote sensing data of satellites and ground acquisition data in real time. Because of the large sky-ground data volume and different data types of satellite remote sensing and ground acquisition at present, how to efficiently process, analyze and store the multi-source data so as to realize the management of multi-source heterogeneous data is a problem to be solved. Disclosure of Invention The application provides a method, a device and a storage medium for managing sky-ground multi-source heterogeneous data, and provides an efficient method for managing sky-ground multi-source heterogeneous data. An embodiment of a first aspect of the present application provides a method for managing sky-ground multi-source heterogeneous data, including: acquiring multi-source heterogeneous data; corresponding preprocessing operation is carried out on different source data in the multi-source heterogeneous data, and corresponding preprocessed data are obtained; Extracting the preprocessed data through unified space-time reference, data quality check and data attribute to obtain bottom characteristic data corresponding to the multi-source heterogeneous data; Extracting attribute information from the bottom layer characteristic data to obtain metadata corresponding to the bottom layer characteristic data; Based on the metadata, utilizing a multi-source data automatic association strategy to obtain an association relation between the multi-source heterogeneous data; And generating a knowledge graph according to the association relation so as to be used for rapidly searching and analyzing the input target object. An embodiment of a second aspect of the present application provides a device for managing sky-ground multi-source heterogeneous data, including: the acquisition module is used for acquiring multi-source heterogeneous data; The preprocessing module is used for carrying out corresponding preprocessing operation on different source data in the multi-source heterogeneous data to obtain corresponding preprocessed data; The processing module is used for extracting the preprocessed data through unified space-time reference, data quality check and data attribute to obtain bottom characteristic data corresponding to the multi-source heterogeneous data; the extraction module is used for extracting attribute information of the corresponding bottom layer characteristic data to obtain metadata corresponding to the bottom layer characteristic data; the mining module is used for utilizing a multi-source data automatic association strategy based on the metadata to obtain association relations among the multi-source heterogeneous data; And the generation module is used for generating a knowledge graph according to the association relation so as to be used for quickly searching and analyzing the input target object. An embodiment of the third aspect of the present application provides a non-transitory computer readable storage medium, where the non-transitory computer readable storage medium stores a computer program, where the computer program when executed by a processor implements the method as described in the first aspect above. The technical scheme provided by the embodiment of the application at least has the following beneficial effects: According to the sky-ground multi-source heterogeneous data management method, device and storage medium, multi-source heterogeneous data are obtained, corresponding preprocessing operation is carried out on different source data in the multi-source heterogeneous data to obtain corresponding preprocessed data, the preprocessed data are subjected to unified space-time reference, data quality verification and data attribute extraction to obtain bottom characteristic data corresponding to the multi-source heterogeneous data, attribute information extraction is carried out on the bottom characteristic data to obtain metadata corresponding to the bottom characteristic data, an automatic multi-source data association strategy is utilized based on the metadata to obtain association relations among the multi-source heterogeneous data, and a knowledge graph is generated according to the association relations so as to be used for rapidly searching and analyzing an input target object. Therefore, the application extracts the bottom characteristic data in a unified format through unified space-time reference, data quality check and data attribute extraction on the preprocessed data, thereby analyzing the bottom characteristic data in the unified format corresponding to the multi-source heterogeneous data. Meanwhile, according to the knowledge graph ge