Search

CN-121998594-A - Unmanned on duty processing system towards multisource remote sensing data

CN121998594ACN 121998594 ACN121998594 ACN 121998594ACN-121998594-A

Abstract

The invention discloses an unmanned processing system for multi-source remote sensing data, and relates to the technical field of remote sensing data processing. The system comprises an automatic production module, an intelligent management module, a cooperative control module and a three-level storage pool, wherein the automatic production module executes end-to-end modeling processing from preprocessing to product generation on multi-source remote sensing data based on an programmable workflow template and supports parallel task level and data level, the intelligent management module realizes state sensing and self-adaptive resource scheduling through metadata driving and event monitoring, and the cooperative control module serves as a central center to provide a unified interface and perform centralized management configuration. And the three-level storage pool realizes automatic data circulation under the control of a system. The invention realizes the full-flow unmanned automatic processing from the access of the optical and radar remote sensing data to the output of the standard product through event driving and intelligent scheduling, and remarkably improves the production efficiency, the manageability and the expandability of the system.

Inventors

  • JIAO XUELEI
  • WANG QIAN
  • LIU JINGWEN
  • ZHANG YI
  • FU JIAO

Assignees

  • 南方(北京)遥感科技有限公司

Dates

Publication Date
20260508
Application Date
20260129

Claims (10)

  1. 1. An unattended processing system for multi-source remote sensing data, comprising: The automatic production module, the intelligent management module, the cooperative control module and the three-level storage pool are connected through a data handling flow; the tertiary storage pool comprises an original storage pool, an intermediate storage pool and an achievement storage pool; The original storage pool is provided with a monitoring unit which is in communication connection with the message middleware; The intelligent management module comprises a metadata base and a dispatcher, and the dispatcher is in communication connection with the message middleware and the metadata base; The automatic production module comprises a workflow engine, a processing algorithm library and a distributed computing node cluster, wherein the workflow engine is in communication connection with the scheduler and the configuration center and is used for loading a processing flow template and instantiating an atomic task queue; the processing algorithm library is packaged with an optical image processing routine and a radar image processing routine; The cooperative control module comprises a service gateway, a configuration center and a state synchronization unit; the service gateway receives an external request, the configuration center stores the processing flow template, and the state synchronization unit is connected with the metadata base and the Web front end.
  2. 2. The unattended processing system for multi-source remote sensing data according to claim 1, wherein the workflow engine is ApacheAirflow engine; one of the distributed computing node cluster operations APACHESPARK and Dask frameworks; the message middleware is one of Redis and RabbitMQ; The processing algorithm library is integrated with GDAL library, orfeoToolbox library and SNAP tool kit.
  3. 3. The system of claim 2, wherein the process flow templates define dependencies and parallelism between data processing steps in a directed acyclic graph.
  4. 4. The unattended processing system for multi-source remote sensing data according to claim 1, wherein the metadata base is a postgreSQL database with PostGIS extensions deployed; The snoop unit monitors the specified directory of the original storage pool based on the Watchdog library.
  5. 5. The system of claim 1, wherein the scheduler is configured with a first predetermined threshold and a second predetermined threshold; and triggering computing resource capacity expansion operation by the scheduler when the average CPU utilization rate of the system is continuously higher than the first preset threshold or the length of the atomic task queue exceeds the second preset threshold.
  6. 6. The unattended processing system for multi-source remote sensing data according to claim 1, wherein the service gateway is RESTfulAPI gateway; The state synchronization unit establishes a persistent connection with the Web front end based on the WebSocket protocol.
  7. 7. The unattended processing system for multi-source remote sensing data according to claim 1, wherein the remote sensing data received by the original storage pool comprises mainstream optical satellite image data and radar satellite image data at home and abroad; The image product output by the result storage pool is a digital orthographic image, a digital earth surface model or a digital elevation model.
  8. 8. The unmanned processing system for multi-source remote sensing data of claim 1, wherein the radar image processing routine comprises code means for performing format conversion, multi-view processing, adaptive filtering, and RPC parameter generation.
  9. 9. The unattended processing system for multi-source remote sensing data according to claim 1, wherein the processing algorithm library further comprises an intelligent embedded line generation module; The intelligent mosaic line generating module constructs a grid diagram for two input overlapped images, assigns a first weight for grid edges based on a semantic classification layer, assigns a second weight for the grid edges based on pixel contrast, and searches a path with the minimum total weight from one side to the other side of an overlapped area as the mosaic line through a diagram searching algorithm.
  10. 10. The unattended processing system for multi-source remote sensing data according to claim 1, wherein the configuration center supports addition, deletion and modification of the processing flow template; the processing algorithm library supports plug-in integration of new algorithm modules.

Description

Unmanned on duty processing system towards multisource remote sensing data Technical Field The invention relates to the technical field of remote sensing data processing, in particular to an unattended processing system for multi-source remote sensing data. Background With the rapid development of high resolution earth observation techniques, remote sensing data is increasing in speed and scale, which were not previously possible. The method and the system efficiently convert the massive satellite original data of multi-source isomerism into standard geographic information products which can be directly used for various industry applications, and become key infrastructure requirements for promoting the development of digital economy. Currently, the industry generally adopts a semi-automatic processing mode which is highly dependent on manual operation and multi-software switching. The mode requires technicians to manually import optical and radar data into independent systems respectively, and frequent manual configuration and intervention is performed in the core links of data preprocessing, engineering creation, quality control, process connection and the like. The traditional method has systematic defects that the automation level is low, large-scale continuous production is difficult to realize, intermediate data and parameters generated in the data processing process are disordered in management, the traceability is poor, the quality is difficult to guarantee, data islands are formed among different processing systems, manual bridging is relied on, the flow is loose and error-prone, the unified intelligent scheduling capability is further lacking, resources cannot be dynamically allocated, the abnormality can be handled, and the overall efficiency and the robustness are insufficient. Therefore, a brand new solution capable of thoroughly breaking the bottleneck of the prior art is urgently needed in the industry, so that the fundamental transition from manual driving to full-automatic intelligent of multi-source remote sensing data processing is realized, and the high-efficiency, reliable and manageable large-scale production requirements are met. Disclosure of Invention In order to overcome the defects in the prior art, the invention provides an unattended processing system for multi-source remote sensing data, which solves the problems in the background art. In a first aspect, an embodiment of the present application provides an unattended processing system for multi-source remote sensing data, including: The automatic production module, the intelligent management module, the cooperative control module and the three-level storage pool are connected through a data handling flow; the tertiary storage pool comprises an original storage pool, an intermediate storage pool and an achievement storage pool; The original storage pool is provided with a monitoring unit which is in communication connection with the message middleware; The intelligent management module comprises a metadata base and a dispatcher, and the dispatcher is in communication connection with the message middleware and the metadata base; The automatic production module comprises a workflow engine, a processing algorithm library and a distributed computing node cluster, wherein the workflow engine is in communication connection with the scheduler and the configuration center and is used for loading a processing flow template and instantiating an atomic task queue; the processing algorithm library is packaged with an optical image processing routine and a radar image processing routine; The cooperative control module comprises a service gateway, a configuration center and a state synchronization unit; the service gateway receives an external request, the configuration center stores the processing flow template, and the state synchronization unit is connected with the metadata base and the Web front end. In some embodiments of the application, the workflow engine is a ApacheAirflow engine; one of the distributed computing node cluster operations APACHESPARK and Dask frameworks; the message middleware is one of Redis and RabbitMQ; The processing algorithm library is integrated with GDAL library, orfeoToolbox library and SNAP tool kit. In some embodiments of the application, the process flow templates define dependencies and parallelism between data processing steps in a directed acyclic graph. In some embodiments of the application, the metadata database is a PostgreSQL database with PostGIS extensions deployed; The snoop unit monitors the specified directory of the original storage pool based on the Watchdog library. In some embodiments of the application, the scheduler is configured with a first preset threshold and a second preset threshold; and triggering computing resource capacity expansion operation by the scheduler when the average CPU utilization rate of the system is continuously higher than the first preset threshold or the length o