CN-122019085-A - Workflow-based surface live analysis self-adaptive scheduling method and device and application thereof

CN122019085ACN 122019085 ACN122019085 ACN 122019085ACN-122019085-A

Abstract

The invention discloses a workflow-based surface live analysis self-adaptive scheduling method, a workflow-based surface live analysis self-adaptive scheduling device and application, wherein the method comprises the following steps of P1, self-adaptive scheduling frame definition; the method comprises the steps of step P2, system self-adaptive identification design, step P3, system operation environment parameterization, step P4, system analysis time parameterization, step P5, analysis observation data definition, step P6, analysis background field definition, and step P7, analysis algorithm definition. The device comprises an adaptive scheduling framework definition module, a system adaptive identification design module, a system running environment parameterization module, an analysis time parameterization module, an analysis observation data definition module, an analysis background field definition module and an analysis algorithm definition module. The workflow-based surface live analysis self-adaptive scheduling method is used in service scenes such as surface live analysis product development and the like, and can effectively improve the automation level, the resource utilization efficiency and the capability of coping with complex scenes of a surface live analysis system.

Inventors

YAO YAN
ZHANG ZHIQIANG
LIANG XIAO
ZHANG ZHISEN
ZHANG TAO
SUN SHUAI
Yao shuang
ZHOU YIKE
SHI CHUNXIANG
GU JUNXIA

Assignees

国家气象信息中心(中国气象局气象数据中心)

Dates

Publication Date: 20260512
Application Date: 20260119

Claims (10)

1. The workflow-based surface live analysis adaptive scheduling method is characterized by comprising the following steps of: step P1, developing a content definition self-adaptive scheduling framework according to the surface live analysis product to form a tree-shaped hierarchical structure definition content file corresponding to the surface live analysis product; Step P2, planning and designing a system self-adaptive identification on the basis of the built self-adaptive scheduling framework, wherein the system self-adaptive identification comprises a surface live analysis system identification, a service scene identification, an application scheme identification and a serial number identification; Step P3, parameterizing the running environment of each system self-adaptive identification, and forming a corresponding running environment configuration file; step P4, parameterizing the analysis time of the surface live analysis system from three layers of an analysis time table, a basic analysis period and an analysis time window; Step P5, defining and classifying analysis observation data of the surface live analysis system; Step P6, parameterizing and defining the background field type and the background field selection strategy of the surface live analysis system; and P7, carrying out parameterization setting on an analysis algorithm of the surface live analysis system to realize modularization and strategic of the analysis algorithm.
2. The workflow-based surface live analysis adaptive scheduling method of claim 1, wherein the definition of the adaptive scheduling framework in step P1 includes the following: (P1-1) defining corresponding job package names and frame contents according to the surface live analysis product development contents; (P1-2) defining job family names and frame contents facing different scenes in a job package node frame, wherein the job family names and frame contents comprise a fast running scene, a standard running scene, a product complement scene, a historical product back calculation scene and different sensitivity test evaluation scenes; (P1-3), in the job family node frame corresponding to different scenes, planning the flow into observation data preprocessing, background field processing, fusion analysis and product post-processing, and defining corresponding sub-job family contents or task elements according to processing logic; (P1-4) adopting modularized structural design, and completing definition of task function module names and execution function frames by extracting function modules related to each scene in the surface live fusion analysis; (P1-5), classifying the logically combinable task function modules, and using the sub-job families to define and represent frames.
3. The workflow-based surface live analysis adaptive scheduling method according to claim 1, wherein in the step P2, the surface live analysis system identifier is used for distinguishing different live analysis systems, the service scene identifier is used for distinguishing a fast running scene, a standard running scene, a product complement scene, a historical product back calculation scene and different sensitive test evaluation scenes, the application scheme identifier is used for distinguishing different surface live analysis technical scheme contents adopted, the serial number identifier uses two digits and is used for identifying different operation scheduling branch contents when the surface live analysis system identifier, the service scene identifier and the application scheme identifier are used for carrying out flow splitting and parallel processing under the same scene, and the surface live analysis system identifier, the service scene identifier, the application scheme identifier and the serial number identifier are separated by an underlined symbol "_".
4. The workflow based surface live analysis adaptive scheduling method of claim 1, wherein in step P3, the runtime environment configuration file adaptively identifies definitions and declarations of runtime environment parameterized content for the system, including a system script runtime master catalog, an observation data input and quality control processing catalog, a background field input and processing catalog, a fusion analysis product generation catalog, and a product post-processing catalog.
5. The workflow-based surface live analysis adaptive scheduling method according to claim 1, wherein in the step P4, the analysis schedule self-defining method is to set an analysis schedule by using Crontab-like expression content definition to realize automatic processing of a product analysis period; Setting a basic analysis time interval, and determining specific time of surface live analysis according to product aging, observation data, background field data and analysis product generation conditions; And dynamically adjusting the analysis time window, namely carrying out parameterization configuration on the starting time and the ending time of the analysis time window, and carrying out dynamic adjustment on the analysis time window according to the actual requirements of the business scene.
6. The workflow-based surface live analysis adaptive scheduling method of claim 1, wherein in step P5, analysis observation data definition and classification is performed from the following dimensions: the data source type is distinguished by designing different identifications for the observation data of different data sources and parameterized definition is carried out, and the priority policy is that a priority rule selected by the observation data is formulated; The method comprises the steps of defining a minimum observed quantity threshold value, and if the observed quantity of the data sources is lower than the minimum observed quantity threshold value, determining the flow scheduling by (1) not waiting for fusion analysis by using the existing data, (2) starting a complementary process, and (3) triggering one-time analysis update after the observed quantity of the analyzed observed data exceeds the minimum observed quantity threshold value.
7. The workflow-based surface live analysis adaptive scheduling method of claim 1, wherein in step P6, parameterizing the definition content comprises: the background field types are distinguished by planning different identifications for background fields with different sources or different resolutions, and parameterized definition is carried out; The background field selection strategy comprises (1) automatically degrading the system to replace the low-resolution background field when the high-resolution background field is lost based on availability, (2) preferentially selecting the available background field based on timeliness of the background field, and triggering fusion analysis scheduling based on the high-quality background field once again after the background field with higher quality is ready; and a background field preprocessing strategy, namely selecting a preprocessing algorithm as a configurable parameter according to the requirements of fusion analysis products.
8. The workflow based surface live analysis adaptive scheduling method of claim 1, wherein the algorithm definition in step P7 comprises the following: the algorithm type is to design different identifications for the analysis algorithms of different technical schemes to distinguish and parameterize and define; The system scheduling engine selects an analysis algorithm defined by configuration for the corresponding workflow instance according to the self-adaptive identification content of the system, and configures according to the business scene target or the data condition of fusion analysis; The adaptive configuration of algorithm parameters, namely the fusion analysis algorithm itself has adjustable parameters including influencing radius and weight functions.
9. The surface live analysis self-adaptive scheduling device based on the workflow is characterized by comprising a self-adaptive scheduling framework definition module, a system self-adaptive identification design module, a system operation environment parameterization module, an analysis time parameterization module, an analysis observation data definition module, an analysis background field definition module and an analysis algorithm definition module; the adaptive scheduling framework definition module is used for realizing the step P1 in the workflow-based surface live analysis adaptive scheduling method according to claim 1; the system adaptive identification design module is used for realizing the step P2 in the workflow-based surface live analysis adaptive scheduling method according to claim 1; The system running environment parameterization module is used for realizing the step P3 in the workflow-based surface live analysis adaptive scheduling method according to claim 1; the analysis time-time parameterization module is used for realizing the step P4 in the workflow-based surface live analysis adaptive scheduling method according to claim 1; The analysis observation data definition module is used for realizing the step P5 in the workflow-based surface live analysis adaptive scheduling method according to claim 1; an analysis background field definition module for implementing step P6 in the workflow-based surface live analysis adaptive scheduling method of claim 1; The analysis algorithm definition module is configured to implement step P7 in the workflow-based surface live analysis adaptive scheduling method according to claim 1.
10. An application of a workflow-based surface live analysis adaptive scheduling method, which is characterized in that the workflow-based surface live analysis adaptive scheduling method according to any one of claims 1 to 8 is used in service scenes of surface live analysis product development, individual case assessment, service system construction and historical product back calculation.

Description

Workflow-based surface live analysis self-adaptive scheduling method and device and application thereof Technical Field The invention relates to the technical field of surface live information processing and system construction. Specifically, a workflow-based surface live analysis adaptive scheduling method, a workflow-based surface live analysis adaptive scheduling device and application of the workflow-based surface live analysis adaptive scheduling device. Background The surface live analysis is to construct a zero moment of seamless lattice point forecast prediction service by integrating various sources and various types of data resources and utilizing a data fusion and assimilation technology. Most of the surface live analysis systems mainly comprise an organic whole by a plurality of functions such as data acquisition and preprocessing, background field processing, data fusion analysis, product post-processing and the like, and relate to numerical calculation and flow control, and at present, the real-time business adopts a workflow mode to perform unified flow scheduling and monitoring management. In the process of product research and development, quality evaluation, business operation and product back calculation, there are product production and comparison analysis requirements of different research schemes or methods, the production requirements are integrated and analyzed by the same category of different source data, different quality product production requirements are started at different time points, various changes such as product back calculation requirements in a historical time period, product fault tolerance back calculation processing requirements in a real-time operation process and the like are caused, repeated development of product codes and system redundancy construction are caused, and a certain challenge is brought to product production scheduling. Therefore, the development of the workflow-based adaptive scheduling method is very critical to ensuring the development and real-time production of the surface live analysis product. Disclosure of Invention Therefore, the technical problem to be solved by the invention is to provide a workflow-based surface live analysis self-adaptive scheduling method, a workflow-based surface live analysis self-adaptive scheduling device and application thereof, so as to solve the problems of code redundancy and system repeated construction of the traditional static scheduling. In order to solve the technical problems, the invention provides the following technical scheme: A workflow-based surface live analysis adaptive scheduling method comprises the following steps: step P1, developing a content definition self-adaptive scheduling framework according to the surface live analysis product to form a tree-shaped hierarchical structure definition content file corresponding to the surface live analysis product; Step P2, planning and designing a system self-adaptive identification on the basis of the built self-adaptive scheduling framework, wherein the system self-adaptive identification comprises a surface live analysis system identification, a service scene identification, an application scheme identification and a serial number identification; Step P3, parameterizing the running environment of each system self-adaptive identification, and forming a corresponding running environment configuration file; step P4, parameterizing the analysis time of the surface live analysis system from three layers of an analysis time table, a basic analysis period and an analysis time window; Step P5, defining and classifying analysis observation data of the surface live analysis system; Step P6, parameterizing and defining the background field type and the background field selection strategy of the surface live analysis system; and P7, carrying out parameterization setting on an analysis algorithm of the surface live analysis system to realize modularization and strategic of the analysis algorithm. The definition of the adaptive scheduling framework in the step P1 includes the following contents: (P1-1) defining corresponding suite names and frame contents according to the surface live analysis product development contents; (P1-2), defining family names and frame contents facing different scenes in a suite node frame, wherein the family names and frame contents generally comprise a fast running scene, a standard running scene, a product complement scene, a historical product back calculation scene, different sensitivity test evaluation scenes and the like; (P1-3), planning the flow into the processes of observation data preprocessing, background field processing, fusion analysis, product post-processing and the like in a family node frame which is used for coping with different scenes, and defining corresponding sub family content or task elements according to processing logic; (P1-4) adopting modularized structural design, and completing the name of a task module a