CN-122021607-A - Meteorological NC structured data extraction and analysis method and device

CN122021607ACN 122021607 ACN122021607 ACN 122021607ACN-122021607-A

Abstract

The application discloses a method and a device for extracting and analyzing weather NC structured data, the method comprises the steps of constructing a feature description file, generating a structured file containing weather semantic configuration items, space-time screening conditions and multi-source data compatible configuration, reading the feature description file by an analysis engine, converting business terms into technical parameters according to a built-in weather semantic dictionary, identifying application scenes, automatically associating coordinate system information of weather data according to the space-time screening conditions in the feature description file, calculating a slice index of target data, calling a bottom layer analysis library to load the target NC file, combining the calculated slice index to locate target variables, executing data type verification, unit conversion and dimension alignment operation, and outputting an extraction result according to a specified format. According to the application, the extraction rule is separated from the analysis logic, and the weather data characteristics and the business scene self-adaptive expansion are combined, so that the problem of weak response capability of sudden demand is solved.

Inventors

He Xingnan
ZHENG RONGGUI
CHEN HAIJUN

Assignees

航天天目(重庆)卫星科技有限公司

Dates

Publication Date: 20260512
Application Date: 20251225

Claims (9)

1. The method for extracting and analyzing the weather NC structured data is characterized by comprising the following steps of: A) Constructing a characteristic description file, wherein the characteristic description file is a structured file containing weather semantic configuration items, space-time screening conditions and multi-source data compatible configuration; B) Semanteme analysis, namely reading the characteristic description file by an analysis engine, converting business terms into technical parameters according to a built-in meteorological semantic dictionary, and identifying an application scene; C) Calculating a space-time index, namely automatically associating coordinate system information of meteorological data according to space-time screening conditions in the feature description file, and calculating a slice index of target data; D) The data extraction and conversion are that a bottom layer analysis library is called to load a target NC file, a calculated slice index is combined to locate a target variable, and data type verification, unit conversion and dimension alignment operations are executed; E) Outputting the result, namely outputting the extraction result according to the appointed format.
2. The method of claim 1, wherein the weather semantic configuration item in step a) comprises a weather element tag field for associating business meaning tags of the weather element with underlying variable names, and an application scenario tag field for identifying an application scenario, wherein the application scenario comprises extreme weather or regular forecasts; The multi-source data compatible configuration comprises a data source tag field and a multi-source adaptation rule field, wherein the analysis engine calls a corresponding data type mapping rule according to the data source tag field and performs dimension alignment according to the multi-source adaptation rule field and an associated preset adaptation rule.
3. The method according to claim 1, wherein the space-time filtering condition in the step C) includes an administrative region matching rule, and the parsing engine automatically retrieves a preset longitude and latitude boundary library according to the administrative region matching rule to generate the grid index.
4. The method according to claim 1, wherein the data type verification and conversion process in step D) comprises the steps of: Calling a corresponding data type mapping rule according to the multi-source data compatible configuration; performing dimension alignment operation on heterogeneous data sources; the unit conversion operation is performed to unify the physical quantity units to the standard units.
5. The method according to claim 1, wherein when the step E) outputs the result, if the application scenario is detected as "extreme weather", a data integrity check identifier is attached to the header of the output file.
6. The device for extracting and analyzing the weather NC structured data is characterized by comprising the following components: The system comprises a characteristic description file generation module, a data processing module and a data processing module, wherein the characteristic description file generation module is configured to receive weather service requirement parameters input by a user, the parameters comprise weather semantic configuration items, space-time filtering conditions and multi-source data compatible configuration, and generate a structural characteristic description file comprising the following fields: The weather element label field is used for associating business meaning labels of the weather elements with bottom variable names; An application scenario tag field for identifying an application scenario, wherein the application scenario comprises extreme weather or a regular forecast; the data source tag is used for identifying a meteorological data source; A multi-source adaptation rule field for specifying an adaptation rule identifier at the time of multi-source data fusion; The time screening conditions comprise preset semantical time periods (such as last_24 h and specific_time_range) or forecast aging (forecast_lead_time); The space screening condition comprises an administrative region name field for indicating that the target region range is automatically matched according to a preset longitude and latitude boundary library; the analysis engine processing module is in communication connection with the characteristic description file generating module and comprises: The semantic analysis sub-module is used for reading the characteristic description file, analyzing an application scene tag field to determine the current processing priority, and converting the meteorological element tag into a variable extraction instruction of a technical layer; The space-time index calculation sub-module is used for calling a preset geographic boundary database according to the administrative region names in the space screening conditions to calculate corresponding space grid indexes; and the data extraction and conversion sub-module is used for calling the bottom NetCDF library to load the target file, extracting the original data according to the space-time index, executing data type verification, unit conversion and dimension alignment operation according to the data source label and the multi-source adaptation rule, and finally outputting a structured data result conforming to the specified format.
7. The apparatus of claim 6, wherein the profile generation module further comprises a very simple configuration interface that enables a user to select weather elements and time ranges via a checkbox, thereby triggering the generation module to automatically generate a default format profile.
8. The apparatus of claim 6, wherein the parsing engine processing module further comprises a data integrity checking unit that automatically performs data integrity checking and appends a check identifier to the header of the output file when the application scene tag is detected as "extreme weather".
9. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any one of claims 1 to 5.

Description

Meteorological NC structured data extraction and analysis method and device Technical Field The invention belongs to the technical field of meteorological big data processing and scientific computing, and particularly relates to a meteorological NC structured data extraction and analysis method based on a characteristic description file. Background In the existing meteorological field, NC (NetCDF) and other structured data files are widely used for storing meteorological elements (such as temperature, humidity, air pressure and the like), and the data structure of the data files comprises multidimensional variables, attribute information and a coordinate system, and has the characteristics of strong specialization, fixed format and complex variable level. The prior art has the following problems in extracting and analyzing NC files: 1. Hard coding relies heavily, the traditional analysis mode needs to directly write codes to specify variable names, dimension indexes and the like through programming languages (such as netCDF library of Python), and if the variables or data types to be extracted are changed, source codes need to be modified and recompiled, so that the flexibility is poor, and the method is particularly unfriendly to non-developers (such as weather service personnel). 2. The reusability is low, and aiming at different NC files (such as weather data in different areas and at different times) or different extraction requirements (such as newly added variables and modified data types), analytical logic needs to be repeatedly developed, so that code redundancy is caused, and the maintenance cost is high. 3. The data type matching is error-prone, namely the data types (such as float32 and int 64) of variables in the NC file are manually declared in codes, if the data types are not matched with the types in the actual file, analysis failure or data distortion is easy to cause, and particularly, the problem is prominent in a multi-source meteorological data fusion scene. 4. And the sudden demand response is lagged, namely, in sudden weather scenes such as extreme weather, weather service personnel (non-programmers) cannot quickly adjust the data extraction rule, and a developer needs to wait for intervention to modify codes, so that the data support is not timely. In the prior art, the extraction rule (variables, data types and the like to be extracted) is strongly coupled with the analysis logic (code implementation), the separation of configuration and logic is not realized, and the special scheme is designed without combining the meteorological data characteristics and the business scene, so that the suitability, the expansibility and the sudden demand response capability are poor. Disclosure of Invention The following presents a simplified summary of embodiments of the invention in order to provide a basic understanding of some aspects of the invention. It should be understood that the following summary is not an exhaustive overview of the invention. It is not intended to identify key or critical elements of the invention or to delineate the scope of the invention. Its purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is discussed later. According to one aspect of the present application, there is provided a method for extracting and parsing weather NC structured data, comprising the steps of: A) Constructing a characteristic description file, wherein the characteristic description file is a structured file containing weather semantic configuration items, space-time screening conditions and multi-source data compatible configuration; B) Semanteme analysis, namely reading the characteristic description file by an analysis engine, converting business terms into technical parameters according to a built-in meteorological semantic dictionary, and identifying an application scene; C) Calculating a space-time index, namely automatically associating coordinate system information of meteorological data according to space-time screening conditions in the feature description file, and calculating a slice index of target data; d) The data extraction and conversion, namely calling a bottom layer analysis library to connect with a target NC file, combining the calculated slice index to locate a target variable, and executing data type verification, unit conversion and dimension operation on the target variable; E) Outputting the result, namely outputting the extraction result according to the appointed format. Further, the weather semantical configuration item in the step a) includes a meteorological _element field for associating a variable name with a business meaning tag, and a data_scenario field for identifying an application scenario, where the application scenario includes extreme weather or a regular forecast. Further, the multi-source data compatible configuration in the step a) includes a data_source (data source tag) field and a compatibilit