CN-121997238-A - Nevzorov water content probe data quality control processing method and system based on airborne in-situ observation

Abstract

The application relates to the technical field of meteorological data analysis, and provides a Nevzorov water content probe data quality control processing method and system based on airborne in-situ observation, wherein the method comprises the steps of acquiring airborne in-situ observation data and classifying based on acquired overhead association information to obtain an initial data set; the method comprises the steps of generating an observation data set through combination of an overhead time threshold, obtaining the quality control data set through quality control processing of Nevzorov probe data, extracting environment and position characteristic parameters, executing cloud entering multi-condition joint analysis and screening effective observation data, constructing a base line correction standard condition, completing water content data deviation correction, analyzing a graded particle spectrum number based on corrected data to obtain particle spectrum distribution information, constructing an inversion model according to association relation between the particle spectrum and liquid water content, completing liquid water content inversion optimization through characteristic matching dynamic adjustment parameters, generating inversion optimization results, improving reliability of the airborne probe observation data and liquid water content inversion precision, and providing accurate data support for cloud micro physical research.

Inventors

• LI JUNXIA
• Zuo Dongfei
• ZHANG XIAOTUO
• HE CHUAN
• ZHANG RONG
• GAO YANG

Assignees

• 中国气象局人工影响天气中心

Dates

Publication Date

20260508

Application Date

20260130

Claims (10)

1. 1. A Nevzorov water content probe data quality control processing method based on airborne in-situ observation is characterized by comprising the following steps: Acquiring airborne in-situ observation data, and performing data classification processing based on acquisition frame-time associated information of the airborne in-situ observation data to obtain an initial data set; Combining the initial data sets based on the threshold condition of the time of the frame to generate an observation data set; performing quality control processing on the observation data set according to a Nevzorov probe data quality control task, identifying an abnormal data subset in an abnormal working state from the observation data set, and deleting the abnormal data subset to obtain a quality control data set; Extracting environmental characteristic parameters and position characteristic parameters in the quality control data set, executing cloud entering multi-condition joint analysis, and screening acquired data in a cloud entering period through multi-condition collaborative matching to serve as effective observation data; Responding to a water content baseline correction task, extracting baseline reference characteristics in the effective observation data to construct correction reference conditions, carrying out deviation correction on the water content data in the effective observation data through the correction reference conditions to obtain corrected observation data, carrying out grading particle spectrum number analysis according to particle spectrum characteristic differences by utilizing the corrected observation data to obtain a gear division result, and extracting particle spectrum distribution information based on the gear division result; And constructing an inversion model through the association relation between the particle spectrum distribution and the liquid water content, importing the particle spectrum distribution information into the inversion model to execute liquid water content inversion optimization, adjusting inversion model parameters through characteristic matching results in the inversion optimization process, and generating an inversion optimization result by combining inversion operation results and parameter adjustment records.
2. 2. The method of claim 1, wherein performing quality control processing on the observation dataset according to Nevzorov probe data quality control tasks, identifying and deleting an abnormal data subset in an abnormal working state from the observation dataset, and obtaining a quality control dataset comprises: Determining quality control dimensions and analysis priorities of the dimensions based on the Nevzorov probe data quality control task, extracting probe working signal data from the observation dataset, performing signal stability screening, constructing an adaptive threshold judgment condition of signal fluctuation through time sequence mode analysis, and marking signal data exceeding the adaptive threshold judgment condition as potential abnormal signals; Noise identification is carried out on the potential abnormal signals, noise signal components are separated through comparison of signal frequency characteristics and a standard working frequency range, meanwhile drift detection is carried out on water content detection data in the observation data set, the change trend of the detection data in different acquisition periods is tracked, and trend fitting deviation is calculated; Performing environment matching state analysis by combining with the flight environment data corresponding to the observation data set, and incorporating the data with the matching degree of the probe working parameters and the contemporaneous environment parameters lower than the set matching value into the data to be inspected; The data to be checked is subjected to physical comparison by referring to a physical working threshold of the Nevzorov probe, whether the data to be checked accords with an actual working rule is judged, and a data set corresponding to a mutation period is marked by synchronously executing a jump detection and identification data mutation point; Carrying out integrity scanning on the observation data set to execute missing data marking, counting the missing data duty ratio of each acquisition period and recording missing positions; And combining the information obtained by signal stability screening, noise identification, drift detection, environment matching state analysis, physical rationality comparison, kick detection and missing data marking to generate abnormal state classification conditions, classifying the data set conforming to the abnormal state classification conditions into an abnormal data subset and removing the abnormal data subset from the observed data set to obtain a quality control data set subjected to multi-dimensional quality control processing.
3. 3. The method of claim 1, wherein the extracting the environmental characteristic parameter and the position characteristic parameter in the quality control data set and performing the cloud entry multi-condition joint analysis, and screening the collected data in the cloud entry period as the effective observation data through multi-condition collaborative matching comprises: Extracting temperature parameters, humidity parameters, air pressure parameters and atmospheric particulate concentration parameters from the quality control data set to serve as environment characteristic parameters, and extracting flight longitude parameters, flight latitude parameters and flight altitude parameters to serve as position characteristic parameters; Performing temperature and humidity gradient analysis on the temperature parameter and the humidity parameter in the environmental characteristic parameters, calculating the temperature and humidity change rates of different layers and generating a temperature and humidity gradient curve; acquiring an optical scattering trend based on the optical detection data in the quality control data set, and judging the uniformity of the atmosphere medium through the time sequence change characteristics of the optical scattering intensity; Performing flying height layer association on flying height parameters in the position characteristic parameters and preset height layer division standards, and determining height layer categories corresponding to all acquisition moments; Performing geographic coordinate meshing on the flight longitude parameter and the flight latitude parameter, dividing an acquisition area into grid units with preset sizes, and marking grid positions corresponding to each acquired data; creating a logic combination rule for cloud judgment based on temperature and humidity gradient analysis, optical scattering trend analysis, flying height layer association and geographic coordinate gridding to obtain results, setting weight coefficients corresponding to the analysis results, constructing a dynamic criterion adjustment strategy, and adjusting a criterion threshold according to the climate characteristic differences of different flying areas; Integrating the cloud boundary recognition model with the logic combination rule, and verifying the judgment result of the logic combination rule through cloud boundary position information output by the cloud boundary recognition model; confidence screening is executed based on the integrated judgment model, and collected data meeting the temperature and humidity gradient condition, the optical scattering trend condition, the flying height layer condition and the geographic coordinate condition simultaneously are marked as candidate cloud entering data; And optimizing the candidate cloud entering data based on space-time continuity, deleting discrete data points with discontinuous space-time distribution, determining acquisition data corresponding to a cloud entering period and taking the acquisition data as effective observation data.
4. 4. A method as claimed in any one of claims 1 to 3, wherein said extracting baseline reference features in said effective observed data in response to a water content baseline correction task to construct corrected reference conditions, and performing bias correction on water content data in said effective observed data by said corrected reference conditions to obtain corrected observed data comprises: Responding to a water content baseline correction task, dividing the effective observation data in time periods, identifying the time periods of which the cloud body is not entered and the atmospheric environment is stable in the acquisition process, and taking the corresponding observation data as clean air section data to execute clean air section extraction; Carrying out statistical analysis on the water content data in the clean air section data, calculating an average value of the water content data in the clean air section as a water content average value reference, and simultaneously calculating a standard deviation of the water content data as a fluctuation range reference; constructing an initial correction reference condition based on the water content mean value reference and the fluctuation range reference, comparing the water content data in the effective observation data with the initial correction reference condition, and determining deviation data exceeding the fluctuation range reference; Grouping the deviation data according to the acquisition height and the acquisition time period, performing piecewise linear correction on the data of different groups, obtaining correction coefficients of each group through linear fitting, and performing primary correction on the deviation data; Carrying out nonlinear deviation analysis on the primarily corrected water content data, identifying nonlinear deviation characteristics, executing nonlinear deviation adjustment, constructing a nonlinear deviation correction function, and substituting the nonlinear deviation correction function into the data to complete secondary correction; the temperature parameter, the humidity parameter and the air pressure parameter in the environmental characteristic parameters are taken into an environmental parameter self-adaptive model, an association mapping relation between the environmental parameters and the correction coefficients is established, and the dynamic update of the correction coefficients is executed according to the real-time environmental parameter change; carrying out residual analysis on the water content data subjected to linear correction and nonlinear correction, calculating residual errors of the corrected data and the clean air section data average value, and evaluating residual error distribution characteristics; And performing correction effect evaluation based on the residual analysis result, triggering a reference condition reconstruction flow if the residual ratio exceeds a preset allowable range, re-extracting clean air section data and updating a water content mean value reference and a fluctuation range reference, and repeating a correction flow until the residual distribution meets the preset requirement to obtain corrected observation data subjected to deviation correction.
5. 5. The method of claim 4, wherein the analyzing the number of the classified particles according to the characteristic differences of the particle spectrum by using the corrected observation data to obtain a gear classification result, and extracting the particle spectrum distribution information based on the gear classification result comprises: Acquiring particle detection data in the corrected observation data, and extracting particle size parameters, particle number concentration parameters and particle speed parameters from the particle detection data as key characteristic parameters of a particle spectrum; Performing particle size distribution analysis on the particle size parameters, counting the particle number proportion of different particle size intervals, generating a particle size distribution histogram, eliminating data fluctuation interference through histogram smoothing, performing time sequence analysis on the particle number concentration parameters, identifying concentration change characteristics, marking concentration peak time periods and concentration valley time periods, calculating concentration change rates of different time periods, establishing a concentration change trend model, determining key difference points of particle spectrum characteristics based on the results obtained by the particle size distribution analysis and the concentration change characteristic analysis, and identifying characteristic demarcation points of different particle groups through a key difference point detection algorithm; Clustering key characteristic parameters of the particle spectrum by introducing a clustering algorithm, and gathering particle data with similar particle diameters and consistent concentration change trend into the same category to obtain an initial clustering result; Constructing a regularized gear setting template based on a preset particle spectrum gear dividing rule, comparing an initial clustering result obtained by clustering division with the regularized gear setting template, adjusting a clustering class boundary to accord with a gear dividing label, extracting spectral width parameters from particle spectrum data of corrected observation data, calculating particle size distribution widths of different particle classes, recording spectral width extremum values, executing peak information capturing on particle size distribution histograms through a peak detection algorithm, determining particle size peak values and concentration peak values of each particle class, marking corresponding characteristic information, generating a gear association logic integration model based on the regularized gear setting result, the spectral width parameter extraction and the peak information capturing, and determining a particle size engagement relationship, a concentration association relationship and a spectral width progressive relationship among all gears based on the gear association logic integration model; Executing structural processing of gear data by combining the particle size engagement relationship, the concentration association relationship and the spectrum width progressive relationship to generate a gear dividing result; And backtracking verification is carried out on a gear dividing result through historical particle spectrum data to compare the dividing result with an adaptation coefficient of a historical standard gear, cross verification is carried out on corrected observation data passing through different flight frames to adjust a gear dividing boundary to adapt to different observation scenes, theoretical verification is carried out on the dividing result through a particle spectrum theoretical model to enable the dividing result to conform to a physical rule formed by a particle spectrum, and particle spectrum distribution information is obtained by combining characteristic dividing points and the gear dividing result after multistage division verification, extracting particle size distribution characteristics, concentration distribution characteristics, spectrum width characteristics and peak characteristics of each gear and integrating according to a gear sequence.
6. 6. The method of claim 1, wherein the constructing an inversion model by the association relation between the particle spectrum distribution and the liquid water content, importing the particle spectrum distribution information into the inversion model to perform liquid water content inversion optimization, adjusting inversion model parameters by the feature matching result in the inversion optimization process, and generating an inversion optimization result by combining the inversion operation result and the parameter adjustment record comprises: Determining a mathematical mapping relation between particle size, particle concentration and liquid water content based on particle physical characteristic information, establishing a correlation relation model of particle spectrum distribution and liquid water content, and establishing an initial inversion model through the correlation relation model; Importing particle size distribution data, concentration distribution data, spectrum width data and peak value data in the particle spectrum distribution information into the initial inversion model, starting iterative inversion operation, and calculating an initial inversion liquid water content result; Residual distribution matching is carried out on the initial inversion liquid water content result and auxiliary observation data in the corrected observation data, residual errors between the initial inversion liquid water content result and the auxiliary observation data are calculated, a residual error distribution curve is generated, the distribution form and the concentration trend of the residual errors are determined through the residual error distribution curve, and a residual error distribution matching result is obtained; Performing correlation index evaluation based on the residual distribution matching result, calculating correlation coefficients of the inversion result and each characteristic parameter in the particle spectrum distribution information, and determining key characteristic parameters affecting inversion accuracy to obtain a correlation index evaluation result; Generating a model parameter dynamic adjustment strategy according to the correlation index evaluation result, taking the residual error size as a feedback signal of parameter adjustment, and adjusting weight coefficients and mathematical model coefficients corresponding to each characteristic parameter in an initial inversion model; Setting a convergence condition judgment standard of inversion operation, monitoring the variation trend of residual errors in the iterative inversion operation process, and judging that the inversion operation reaches a convergence state when the variation of the residual errors is smaller than a preset convergence threshold value and the continuous iteration times reach a preset value; If the convergence state is not reached, an error feedback loop is triggered, a current residual error analysis result is fed back to a model parameter adjustment link, the model parameters are readjusted, and iterative inversion operation is continuously executed; In the inversion optimization process, recording an adjustment value, adjustment time, adjustment basis and a corresponding inversion result of each model parameter adjustment, and generating an optimization record; After the inversion operation reaches a convergence state, performing feature matching optimization on the inversion result before outputting so that the feature matching degree of the inversion result and the particle spectrum distribution information meets the preset requirement; and combining the final inversion operation result with the optimization record, and finishing to generate an inversion optimization result comprising an inversion numerical value, a parameter adjustment track, a residual analysis result and a correlation index.
7. 7. The method of claim 6, wherein performing residual distribution matching on the initially inverted liquid water content result and auxiliary observation data in the corrected observation data, calculating a residual between the initially inverted liquid water content result and the auxiliary observation data, and generating a residual distribution curve, and determining a distribution form and a concentration trend of the residual through the residual distribution curve, to obtain a residual distribution matching result, includes: Extracting auxiliary observation data related to liquid water content inversion from the corrected observation data, performing time sequence alignment processing, preprocessing the auxiliary observation data subjected to time sequence alignment, and performing layering marking on the preprocessed auxiliary observation data based on flight height parameters in the quality control data set to divide auxiliary observation data subsets corresponding to different flight height layers; Performing point-by-point difference operation on the initial inversion liquid water content result and auxiliary observation data corresponding to the time sequence based on a preset residual error calculation model to obtain residual values at all acquisition moments, wherein the calculation mode of the residual values is that the initial inversion liquid water content result value subtracts a standardized value of the corresponding auxiliary observation data; Sequencing the calculated residual values according to the acquisition time sequence, constructing a residual sequence, generating a residual distribution curve based on the residual sequence by adopting a polynomial fitting algorithm, and optimizing the smoothness of the curve by adjusting fitting orders in the curve generation process; extracting features of the residual distribution curve, identifying peak points, valley points and inflection points in the curve, counting the residual value and acquisition time corresponding to each feature point, and analyzing the feature difference of the residual distribution curve under different height layers by combining the flight height layer information of the layering mark; Carrying out distribution form judgment on the residual values by adopting a statistical analysis method, calculating a skewness coefficient and a kurtosis coefficient of the residual values, judging the symmetry degree of residual distribution by the skewness coefficient, judging the steepness degree of the residual distribution by the kurtosis coefficient, and determining whether the residual distribution accords with normal distribution characteristics; calculating an arithmetic average value, a median and a mode of the residual values, reflecting the central position of the residual error by the arithmetic average value, eliminating the influence of the extreme residual values on the judgment of the centralized trend by the median, and representing the residual value with the highest occurrence frequency by the mode; and combining residual distribution curve characteristics, distribution form judgment results and central trend statistical data, and combining residual difference information of different flying height layers to generate a residual distribution matching result comprising a residual time sequence change rule, a distribution type and a central position parameter.
8. 8. The method of claim 6, wherein generating a model parameter dynamic adjustment strategy according to the correlation index evaluation result, using the residual size as a feedback signal for parameter adjustment, and adjusting weight coefficients and mathematical model coefficients corresponding to each feature parameter in the initial inversion model comprises: Carrying out quantization analysis on the correlation index evaluation result, extracting correlation coefficient values between each characteristic parameter and an inversion result, sequencing the characteristic parameters according to the sequence of the absolute value of the correlation coefficient from large to small, dividing three levels of a core influence characteristic parameter, a secondary influence characteristic parameter and a weak influence characteristic parameter, wherein the core influence characteristic parameter is a characteristic parameter of which the absolute value of the correlation coefficient is larger than a preset core threshold, the secondary influence characteristic parameter is a characteristic parameter of which the absolute value of the correlation coefficient is between a preset secondary threshold and the core threshold, and the weak influence characteristic parameter is a characteristic parameter of which the absolute value of the correlation coefficient is smaller than the preset secondary threshold; Constructing a parameter adjustment priority rule based on a hierarchical division result, setting a weight coefficient adjustment priority corresponding to a core influence characteristic parameter higher than a secondary influence characteristic parameter, setting the adjustment priority of the secondary influence characteristic parameter higher than a weak influence characteristic parameter, simultaneously establishing a linear association mapping between the residual size and the parameter adjustment amplitude, constructing a residual feedback adjustment function, wherein the input of the residual feedback adjustment function is an absolute value of a residual value, outputting the absolute value as a corresponding parameter adjustment amount, and outputting the larger parameter adjustment amount when the residual value is larger; For weight coefficients in an initial inversion model, carrying out iterative adjustment on the weight coefficients corresponding to the core influence characteristic parameters by adopting a gradient descent algorithm, taking residual error minimization as an objective function, determining the adjustment direction and the adjustment amplitude of the weight coefficients based on a residual error feedback adjustment function in each iterative process, and synchronously recording the adjustment track of the weight coefficients; A local fine tuning strategy is adopted for the weight coefficient corresponding to the secondary influence characteristic parameter, and the fine tuning amplitude is adjusted by combining the observation environment difference corresponding to the flight frame number, so that the weight coefficient is adapted to the characteristic response requirements under different observation scenes; For the mathematical model coefficient, determining an adjustment boundary of the mathematical model coefficient based on the association theory of particle spectrum distribution and liquid water content and combining the distribution form information in the residual distribution matching result; Constructing parameter adjustment constraint conditions, taking the working parameter range and the atmospheric physical characteristic parameter range of the Nevzorov probe as constraint basis, checking the weight coefficient and the mathematical model coefficient in the adjustment process in real time, triggering a backtracking mechanism if the parameters are out of the constraint range after adjustment, recovering to the parameter values before adjustment, and redefining the adjustment scheme; taking part of data in the quality control data set as a cross verification data set, importing the verification data set into an initial inversion model for inversion operation after each parameter adjustment, and calculating error values of a cross verification inversion result and actual observation data in the verification data set to obtain a cross verification result; And generating a model parameter dynamic adjustment strategy based on the parameter adjustment priority rule, the residual error feedback adjustment function, the parameter adjustment track record, the parameter adjustment constraint condition and the cross verification result, wherein the dynamic adjustment strategy comprises adjustment rules of characteristic parameters of each level, correlation standards of residual error and adjustment amplitude, a parameter adjustment constraint range and an adjustment effect verification flow.
9. 9. A Nevzorov moisture content probe data quality control processing system, characterized in that it comprises a processor and a memory, wherein the memory stores a computer program, which when executed by the processor, causes the processor to execute the steps of the Nevzorov moisture content probe data quality control processing method based on-board in-situ observation according to any one of claims 1 to 8.
10. 10. A computer readable storage medium comprising a computer program for causing a Nevzorov water content probe data quality control processing system to perform the steps of the Nevzorov water content probe data quality control processing method based on-board in-situ observations as claimed in any one of claims 1 to 8 when the computer program is run on a Nevzorov water content probe data quality control processing system.

Description

Nevzorov water content probe data quality control processing method and system based on airborne in-situ observation Technical Field The application belongs to the technical field of meteorological data analysis, and particularly relates to a quality control processing method and system for Nevzorov water content probe data based on airborne in-situ observation. Background The airborne in-situ observation technology is one of core means of cloud micro-physics research, wherein Nevzorov water content probes are used as key equipment for directly collecting the atmospheric liquid water content and the total water content, and the reliability of observation data directly determines the scientificity of cloud micro-physics process analysis. In the prior art, airborne in-situ observation data processing generally covers links such as data acquisition, preliminary screening, cloud entry judgment, data correction and inversion calculation, and specifically, the data acquisition stage synchronously acquires Nevzorov probe data, cloud particle probe data, meteorological environment data, navigation positioning data and other multi-source data through airborne equipment, the preliminary screening stage generally carries out simple data classification based on an acquisition period or equipment identification, the cloud entry judgment link carries out threshold judgment by relying on single optical scattering or temperature and humidity parameters, the data correction stage usually adopts a fixed baseline or single environmental parameter to carry out deviation correction, and the inversion calculation link carries out liquid water content inversion based on a preset empirical formula or a simple linear model. In addition, in the prior art, part of schemes can be locally optimized for a certain link, for example, the data reliability is improved by improving a quality control rule, or the calculation accuracy is improved by optimizing an inversion model, but the optimization focuses on a single link, and no full-chain collaborative processing logic is formed. Therefore, how to improve the inversion accuracy of the liquid water content is a technical problem that needs to be overcome at present. Disclosure of Invention The application provides a Nevzorov water content probe data quality control processing method and system based on airborne in-situ observation. The embodiment of the application provides a quality control processing method of Nevzorov water content probe data based on airborne in-situ observation, which is applied to a Nevzorov water content probe data quality control processing system, and comprises the steps of acquiring airborne in-situ observation data, and executing data classification processing based on acquired frame-time associated information of the airborne in-situ observation data to obtain an initial data set; performing combination processing based on an initial data set and an initial time threshold condition to generate an observation data set, performing quality control processing on the observation data set according to a Nevzorov probe data quality control task, identifying an abnormal data subset in an abnormal working state from the observation data set, deleting the abnormal data subset to obtain a quality control data set, extracting environment characteristic parameters and position characteristic parameters in the quality control data set, performing cloud entrance multi-condition joint analysis, screening acquired data in an entrance cloud period through multi-condition collaborative matching to serve as effective observation data, responding to a water content baseline correction task, extracting baseline reference characteristics in the effective observation data to construct a correction reference condition, performing deviation correction on the water content data in the effective observation data through the correction reference condition to obtain corrected observation data, analyzing a step particle spectrum number according to particle spectrum characteristic differences to obtain a step division result, extracting particle spectrum distribution information based on the step division result, constructing an inversion model through the association relation between the particle spectrum distribution and liquid water content, guiding the particle spectrum distribution information into the inversion model, performing inversion model inversion optimization through the characteristic matching optimization process, and generating an inversion optimization result by combining the inversion operation result and the parameter adjustment record. The embodiment of the application provides a Nevzorov water content probe data quality control processing system, which comprises a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to execute the steps of the method. An embodiment of th