CN-121997066-A - Real-time water level and flow data checking and correcting method based on historical water conditions

CN121997066ACN 121997066 ACN121997066 ACN 121997066ACN-121997066-A

Abstract

The invention relates to the technical field of hydrologic monitoring data processing, in particular to a method for checking and correcting real-time water level and flow data based on historical water conditions. The method comprises the steps of obtaining historical hydrologic monitoring data of a target river basin, constructing a historical hydrologic sample library, extracting multidimensional feature factors of each sample based on the historical hydrologic sample library, constructing a sensitive feature factor library, extracting historical hydrologic feature vectors based on the sensitive feature factor library to form a related site data set, calculating similarity weight sets of each related site and the target site according to the related site data set, and obtaining real-time hydrologic monitoring data of the target site. According to the invention, through fusing multidimensional hydrologic characteristics and mining history similarity rules and constructing an intelligent distinguishing and correcting integrated process, the accuracy, adaptability and automatic processing capacity of real-time water level flow data anomaly detection are remarkably improved.

Inventors

MEI JUNYA
JIA ZHIWEI
XIE TIANXIONG
FANG HAOTIAN
HAN RUN
WEI LINYUN
CHEN WEIWEI
ZHOU BO
MOU YUN
ZHANG TING
PAN SHIHAN
DENG SHAN
WU QIONG
ZHANG LI
CHEN CHAO

Assignees

长江水利委员会水文局

Dates

Publication Date: 20260508
Application Date: 20260112

Claims (10)

1. A method for checking and correcting real-time water level and flow data based on historical water conditions is characterized by comprising the following steps: step S1, acquiring historical hydrologic monitoring data of a target river basin, and constructing a historical water regime sample library; Step S2, extracting historical water regime feature vectors based on a sensitive feature factor library to form an associated site data set; S3, acquiring real-time hydrologic monitoring data of a target site; s4, calculating the comprehensive similarity of the real-time hydrologic monitoring data and the historical samples based on the similarity weight set and the associated site data set; S5, constructing a similarity-rationality mapping model, inputting comprehensive similarity, outputting a rationality label and determining a rationality threshold; And S6, calculating the reasonable probability of all samples in the historical water regime sample library, sorting in a descending order according to the comprehensive similarity to form a neighbor sample set, calculating the comprehensive reasonable probability of the neighbor sample set, and comparing the comprehensive reasonable probability with a rationality threshold value to generate a rationality judgment result.
2. The method for checking and correcting water level and flow data in real time based on historical water conditions according to claim 1, wherein step S2 comprises: Extracting sensitive characteristic factors of sites associated with the target site in a sensitive characteristic factor library to form an associated site characteristic factor set; mapping the characteristic factor set of the associated site to a historical water regime sample library to form a historical water regime characteristic vector set of the associated site; clustering analysis is carried out on the historical water regime feature vector set to form water regime feature similarity among sites; Dividing a target river basin into a plurality of water regime associated site sets based on water regime feature similarity, and forming associated site data sets; calculating a similarity value between the target site and each associated site based on the associated site data set, and generating a site similarity set; Distributing weights to each associated site in the site similarity set, and determining a similarity weight set; the weight value of each associated site is positively correlated with the corresponding similarity, and the sum of the weights of all associated sites is 1.
3. The method for checking and correcting water level and flow data in real time based on historical water conditions according to claim 1, wherein after step S3, further comprises: Removing abnormal values in the real-time hydrologic monitoring data, and executing linear interpolation to generate a real-time continuous sequence; extracting a real-time water level mean value, a water level change rate and a water level peak value of a real-time continuous sequence according to factor dimensions in a sensitive characteristic factor library to form a real-time water level characteristic factor set; extracting a real-time flow average value, a flow change rate and a flow peak value of a real-time continuous sequence according to factor dimensions in a sensitive characteristic factor library to form a real-time flow characteristic factor set; extracting the rainfall in the corresponding time period in the real-time continuous sequence to form a real-time rainfall characteristic factor set; And constructing the real-time water regime feature factors by using the real-time water level feature factor set, the real-time flow feature factor set and the real-time rainfall feature factor set.
4. The method for checking and correcting water level and flow data in real time based on historical water conditions according to claim 1, wherein step S4 comprises: Calculating a multidimensional gap between the real-time hydrologic monitoring data and the historical sample based on the similarity weight set and the associated site data set; Performing gap normalization on the multi-dimensional gap calculation result; Converting the difference normalization result into a similarity index based on a negative exponential decay mapping relation, and integrating all the similarity indexes to form a similarity index vector; identifying a hydrologic event type corresponding to the real-time hydrologic monitoring data, wherein the hydrologic event type comprises a flood peak process, a withered water process and a stable process; extracting a corresponding dynamic weight distribution scheme in a preset event type-weight mapping table according to the hydrologic event type; weighting the similarity index vector by using a dynamic weight distribution scheme to form a similarity vector corresponding to the hydrologic event; Calculating the time attenuation weight of the historical sample; And fusing the similarity vector and the time attenuation weight into comprehensive similarity.
5. The method for checking and correcting water level and flow data based on historical water conditions according to claim 5, wherein the difference normalization result is converted into a similarity index based on a negative exponential decay mapping relation, and the conversion formula is specifically as follows: ; Wherein, the As the similarity index vector, , Is the first Difference normalization results of the individual characteristic factors; The larger the similarity index vector is, the higher the similarity between the real-time water condition and the historical sample is.
6. The method for checking and correcting water level and flow data in real time based on historical water conditions according to claim 1, wherein step S5 comprises: Extracting the comprehensive similarity and corresponding rationality labels of all samples in the historical water regime sample library to be used as a training sample set; Solving a weight vector of the similarity-rationality mapping model through an SCE-UA optimization algorithm, wherein the weight vector reflects the contribution degree of the similarity of each characteristic factor to the rationality judgment; Extracting system error information in the training sample to form a residual error correction value; uniformly arranging the weight vectors and residual correction values, and constructing a similarity-rationality mapping model; Inputting comprehensive similarity according to the similarity-rationality mapping model, and outputting a rationality label, wherein the rationality label comprises rationality and abnormality; classifying the rationality labels according to the categories according to the hydrologic year type and the water regime type to form rationality probability distribution; and extracting confidence interval probability corresponding to the target hydrologic year type in the rationality probability distribution, and determining a rationality threshold.
7. The method for checking and correcting real-time water level and flow data based on historical water conditions according to claim 1, wherein in step S6, the reasonable probability of all samples in the real-time hydrologic monitoring data and the historical water conditions sample library is calculated, and the samples are sorted in descending order according to the comprehensive similarity, and forming the neighbor sample set comprises: extracting all reasonable labels in the historical water regime sample library as reasonable samples to form a historical reasonable sample sequence; Converting the ratio of the number of samples of the historical reasonable sample sequence to the total number of samples into reasonable probability; according to reasonable probability and comprehensive similarity, performing descending order sequencing on all historical samples to generate a neighbor sample candidate sequence; And selecting the first K historical samples of the candidate sequence of the neighbor samples, and marking the first K historical samples as a neighbor sample set.
8. The method for checking and correcting water level and flow data in real time based on historical water conditions according to claim 7, wherein the first K historical samples of the candidate sequence of neighbor samples are selected and marked as a neighbor sample set, wherein the number K of neighbor samples is determined by the following adaptive method: Calculating inflection points of the comprehensive similarity sequences of the neighbor sample candidate sequences, and defining the descending gradient of the comprehensive similarity at the inflection points to exceed a preset threshold value; preliminarily determining the number of samples before the inflection point as k; Calculating the integrated similarity mean of the first k samples ; Calculating the dynamic time warping distance between the real-time water condition time sequence data and the historical water condition time sequence data of the previous k samples, and calculating the average value ; Setting a timing matching threshold ; If it is And is also provided with Then k=k is determined; otherwise, sequentially increasing the sample numbers, and re-calculating the comprehensive similarity mean value and the dynamic time warping distance mean value of the current sample set after each increase until the conditions are met And is also provided with Determining the current sample number as K; Wherein, the For a minimum acceptable similarity mean preset according to the hydrographic characteristics of the target site, And a maximum acceptable time sequence difference threshold value preset according to the hydrologic process fluctuation characteristic.
9. The method for real-time water level and flow data checking and correcting based on historical water conditions according to claim 1, wherein the calculating of the comprehensive reasonable probability of the neighbor sample set in step S6 comprises: extracting rationality labels of the neighbor sample set to form a neighbor rationality label set; weighting is distributed to the neighbor rationality label set according to the time attenuation weight, and a weighted rationality parameter is generated; and integrating the K weighted rationality parameters into the comprehensive rationality probability of the neighbor sample set.
10. The method for checking and correcting water level and flow data in real time based on historical water conditions according to claim 1, wherein after step S6, further comprises: When the rationality judging result is abnormal, extracting a history sample with highest comprehensive similarity in the neighbor sample set, and marking the history sample as a correction reference sample; Constructing a correlation between real-time data of a target site and corrected reference sample data based on the corrected reference sample, and correcting the data according to the correlation; Carrying out rationality check on the corrected data; if the rationality judging result is reasonable, outputting corrected data, otherwise, extracting a history sample similar to the second time to construct a correlation; And outputting a manual review prompt when the rationality judging structures are abnormal after the continuous 5 times of correction.

Description

Real-time water level and flow data checking and correcting method based on historical water conditions Technical Field The invention relates to the technical field of hydrologic monitoring data processing, in particular to a method for checking and correcting real-time water level and flow data based on historical water conditions. Background The hydrologic monitoring data (especially water level and flow data) are core basic data of flood and drought disaster defense, water resource scheduling, water ecological protection and hydraulic engineering operation management. In the real-time monitoring process, the data are easily affected by factors such as faults of monitoring equipment, insufficient stability of the monitoring equipment, extreme environmental interference, artificial operation errors and the like, abnormal deviation from the real water conditions occurs, the abnormal data not only can affect the use of the real-time water level and flow data, but also can affect the scientificity of hydrologic analysis and water conservancy decision. The existing real-time water level and flow data rationality checking and correcting method mainly has the following problems: Firstly, the judgment is carried out by multiple relying fixed threshold values, the abnormality is identified only by setting the upper limit and the lower limit of the water level and the flow rate or the change rate threshold value, the space-time correlation and the history similarity rule of the watershed water conditions are not considered, the reasonable fluctuation under the complex water conditions is easy to misjudge, and the slow variation is easy to miss judge. And secondly, the characteristic consideration is single, only the isolated data of the water level or the flow is checked, key influencing factors such as the early water condition state and the like are not combined, and the judgment basis is insufficient. And thirdly, the depth utilization of the historical data is lacking, the depth association of the real-time water situation and the similar water situation of the history is not established, the reasonable extremum and the abnormal data are difficult to distinguish, and the adaptability and the accuracy are insufficient. Fourth, lack effective abnormal data correction means, abnormal data still need manual experience correction, and efficiency is low. Disclosure of Invention Accordingly, the present invention is directed to a method for checking and correcting water level and flow data in real time based on historical water conditions, so as to solve at least one of the above-mentioned problems. In order to achieve the above purpose, a method for checking and correcting data of water level and flow rate in real time based on historical water conditions comprises the following steps: step S1, acquiring historical hydrologic monitoring data of a target river basin, and constructing a historical water regime sample library; Step S2, extracting historical water regime feature vectors based on a sensitive feature factor library to form an associated site data set; S3, acquiring real-time hydrologic monitoring data of a target site; s4, calculating the comprehensive similarity of the real-time hydrologic monitoring data and the historical samples based on the similarity weight set and the associated site data set; S5, constructing a similarity-rationality mapping model, inputting comprehensive similarity, outputting a rationality label and determining a rationality threshold; And S6, calculating the reasonable probability of all samples in the historical water regime sample library, sorting in a descending order according to the comprehensive similarity to form a neighbor sample set, calculating the comprehensive reasonable probability of the neighbor sample set, and comparing the comprehensive reasonable probability with a rationality threshold value to generate a rationality judgment result. The invention realizes systematic extraction and screening of multidimensional and high-order features in the hydrologic process by constructing the historical water regime sample library and the sensitive feature factor library, solves the problem of single feature consideration in the traditional method, fully fuses the features of water level, flow, rainfall and space-time variation thereof, and provides a more comprehensive and explanatory basis for rationality judgment. By introducing a correlation site similarity weight and comprehensive similarity calculation mechanism, the space-time correlation and history similarity rules of the watershed water conditions are effectively utilized, the misjudgment of reasonable fluctuation of the complex water conditions and the missed judgment of slow-change anomalies are avoided, and the adaptability and accuracy of data inspection are improved. By constructing a similarity-rationality mapping model and combining comprehensive reasonable probability judgment of a neighbor sample set, de