CN-122027437-A - Hydrologic data grading cooperative treatment method

CN122027437ACN 122027437 ACN122027437 ACN 122027437ACN-122027437-A

Abstract

The invention discloses a hydrologic data grading cooperative treatment method, which aims to solve the problem of poor consistency of hydrologic multisource data fusion distortion and cross-node. The method comprises the steps of establishing a mapping relation library, converging real-time data of each service system device through two channels, executing automatic switching of main and standby devices based on physical measuring ranges and priorities, fusing to generate station measurement main data, acquiring the physical objective state of the full-link monitoring device, triggering a linkage mechanism when faults or overranges occur, automatically removing a calculation base number from abnormal devices, blocking data to be stored in a warehouse, dynamically optimizing and updating a quality control rule triggering threshold value based on manual correction record calculation false alarm rate and false alarm rate of a user, packaging and synchronizing a standardized data set, writing the standardized data set into a local buffer queue when network faults occur, and sequentially transmitting the standardized data after recovery. The invention obviously improves the robustness and strong consistency of the hierarchical processed bottom data.

Inventors

CHEN JINHAO
ZENG CHENGJIN
FENG JIN
HAN CHUNHUA
JIN HUI
QIN WEI
XU BOLIU
WANG SHUYING
ZENG GUOXI
PAN SHUANG
WEI JINLI
HUANG SHIWEN

Assignees

浙江省水文管理中心

Dates

Publication Date: 20260512
Application Date: 20260228

Claims (10)

1. The hydrologic data grading cooperative governance method is characterized by comprising the following steps of: The method comprises the steps of S1, establishing a mapping relation library of station measurement-equipment-element index values, converging equipment real-time data scattered in each service system to a data base comprising an original bin and an application bin, merging equipment-level original data to a corresponding station measurement based on the mapping relation library, marking monitoring equipment in the current data application state as main equipment, and marking the rest as auxiliary equipment to form a standardized data set taking the station measurement as a unit and comprising main and auxiliary equipment identifications; S2, monitoring a real-time data full link of the equipment at a main node, calculating the equipment circulation rate in real time by taking the equipment as a basic unit, and calculating the water level difference rate of the main equipment and auxiliary equipment; S3, checking the converged data by using a quality control rule set at the master node, simultaneously acquiring the physical objective state of the monitoring equipment in real time and carrying out dynamic marking, and triggering a linkage control mechanism to block real-time data of the abnormal equipment from entering an application bin when the monitoring equipment is marked as an abnormal state; S4, the main node receives a state correction instruction stream fed back by each level of sub-node based on push data, wherein the state correction instruction stream comprises manual correction operation and neglect operation triggered by each level of users aiming at alarm data; And S5, packaging the standardized data set processed by the main node and the optimized quality control rule set into a standard format message, and synchronously distributing the standard format message to each level of sub-nodes and various service applications to realize synchronous coordination of cross-level data and rules.
2. The hierarchical collaborative governance method for hydrographic data according to claim 1, wherein the step S3 further comprises a bottom-up collaborative mechanism for governance rules, each level of subnodes obtains flood prevention characteristic values of measuring stations in jurisdictions and operation conditions of matched hydraulic engineering, performs station-by-station local tuning on threshold values in the quality control rule set, reversely synchronizes the tuned rule parameters to a provincial rule base of a main node through a longitudinal synchronous data stream to be immediately effective, and simultaneously automatically maintains collaborative modification records comprising modifier, modification time and modification reasons in the main node.
3. The method for hierarchical collaborative management of hydrologic data according to claim 1 or 2, wherein in step S3, the coordinated control mechanism specifically includes automatically removing the computing base numbers of the smoothness rate and the water level difference rate of the monitoring device corresponding to the abnormal state, blocking real-time data of the monitoring device from entering the application bin, and triggering an automatic switching process of the standby device.
4. The method for hierarchical collaborative management of hydrographic data according to claim 1, wherein in step S1, when a plurality of sets of monitoring devices exist on the same element of the same station for reporting data concurrently, automatic switching from a standby device to a current data application device is performed based on a preset effective range of a water level device and a priority of a station application device, the monitoring device in the current data application state is marked as a primary device, and the rest devices are marked as secondary devices.
5. The method for hierarchical collaborative management of hydrographic data according to claim 1, wherein the calculation formula of the correlation ratio in step S2 is as follows: The device smoothness rate= (the number of devices with data reporting in the first 1 hour/the total number of effective devices) x 100%, wherein the total number of effective devices refers to the number of devices which are associated with a measuring station and are not marked as stop reporting or fault reporting by a system; the water level difference rate= (number of stations with water level difference of the main and sub devices greater than a preset limit value/total number of stations with main and sub devices) ×100%.
6. The method for hierarchical collaborative management of hydrographic data according to claim 1, wherein the calculation formula of the correlation ratio in step S4 is as follows: The false positive rate = number of anomalies marked "ignore" by the user/total number of triggers by the rule; the false negative rate = the number of anomalies manually corrected by the user but not identified by the rule/the total number of manual corrections by the user.
7. The method of claim 1 or 4, wherein the station code in the mapping relation library in step S1 is composed of 8-bit character strings, including 3 bits of a river basin water system code, 1 bit of a station type code, 2 bits of a administrative district code and 2 bits of a station serial number.
8. The method for hierarchical collaborative management of hydrographic data according to claim 1, wherein the specific rule for acquiring the physical objective state of the monitoring device in real time and performing the dynamic marking in the step S3 includes marking a device with a short-time failure as a failure, marking a device which cannot be repaired or is removed in a short time as a stop, and automatically marking water level data as an overscale abnormality when the water level data exceeds the upper and lower limits of the effective range of the water level device.
9. The method of hierarchical collaborative management of hydrologic data according to claim 1, wherein the step S1 further comprises directly aggregating artificial flood reporting data, station measurement basic information and historical characteristic data in each service system into the application bin through a second channel.
10. The method of claim 1, wherein the step S5 further comprises deploying channel monitoring agents at the main node and the sub nodes respectively, collecting the communication state, the transmission rate, the backlog data amount and the error rate index of the synchronous distribution channel in real time, and automatically sending an alarm when the channel interruption exceeds a set duration, the backlog data exceeds a threshold value or the error rate exceeds a standard.

Description

Hydrologic data grading cooperative treatment method Technical Field The invention belongs to the field of hydrology, relates to hydrological data processing in a multi-level architecture system, and particularly relates to a hydrological data grading collaborative treatment method. Background The hydrologic data is a core foundation support for flood and drought disaster defense, water resource scheduling and digital twin river basin construction. Along with the rapid expansion of the monitoring and sensing system of the Internet of things, a central main node of the provincial data center gathers massive multi-source real-time water and rain condition data. However, in the context of data volume proliferation and application diversification, the existing hydrologic data streaming and processing architecture suffers from the following prominent technical drawbacks: First, heterogeneous multi-source data fusion and streaming mechanisms have drawbacks. When the existing system processes heterogeneous data of multiple concurrent devices of the same measuring station, a bidirectional real-time synchronization and device state physical linkage mechanism of a bottom layer is lacking, so that the system cannot block abnormal data at the bottom layer according to physical objective states (such as overscan and faults) of the devices, and perform primary and backup link hot cutting, so that data distortion and poor consistency which are converged to a data base are caused. Secondly, the monitoring and early warning rules are stiff and lack self-adaptive evolution capability. When the traditional system is used for data cleaning and alarming, a static hard coding threshold value (such as a fixed water level difference limit value) is often adopted, and the traditional system is difficult to adapt to a complex and changeable real physical hydrologic environment. The rigid rule not only causes a large number of false positives or false negatives to be generated by the monitoring system, but also lacks a bidirectional state synchronization and rule cooperation mechanism under the distributed network. Each level of sub-node cannot perform local optimization and reverse synchronization of algorithm threshold based on local flood prevention features, and the main node also lacks the capability of receiving a sub-node state correction instruction stream to perform global threshold self-adaptive optimization, so that the full network computing power cannot effectively co-evolve. Finally, the multi-level node data consistency under the province, city and county distributed architecture is not guaranteed enough. The existing data flow synchronization mainly depends on a conventional transmission protocol, a directional route pushing mechanism based on a data logic identifier cannot be established at the bottom layer, and when network oscillation or interruption caused by a severe hydrologic environment occurs, a reliable local queue cache and a sequential transmission mechanism after network recovery are lacked. This is very likely to cause loss of the underlying data packets, and the final consistency of the data between the province, city and county multilevel nodes cannot be guaranteed. Disclosure of Invention Aiming at the defects of the prior art, the invention provides a hydrologic data grading collaborative treatment method, which aims to solve the technical problems of hydrologic data multisource fusion distortion, distributed inter-node regular collaborative fault, fixed monitoring threshold value rigidification and poor data consistency of all levels of nodes through physical state linkage warehouse entry blocking, logic identifier-based directional route pushing, edge instruction stream feedback-based rule collaborative optimizing and cache queue-based discontinuous network transmission mechanism. The invention aims at solving the technical problems by mainly adopting the following technical scheme that the hydrologic data grading cooperative treatment method comprises the following steps: The method comprises the steps of S1, establishing a mapping relation library containing a station-equipment-element-index value of administrative district codes, converging equipment real-time data scattered in each business system to a data base containing an original bin and an application bin, wherein the equipment real-time data adopts a first channel to be stored in the original bin in a unit of equipment, enters the application bin after being cleaned, automatically merges the equipment-level original data to a corresponding station based on the mapping relation library, marks monitoring equipment in a current data application state (namely highest priority and in an effective range) as main equipment, and marks the rest as auxiliary equipment to form a standardized data set containing main and auxiliary equipment identifiers in the unit of the station; The bottom integration from the heterogeneous system device-level isla