CN-121509452-B - Data collaborative management method and system for multi-site water conservancy monitoring network

CN121509452BCN 121509452 BCN121509452 BCN 121509452BCN-121509452-B

Abstract

The application relates to the technical field of data processing, and discloses a data collaborative management method and system of a multi-station water conservancy monitoring network. The method comprises the steps of carrying out hierarchical identification coding on a plurality of monitoring stations to generate global identifiers, packaging the global identifiers into standard data packets, carrying out transmission scheduling according to geographic positions and station types through a time division multiplexing mechanism, calculating correlation consistency indexes among the stations by utilizing a multi-dimensional data quality evaluation algorithm to carry out anomaly correction, storing the correlation consistency indexes according to the time dimension fragments, constructing a three-dimensional collaborative index system, extracting multi-station monitoring data to calculate fusion indexes, and generating collaborative decision suggestions. The method solves the problems of non-uniform identification, uncoordinated transmission, difficult quality guarantee, low query efficiency and incapability of collaborative decision-making of the multi-site water conservancy monitoring data in the prior art, and improves the data management efficiency and the scientificity of irrigation decision-making of the multi-site water conservancy monitoring network.

Inventors

CAI CHANGJU
LI YUN
LI CHANGJIANG
PENG YUN
ZHAO SONGBO
HUANG CHENG
ZHA YUANYUAN
LU YIXIN
Shi Handan
GUO XIANG
DENG WENQIANG
WU HAIKUAN
ZHU BOWEN
ZHANG YANG
LI ZHUOYAN
MENG TIANYI
SHI LIANGSHENG
FANG XIAOYU
An Meiyun
BO YANPING
LI XINAN
ZHU YAN
KANG JINGANG

Assignees

贵州省水利科学研究院

Dates

Publication Date: 20260505
Application Date: 20251027

Claims (10)

1. The data collaborative management method of the multi-site water conservancy monitoring network is characterized by comprising the following steps of: Step S1, carrying out hierarchical identification coding processing on a plurality of monitoring stations to generate a global identifier, and packaging the original data collected by each station into a standard data packet containing the global identifier, a time stamp and a data quality identifier; s2, dividing a transmission scheduling area according to geographical position information of a station, setting transmission priority according to the type of the station, and distributing transmission time slots through a time division multiplexing mechanism to generate a transmission scheduling instruction; step S3, carrying out rationality inspection on the standard data packet, extracting data sequences of a plurality of stations in the same irrigation area, calculating association consistency indexes among the stations through a multidimensional data quality evaluation algorithm, positioning abnormal data according to the association consistency indexes, and correcting to generate a collaborative data set; step S4, storing the collaborative data set according to the time dimension in a slicing way, establishing a spatial index, a time index and a type index, and constructing a three-dimensional collaborative index system; and S5, extracting monitoring data of a plurality of stations in the irrigation area from the three-dimensional collaborative index system, calculating multi-station fusion indexes, and generating collaborative decision suggestions according to the multi-station fusion indexes.
2. The method for collaborative management of data of a multi-site water conservancy monitoring network according to claim 1, wherein step S1 comprises: three-layer identification codes are carried out on a plurality of monitoring stations, a first layer adopts a provincial code and a shrub area serial number to combine to generate a shrub area level code, a second layer generates a type code according to the type of the stations, a third layer generates a device code according to the type of the devices and the device serial number, and the three-layer codes are combined to obtain a global identifier; Extracting the original data collected by each monitoring station, obtaining a time stamp corresponding to each original data, judging whether each original data exceeds a preset value range, and marking a data quality identifier according to a judging result; Organizing the global identifier, the time stamp, the data type tag, the numerical value and the unit of the original data and the data quality identifier according to a preset format to form a data encapsulation rule; and presetting global identifiers of corresponding stations in data transmission units of all monitoring stations, and carrying out packaging processing on the original data according to data packaging rules to obtain standard data packets.
3. The method for collaborative management of data of a multi-site water conservancy monitoring network according to claim 1, wherein the step S2 comprises: extracting a irrigated area level code of a global identifier from a standard data packet, acquiring geographic position information of each monitoring site according to the irrigated area level code, and dividing a plurality of irrigated areas into transmission scheduling areas; extracting the type code of the global identifier from the standard data packet, determining the site type of each monitoring site according to the type code, and setting transmission priority and transmission period for each monitoring site according to the site type; Dividing a transmission period into a plurality of time slots, sequentially distributing the time slots to different monitoring stations according to the transmission priority of each monitoring station, and obtaining a time slot distribution scheme; generating a transmission scheduling instruction based on a time slot allocation scheme, and transmitting the transmission scheduling instruction to a data transmission unit of each monitoring station in each transmission scheduling area, so that each monitoring station transmits a standard data packet in the allocated time slot.
4. The method for collaborative management of data of a multi-site water conservancy monitoring network according to claim 1, wherein the step S3 comprises: Extracting the numerical value in the received standard data packet, judging whether the numerical value exceeds a preset reasonable value range, marking the standard data packet exceeding the range as rationality abnormality, calculating the data change rate of continuous standard data packets of the same monitoring station, judging whether the data change rate exceeds a preset change rate threshold value, marking the standard data packet exceeding the threshold value as mutation abnormality, and obtaining a single-station inspection result; Extracting a irrigated area level code from the standard data packets, screening out the standard data packets of a plurality of monitoring stations in the same irrigated area, sequencing the standard data packets according to the time stamps, and extracting the standard data packets in a preset time period to form a data sequence; Inputting the data sequence into a multidimensional data quality evaluation algorithm for processing, extracting the soil moisture content change amount and time interval in the data sequence, calculating the soil moisture content change rate, extracting the accumulated flow value of a head flow station and the accumulated flow value of a field flow station, calculating the flow difference, extracting the time stamp calculation time deviation of the two flow stations, calculating the associated consistency index according to the soil moisture content change rate, the flow difference and the time deviation, and determining that the group of site data are inconsistent when the associated consistency index is smaller than a preset threshold value; counting the frequency of marked abnormal of each monitoring station in all possible combinations in the inconsistent station combination, judging the monitoring station with the highest frequency as an abnormal data source, extracting normal standard data packets in the history preset days of the abnormal data source, establishing a time sequence prediction model, inputting an actual measurement value at the previous moment and a smooth prediction value at the previous moment into the time sequence prediction model for exponential smoothing calculation, obtaining correction value at the current moment to replace abnormal data, and generating a collaborative data set.
5. The method for collaborative management of data of a multi-site water conservancy monitoring network according to claim 1, wherein step S4 comprises: Extracting the time stamp of the standard data packet from the collaborative data set, performing time dimension slicing on the collaborative data set according to the time stamp, and generating data slicing by taking the preset time length as the slicing granularity; extracting a global identifier from standard data packets in each data slice, performing secondary-stage storage on the collaborative data set according to the global identifier, and aggregating and storing the data of the same monitoring site; extracting the irrigated area level code from the global identifier of the standard data packet to establish a spatial index, extracting the time stamp from the standard data packet to establish a time index, and extracting the type code from the global identifier to establish a type index; and carrying out combined association on the spatial index, the time index and the type index, and constructing a three-dimensional collaborative index system according to the combined relation of the irrigation area level code, the time stamp and the type code, so that the combined query through irrigation area positioning, time screening and type filtering is positioned to target monitoring data.
6. The method for collaborative management of data in a multi-site water conservancy monitoring network according to claim 5, wherein the extracting a global identifier from standard data packets in each data slice, performing secondary-level hierarchical storage on collaborative data sets according to the global identifier, aggregating and storing data of a same monitoring site, comprises: extracting a global identifier from each standard data packet in the data fragment, carrying out hash calculation on the global identifier to obtain a hash value, and distributing the standard data packets to a preset number of storage partitions according to the hash value; Counting the distribution condition of the global identifiers of the standard data packets in each storage partition, calculating the number of the standard data packets corresponding to each global identifier, judging whether data inclination exists according to the number, and triggering partition rebalancing when the number of the standard data packets in a single storage partition exceeds the preset multiple of the average value; Grouping global identifiers in the storage partition triggering rebalancing according to site types, migrating standard data packets of the same site type to a new storage partition, and updating the mapping relation between the global identifiers and the storage partition; And establishing an index table of the global identifier in each storage partition, wherein the index table records the physical offset of the global identifier and the corresponding standard data packet thereof in the storage partition, so that the data aggregation position of the same monitoring site is quickly positioned through the global identifier.
7. The method for collaborative management of data of a multi-site water conservancy monitoring network according to claim 1, wherein the step S5 comprises: Positioning a irrigated area level code of a target irrigated area through a spatial index in a three-dimensional collaborative index system, extracting global identifiers of a plurality of monitoring stations in the irrigated area according to the irrigated area level code, screening time stamps in a preset evaluation period through a time index, and filtering out standard data packets of a soil moisture content station, a field flow station and a canal head flow station through a type index to obtain monitoring data of the target irrigated area; Extracting initial water content and end water content of a soil moisture content station from monitoring data, extracting accumulated flow value of a field flow station and accumulated water supply of a canal head flow station, calculating the ratio of the accumulated flow value of the field flow station to the accumulated water supply of the canal head flow station to obtain a water utilization coefficient, calculating the product of the soil water content difference and the irrigation area, the depth of a soil root system layer and the volume weight of soil, dividing the product by the accumulated flow value of the field flow station to obtain soil water supplementing efficiency, and taking the water utilization coefficient and the soil water supplementing efficiency as multi-site fusion indexes; Comparing the multi-station fusion index with a preset threshold, generating a first decision suggestion for optimizing a water delivery channel when the water utilization coefficient is lower than a first preset threshold, generating a second decision suggestion for adjusting an irrigation system when the soil water supplementing efficiency is lower than a second preset threshold, and generating a third decision suggestion for reducing the irrigation water when the water content exceeds the upper limit of the water holding capacity of the field after the completion; Pushing the first decision proposal, the second decision proposal or the third decision proposal to the management terminal as a collaborative decision proposal.
8. A data collaborative management system of a multi-site water conservancy monitoring network, which is used for realizing the data collaborative management method of the multi-site water conservancy monitoring network according to any one of claims 1-7, wherein the data collaborative management system of the multi-site water conservancy monitoring network comprises: The coding module is used for carrying out hierarchical identification coding processing on a plurality of monitoring stations, generating a global identifier, and packaging the original data collected by each station into a standard data packet containing the global identifier, a time stamp and a data quality identifier; The division module is used for dividing a transmission scheduling area according to the geographical position information of the station, setting transmission priority according to the type of the station, distributing transmission time slots through a time division multiplexing mechanism and generating a transmission scheduling instruction; the correction module is used for carrying out rationality test on the standard data packet, extracting data sequences of a plurality of stations in the same irrigation area, calculating association consistency indexes among the stations through a multidimensional data quality evaluation algorithm, positioning abnormal data according to the association consistency indexes and correcting the abnormal data to generate a collaborative data set; The segmentation module is used for storing the collaborative data set in a segmented manner according to the time dimension, establishing a spatial index, a time index and a type index, and constructing a three-dimensional collaborative index system; The extraction module is used for extracting monitoring data of a plurality of stations in the irrigation area from the three-dimensional collaborative index system, calculating multi-station fusion indexes and generating collaborative decision suggestions according to the multi-station fusion indexes.
9. A data collaborative management apparatus of a multi-site water conservancy monitoring network, comprising a memory and a processor, the memory storing a computer program executable on the processor, the processor implementing the data collaborative management method of the multi-site water conservancy monitoring network of any one of claims 1 to 7 when executing the computer program.
10. A computer readable storage medium having stored thereon a computer program, which when run by a processor causes the processor to perform a data collaborative management method of a multi-site water conservancy monitoring network according to any one of claims 1 to 7.

Description

Data collaborative management method and system for multi-site water conservancy monitoring network Technical Field The application relates to the technical field of data processing, in particular to a data collaborative management method and system of a multi-site water conservancy monitoring network. Background The existing water conservancy monitoring system is generally provided with a plurality of monitoring stations such as soil moisture content stations, field flow stations, canal head flow stations and the like in irrigation areas and used for collecting key parameters such as soil moisture content, irrigation flow and the like, the monitoring stations transmit collected data to a cloud management platform through a wireless communication network, and management staff check monitoring data of all stations through the platform to master irrigation water conditions of the irrigation areas. In terms of data transmission, each monitoring station generally adopts wireless communication technologies such as NB-IoT, GPRS or 4G to upload data to the cloud, in terms of data storage, the cloud platform stores the received data in a database for subsequent query and analysis, and in terms of data analysis, management staff evaluates irrigation effects and makes an irrigation plan according to the data reported by each station. The prior art has the following defects that firstly, in the aspect of data identification, each monitoring station adopts independent coding rules, a unified station identification system is lacking, data formats are inconsistent and cross-station data association analysis is difficult to perform, secondly, in the aspect of data transmission, each monitoring station independently transmits data, a coordination mechanism is lacking, network congestion and data transmission delay are easy to cause when a plurality of stations transmit simultaneously, the real-time performance of the data is influenced, in the aspect of data quality management, the prior system generally only independently checks the data of a single station, and lacks a mechanism for cross-validation by utilizing the inherent association relation of multi-station data, so that abnormal data is difficult to accurately identify and correct, in the aspect of data storage, the prior system generally adopts a centralized storage mode, and the fragmentation and index optimization aiming at the multi-station time sequence data characteristics are lacking, and the query efficiency is remarkably reduced when the data quantity is increased. The method is characterized in that a unified site identification system and a data encapsulation standard are lacked in the prior art, multi-site data cannot be subjected to differentiated management according to site attributes during transmission scheduling, high-priority and low-priority data are transmitted in a mixed mode, real-time performance of key data cannot be guaranteed, systematic anomalies cannot be easily identified by simply relying on single-site inspection due to lack of a quality evaluation mechanism based on data association relation among sites, for example, when channel head water supply is normal but field flow is abnormally reduced, channel leakage or flow meter faults cannot be judged, abnormal data source positioning is inaccurate, due to lack of a multi-dimensional index system for multi-site data, when data of a plurality of sites in a certain irrigation area in a specific time period need to be comprehensively analyzed, all the data must be traversed for screening, query efficiency is low, real-time decision requirements are difficult to support, finally, the whole process utilization efficiency index reflecting the fact that irrigation water flows from the head to the field to the soil cannot be calculated accurately and scientific decision advice based on multi-site collaborative analysis cannot be generated. Disclosure of Invention The application provides a data collaborative management method and a system for a multi-site water conservancy monitoring network, which are used for realizing unified identification and formatting processing of different types of monitoring site data by establishing a multi-level site identification coding system and a standard data packet encapsulation rule, avoiding network conflict by a time division multiplexing transmission scheduling mechanism based on site attributes, carrying out cross verification and anomaly correction by utilizing a site association relationship by a multi-dimensional data quality assessment algorithm, realizing quick retrieval of multi-site data by a time dimension fragmentation storage and three-dimensional collaborative indexing system, and solving the problems of non-uniform transmission, uncoordinated quality, low query efficiency and incapacity of collaborative decision of the multi-site water conservancy monitoring network in the prior art by calculating comprehensive indexes such as water utilizati