CN-122024920-A - Water quality monitoring data filling method, device, equipment, storage medium and product

CN122024920ACN 122024920 ACN122024920 ACN 122024920ACN-122024920-A

Abstract

The application discloses a water quality monitoring data filling method, a device, equipment, a storage medium and a product, which relate to the technical field of data processing, and the disclosed water quality monitoring data filling method comprises the steps of acquiring a water quality characteristic data set of a target water body, wherein the water quality characteristic data set comprises a time coding sequence, a flow sequence and a total nitrogen concentration sequence with missing values; inputting a water quality characteristic data set into a pre-trained data filling model to obtain a total nitrogen concentration filling value sequence output by the data filling model, wherein the data filling model is a model of an encoder decoder architecture based on a long-short-term memory network, respectively carrying out linear mapping treatment on each filling value in the total nitrogen concentration filling value sequence according to the total nitrogen concentration numerical range in the total nitrogen concentration sequence to obtain a mapping filling value sequence, and combining the mapping filling value sequence and the total nitrogen concentration sequence to obtain a target total nitrogen concentration sequence. The application can improve the accuracy of filling the water quality monitoring data.

Inventors

LIU ZHIQIANG
PAN GUANYU
ZHANG GUANGTAO
GAO JUNXUAN
LIN JIE
HUANG YUTING
SUN TAOTAO

Assignees

南方科技大学

Dates

Publication Date: 20260512
Application Date: 20251231

Claims (10)

1. The water quality monitoring data filling method is characterized by comprising the following steps of: acquiring a water quality characteristic data set of a target water body, wherein the water quality characteristic data set comprises a time coding sequence, a flow sequence and a total nitrogen concentration sequence with a missing value; Inputting the water quality characteristic data set into a pre-trained data filling model to obtain a total nitrogen concentration filling value sequence output by the data filling model, wherein the data filling model is a model of an encoder decoder architecture based on a long-term and short-term memory network; respectively performing linear mapping treatment on each filling value in the total nitrogen concentration filling value sequence according to the total nitrogen concentration numerical range in the total nitrogen concentration sequence to obtain a mapping filling value sequence; and combining the mapping filling value sequence and the total nitrogen concentration sequence to obtain a target total nitrogen concentration sequence.
2. The water quality monitoring data population method of claim 1, wherein the step of obtaining a water quality characteristic dataset of the target water body comprises: Acquiring flow data, total nitrogen concentration data and a data acquisition time stamp of a target water body in a preset time window; sequencing the flow data, the total nitrogen concentration data and the data acquisition time stamp based on a time sequence to obtain an initial flow sequence, an initial total nitrogen concentration sequence and a time stamp sequence; performing sinusoidal coding on each time stamp in the time stamp sequence to obtain the time coding sequence; carrying out standardization processing on the initial flow sequence to obtain a standardized flow sequence; Based on a quantile statistical algorithm, respectively carrying out truncation treatment on outliers in the standardized flow sequence and the initial total nitrogen concentration sequence to obtain the flow sequence and the total nitrogen concentration sequence; And integrating the time coding sequence, the flow sequence and the total nitrogen concentration sequence to obtain a water quality characteristic data set.
3. The water quality monitoring data population method of claim 1, wherein prior to the step of inputting the water quality characteristic dataset into a pre-trained data population model to obtain a sequence of total nitrogen concentration population values output by the data population model, further comprising: Acquiring a historical water quality characteristic dataset as a training sample; Inputting the training sample into an initial data filling model to obtain a training total nitrogen concentration filling value sequence; Calculating a mixed loss function based on the training total nitrogen concentration filling value sequence and the real total nitrogen concentration observation value sequence to obtain mixed loss values, wherein the time stamp of each observation value in the real total nitrogen concentration observation value sequence corresponds to the time stamp of each training filling value in the training total nitrogen concentration filling value sequence, and the mixed loss function comprises a trend loss term function and a fidelity loss term function; And iteratively updating parameters of the initial data filling model according to the mixed loss value to obtain a trained data filling model.
4. A water quality monitoring data filling method as claimed in claim 3, wherein said step of calculating a mixing loss function based on said training total nitrogen concentration filling value sequence and a true total nitrogen concentration observation value sequence, to obtain a mixing loss value, comprises: Respectively performing linear mapping treatment on each training filling value in the training total nitrogen concentration filling value sequence according to the total nitrogen concentration numerical range in the real total nitrogen concentration observation value sequence to obtain a training mapping filling value sequence; Calculating a first mean square error value between the training mapping filling value sequence and the real total nitrogen concentration observation value sequence, and taking the first mean square error value as a trend loss value; calculating a second mean square error value between the training mapping filling value sequence and the training total nitrogen concentration filling value sequence, and taking the second mean square error value as a fidelity loss value; and carrying out weighted summation on the trend loss value and the fidelity loss value to obtain a mixed loss value.
5. The water quality monitoring data population method of claim 1, wherein the data population model comprises an encoder, a decoder, and a fully connected output layer; the step of inputting the water quality characteristic data set into a pre-trained data filling model to obtain a total nitrogen concentration filling value sequence output by the data filling model comprises the following steps: Inputting the water quality characteristic dataset to the encoder to extract a time-series correlation characteristic between the flow sequence and the total nitrogen concentration sequence by the encoder; Inputting the time sequence correlation characteristic to the decoder so as to calculate a hidden state vector of a time step where the missing value is located according to the time sequence correlation characteristic by the decoder; and inputting the hidden state vector to the fully-connected output layer, so that the hidden state vector is mapped into a total nitrogen concentration filling value through the fully-connected output layer, and a total nitrogen concentration filling value sequence is obtained.
6. The filling method of water quality monitoring data according to claim 1, wherein the step of performing linear mapping processing on each filling value in the total nitrogen concentration filling value sequence according to the total nitrogen concentration numerical range in the total nitrogen concentration sequence to obtain a mapped filling value sequence comprises the following steps: determining a total nitrogen concentration numerical range in the total nitrogen concentration sequence and a filling value numerical range in the total nitrogen concentration filling value sequence; And carrying out linear mapping processing on each filling value in the total nitrogen concentration filling value sequence according to the total nitrogen concentration numerical range and the filling value numerical range so that each filling value and an observed value in the total nitrogen concentration sequence are in the same numerical interval, and obtaining a mapping filling value sequence.
7. A water quality monitoring data filling device, characterized in that the water quality monitoring data filling device comprises: The system comprises a data set acquisition module, a target water body acquisition module and a target water body acquisition module, wherein the data set acquisition module is used for acquiring a water quality characteristic data set of the target water body, and the water quality characteristic data set comprises a time coding sequence, a flow sequence and a total nitrogen concentration sequence with a missing value; The data filling module is used for inputting the water quality characteristic data set into a pre-trained data filling model to obtain a total nitrogen concentration filling value sequence output by the data filling model, wherein the data filling model is a model of an encoder decoder architecture based on a long-term and short-term memory network; The filling value mapping module is used for respectively carrying out linear mapping treatment on each filling value in the total nitrogen concentration filling value sequence according to the total nitrogen concentration numerical range in the total nitrogen concentration sequence to obtain a mapping filling value sequence; and the sequence merging module is used for merging the mapping filling value sequence and the total nitrogen concentration sequence to obtain a target total nitrogen concentration sequence.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program configured to implement the steps of the water quality monitoring data population method of any one of claims 1 to 6.
9. A storage medium, characterized in that the storage medium is a computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the steps of the water quality monitoring data filling method according to any one of claims 1 to 6.
10. A computer program product, characterized in that the computer program product comprises a computer program which, when executed by a processor, implements the steps of the water quality monitoring data filling method as claimed in any one of claims 1 to 6.

Description

Water quality monitoring data filling method, device, equipment, storage medium and product Technical Field The application relates to the technical field of data processing, in particular to a water quality monitoring data filling method, a device, equipment, a storage medium and a product. Background In water quality monitoring, total nitrogen concentration is used as a key index, and continuous observation data is important for water quality assessment. However, due to the influences of factors such as equipment faults, transmission interruption and severe environments, the obtained total nitrogen concentration sequence data have missing values with different lengths and observation breakpoints, and in order to cope with the data missing problem, statistical interpolation and deep learning models are generally adopted at present to carry out data filling, but the filling result is easy to generate overall trend deviation, so that the data filling effect is poor, and the reliability of subsequent data analysis is influenced. In summary, how to improve the accuracy of filling water quality monitoring data is a significant technical problem in the art. Disclosure of Invention The application mainly aims to provide a water quality monitoring data filling method, a device, equipment, a storage medium and a product, and aims to improve the accuracy of water quality monitoring data filling. In order to achieve the above object, the present application provides a water quality monitoring data filling method, which includes: acquiring a water quality characteristic data set of a target water body, wherein the water quality characteristic data set comprises a time coding sequence, a flow sequence and a total nitrogen concentration sequence with a missing value; Inputting the water quality characteristic data set into a pre-trained data filling model to obtain a total nitrogen concentration filling value sequence output by the data filling model, wherein the data filling model is a model of an encoder decoder architecture based on a long-term and short-term memory network; respectively performing linear mapping treatment on each filling value in the total nitrogen concentration filling value sequence according to the total nitrogen concentration numerical range in the total nitrogen concentration sequence to obtain a mapping filling value sequence; and combining the mapping filling value sequence and the total nitrogen concentration sequence to obtain a target total nitrogen concentration sequence. In one embodiment, the step of acquiring the water quality characteristic dataset of the target water body includes: Acquiring flow data, total nitrogen concentration data and a data acquisition time stamp of a target water body in a preset time window; sequencing the flow data, the total nitrogen concentration data and the data acquisition time stamp based on a time sequence to obtain an initial flow sequence, an initial total nitrogen concentration sequence and a time stamp sequence; performing sinusoidal coding on each time stamp in the time stamp sequence to obtain the time coding sequence; carrying out standardization processing on the initial flow sequence to obtain a standardized flow sequence; Based on a quantile statistical algorithm, respectively carrying out truncation treatment on outliers in the standardized flow sequence and the initial total nitrogen concentration sequence to obtain the flow sequence and the total nitrogen concentration sequence; And integrating the time coding sequence, the flow sequence and the total nitrogen concentration sequence to obtain a water quality characteristic data set. In an embodiment, before the step of inputting the water quality feature data set into a pre-trained data filling model to obtain the total nitrogen concentration filling value sequence output by the data filling model, the method further includes: Acquiring a historical water quality characteristic dataset as a training sample; Inputting the training sample into an initial data filling model to obtain a training total nitrogen concentration filling value sequence; Calculating a mixed loss function based on the training total nitrogen concentration filling value sequence and the real total nitrogen concentration observation value sequence to obtain mixed loss values, wherein the time stamp of each observation value in the real total nitrogen concentration observation value sequence corresponds to the time stamp of each training filling value in the training total nitrogen concentration filling value sequence, and the mixed loss function comprises a trend loss term function and a fidelity loss term function; And iteratively updating parameters of the initial data filling model according to the mixed loss value to obtain a trained data filling model. In one embodiment, the step of calculating a mixing loss function based on the training total nitrogen concentration filling value sequence and the real total nitro