CN-121765442-B - Data analysis method and system based on detector in pipeline

CN121765442BCN 121765442 BCN121765442 BCN 121765442BCN-121765442-B

Abstract

The application provides a data analysis method and a system based on a detector in a pipeline, and relates to the technical field of data processing, wherein the method comprises the steps of obtaining historical pipeline signals; the method comprises the steps of preprocessing pipeline signals, training a pipeline anomaly perception LSTM model by utilizing historical pipeline signals to obtain a target pipeline anomaly perception LSTM model, obtaining real-time pipeline signals, inputting the real-time pipeline signals into the target pipeline anomaly perception LSTM model to obtain judging parameters of the real-time pipeline signals, wherein the judging parameters comprise a predicted signal, a residual sequence and confidence, determining the abnormal signals according to the judging parameters, wherein the abnormal signals comprise invalid signals, weak signals and jump signals, and determining pipeline space positions corresponding to the abnormal signals based on mapping information, so that the problems that the existing pipeline detection method is low in efficiency, complex abnormal signals cannot be automatically identified and processed, and the accuracy of detection results is low are solved.

Inventors

SONG HUADONG
SONG TING
QU TIANYI
QU YUHANG
ZHANG YUSONG
Wang Qiangjia
WANG QINGYA
LIU JIN
HU WENGUANG
GUO XIAOTING
CHEN HONGHE
ZENG YANLI
LIU GUANLIN
XU CHUNFENG
LI XIN

Assignees

国机传感科技有限公司
沈阳仪表科学研究院有限公司

Dates

Publication Date: 20260512
Application Date: 20260302

Claims (8)

1. A method of analyzing data based on an in-line detector, comprising: The method comprises the steps of obtaining historical pipeline signals, wherein the pipeline signals comprise magnetic flux leakage signals, eddy current signals, stress signals and mileage wheel signals, the mileage wheel signals comprise mapping information of time indexes and corresponding mileage numbers of pipeline detection, and the mapping information is characterized by a corresponding relation between pipeline signal acquisition time and the mileage numbers of the pipeline to be detected; The method comprises the steps of preprocessing a pipeline signal, wherein the preprocessing comprises time synchronization processing, improved normalization processing, adaptive signal drift removal processing, defect characteristic protection type noise filtering and multi-threshold local peak value holding processing which are sequentially executed, and the improved normalization processing introduces defect characteristic weight factors, wherein the specific formula is as follows: ; In the formula, For the original pipeline signal values before preprocessing, The normalized pipeline signal value; is the mean value of the original pipeline signal sequence, Standard deviation of the original pipeline signal sequence; 、 Respectively the minimum value and the maximum value of the original pipeline signal sequence; the weight factor is enhanced for the defect characteristics, and the value range is 0.3 less than or equal to Less than or equal to 0.6, and carrying out statistics calibration through historical defect signal samples; Training a pipeline anomaly perception LSTM model by utilizing the history pipeline signal after preprocessing operation to obtain a target pipeline anomaly perception LSTM model, wherein the pipeline anomaly perception LSTM model is improved by an LSTM network model; The pipeline anomaly perception LSTM model comprises an input layer, a preprocessing unit, a timing sequence characteristic extraction unit, a detection unit and a detection unit, wherein the input layer of the pipeline anomaly perception LSTM model is integrated with the preprocessing unit which is configured to separate local peaks of the pipeline signals from trend components and extract the timing sequence characteristic of the pipeline signals, and the trend components comprise a substrate interference signal generated by the uniformity change of materials and wall thickness of a pipeline to be detected, an electromagnetic noise signal generated by the external magnetic field interference and the speed change of a detector and a motion noise signal generated by the running speed change of the detector; A hidden layer of the pipeline anomaly aware LSTM model introduces a gating confidence mechanism, the hidden layer being configured to output a prediction signal and a confidence based on the timing characteristics; the output layer of the pipeline anomaly perception LSTM model is provided with a residual error detection unit, wherein the residual error detection unit is configured to calculate the difference value between a predicted signal and a corresponding pipeline signal to generate a residual error sequence; Acquiring a real-time pipeline signal and executing the preprocessing operation; Inputting the real-time pipeline signal after the preprocessing operation into the target pipeline anomaly perception LSTM model to obtain the judgment parameters of the real-time pipeline signal, wherein the judgment parameters comprise a predicted signal, a residual sequence and a confidence coefficient; Determining an abnormal signal according to the judging parameter, wherein the abnormal signal comprises an invalid signal, a weak signal and a jump signal; and determining the pipeline space position corresponding to the abnormal signal based on the mapping information.
2. The method for analyzing data based on an in-pipeline detector according to claim 1, wherein the step of training the pipeline anomaly-aware LSTM model by using the historical pipeline signal data after the preprocessing operation to obtain the target pipeline anomaly-aware LSTM model comprises the steps of: Labeling the history pipeline signals after the preprocessing operation to obtain a normal signal sample set and an abnormal signal sample set, and dividing the normal signal sample set and the abnormal signal sample set into a training set, a verification set and a test set according to a preset proportion; Inputting the training set into a pipeline anomaly perception LSTM model for training, and carrying out iterative verification on the pipeline anomaly perception LSTM model by utilizing the verification set in the training process, wherein a loss function of the model is formed by self-adaptive weighted fusion of a weighted prediction error item and a confidence coefficient constraint item; And stopping training when the signal identification accuracy of the pipeline anomaly sensing LSTM model on the test set reaches a preset accuracy threshold and the loss function value converges to a preset loss threshold, and obtaining the target pipeline anomaly sensing LSTM model.
3. The method of claim 1, wherein the step of determining an anomaly signal based on the decision parameters comprises: Acquiring a signal value of a real-time pipeline signal; And if the signal value of the preset number of adjacent signals in the real-time pipeline signals is 0 and the corresponding confidence coefficient is smaller than a preset confidence coefficient threshold value, the corresponding signal section of the real-time pipeline signals is an invalid signal.
4. The method of claim 1, wherein the step of determining an anomaly signal based on the decision parameters comprises: calculating a prediction variance of a prediction signal in a sliding window, wherein the prediction variance is as follows: ; Wherein N is the total number of predicted signals in the sliding window, x i is the signal value of the ith predicted signal in the sliding window; A signal average value of the prediction signal in the sliding window; Acquiring a signal energy average value of a real-time pipeline signal corresponding to the prediction signal in the sliding window; if the prediction variance of the prediction signal is smaller than a preset variance threshold and the signal energy value is smaller than a preset energy threshold, the real-time pipeline signal corresponding to the prediction signal in the sliding window is a weak signal.
5. The method of claim 1, wherein the step of determining an anomaly signal based on the decision parameters comprises: Calculating the difference absolute value of adjacent residual values in the residual sequence; Carrying out weighted calculation by utilizing the differential absolute value to obtain a residual mutation characteristic value; and if the residual abrupt change characteristic value is larger than a preset jump threshold value, the real-time pipeline signal corresponding to the residual abrupt change characteristic value is a jump signal.
6. The method for analyzing data based on an in-pipeline detector according to claim 1, wherein the step of determining the pipeline spatial position corresponding to the abnormal signal based on the mapping information comprises: determining single-point mileage coordinates corresponding to the abnormal signals based on the mapping information; if the single-point mileage coordinates show continuous distribution characteristics on a time axis, performing confidence weighting smooth fusion processing on continuous abnormal signal sections by using a gating confidence mechanism of the target pipeline abnormal perception LSTM model to obtain continuous abnormal signal section mileage ranges; And determining the pipeline space position corresponding to the abnormal signal based on the single-point mileage coordinates and the abnormal signal section mileage range.
7. The method of in-line detector-based data analysis of claim 6, further comprising: and generating a pipeline detection report according to the type of the abnormal signal, the judging parameter and the pipeline space position corresponding to the abnormal signal, wherein the pipeline detection report is used for displaying the mileage of the abnormal signal in the pipeline to be detected, the time index corresponding to the abnormal signal and the confidence corresponding to the abnormal signal.
8. An in-line detector-based data analysis system, comprising: The system comprises a first acquisition module, a first acquisition module and a second acquisition module, wherein the first acquisition module is configured to acquire historical pipeline signals, the pipeline signals comprise magnetic flux leakage signals, vortex signals, stress signals and mileage wheel signals, the mileage wheel signals comprise mapping information of time indexes of pipeline detection and corresponding mileage numbers, and the mapping information is characterized by a corresponding relation between pipeline signal acquisition time and the mileage numbers of the pipeline to be detected; The preprocessing module is configured to perform preprocessing operation on the pipeline signals, the preprocessing operation comprises time synchronization processing, improved normalization processing, adaptive signal drift removal processing, defect characteristic protection type noise filtering and multi-threshold local peak value holding processing which are sequentially executed, and the improved normalization processing introduces defect characteristic weight factors, wherein the specific formula is as follows: ; In the formula, For the original pipeline signal values before preprocessing, The normalized pipeline signal value; is the mean value of the original pipeline signal sequence, Standard deviation of the original pipeline signal sequence; 、 Respectively the minimum value and the maximum value of the original pipeline signal sequence; the weight factor is enhanced for the defect characteristics, and the value range is 0.3 less than or equal to Less than or equal to 0.6, and carrying out statistics calibration through historical defect signal samples; The system comprises a training module, a target pipeline anomaly perception LSTM model, a pipeline anomaly perception LSTM model and a pipeline anomaly perception LSTM model, wherein the training module is configured to train the pipeline anomaly perception LSTM model by utilizing the history pipeline signal after preprocessing operation to obtain the target pipeline anomaly perception LSTM model, the pipeline anomaly perception LSTM model is improved by an LSTM network model, and the pipeline anomaly perception LSTM model comprises an input layer, a hidden layer and an output layer; The pipeline anomaly perception LSTM model comprises an input layer, a preprocessing unit, a timing sequence characteristic extraction unit, a detection unit and a detection unit, wherein the input layer of the pipeline anomaly perception LSTM model is integrated with the preprocessing unit which is configured to separate local peaks of the pipeline signals from trend components and extract the timing sequence characteristic of the pipeline signals, and the trend components comprise a substrate interference signal generated by the uniformity change of materials and wall thickness of a pipeline to be detected, an electromagnetic noise signal generated by the external magnetic field interference and the speed change of a detector and a motion noise signal generated by the running speed change of the detector; A hidden layer of the pipeline anomaly aware LSTM model introduces a gating confidence mechanism, the hidden layer being configured to output a prediction signal and a confidence based on the timing characteristics; the output layer of the pipeline anomaly perception LSTM model is provided with a residual error detection unit, wherein the residual error detection unit is configured to calculate the difference value between a predicted signal and a corresponding pipeline signal to generate a residual error sequence; a second acquisition module configured to acquire real-time pipeline signals and perform the preprocessing operation; The input module is configured to input the real-time pipeline signal after the preprocessing operation into the target pipeline anomaly perception LSTM model to obtain the judgment parameters of the real-time pipeline signal, wherein the judgment parameters comprise a prediction signal, a residual sequence and a confidence coefficient; The first determining module is configured to determine an abnormal signal according to the judging parameter, wherein the abnormal signal comprises an invalid signal, a weak signal and a jump signal; and a second determining module configured to determine a pipeline space position corresponding to the abnormal signal based on the mapping information.

Description

Data analysis method and system based on detector in pipeline Technical Field The application relates to the technical field of data processing, in particular to a data analysis method and system based on an in-pipeline detector. Background The pipeline is used as an important carrier for energy transportation, chemical raw material transportation and the like, and along with the increase of the service life of the pipeline and the complexity and the change of the running environment, the interior of the pipeline is extremely easy to generate defects such as cracks, corrosion and the like, and if the defects are not found and treated in time, the safe running of the pipeline is seriously threatened, and even serious safety accidents are caused. Therefore, the regular and accurate safety detection of the pipeline is particularly urgent, and the pipeline detector becomes a key means for guaranteeing the safe operation of the pipeline by virtue of the advantages of high efficiency, accuracy and the like. In the actual detection process, not only various defects in the pipeline need to be accurately detected, but also signals acquired in the detection process need to be accurately analyzed to obtain a reliable detection result. At present, common pipeline detection methods are rich and various, wherein the magnetic leakage method judges the existence and degree of defects by detecting the leakage condition of a magnetic field at the defect position of the pipeline, the eddy current method detects the defects according to the change of eddy current in the pipeline by utilizing an electromagnetic induction principle, and meanwhile, the travelling distance of a detector in the pipeline is measured by means of a mileage wheel so as to assist in positioning the defect position. In addition, in order to obtain more comprehensive detection information, multiple sensors are generally used to collect data simultaneously. However, the data lacks an effective fusion means, and is difficult to deeply integrate the data acquired by various sensors, so that the data information cannot be fully mined and utilized, and the comprehensiveness and accuracy of the detection result are affected. In the actual detection signal, invalid signals, weak signals and jump signals are mixed, so that the signals are difficult to accurately judge, and especially in the aspect of jump signal detection, the jump signals which are small in quantity and difficult to detect are difficult to detect at present. In addition, the existing abnormal signal detection method is low in efficiency, cannot automatically identify and process complex abnormal signals, is easily interfered by human factors, and causes inaccurate data analysis, so that the accuracy and reliability of detection results are seriously affected. Disclosure of Invention The application provides a data analysis method and a system based on an in-pipeline detector, which are used for solving the technical problems that the existing pipeline detection method is low in efficiency and cannot automatically identify and process complex abnormal signals. The first aspect of the application provides a data analysis method based on an in-pipeline detector, comprising the following steps: The method comprises the steps of obtaining historical pipeline signals, wherein the pipeline signals comprise magnetic flux leakage signals, eddy current signals, stress signals and mileage wheel signals, the mileage wheel signals comprise mapping information of time indexes and corresponding mileage numbers of pipeline detection, and the mapping information is characterized by a corresponding relation between pipeline signal acquisition time and the mileage numbers of the pipeline to be detected; The pretreatment operation comprises time synchronization processing, improved normalization processing, adaptive signal drift removal processing, defect characteristic protection type noise filtering and multi-threshold local peak value holding processing which are sequentially executed; Training the pipeline anomaly perception LSTM model by utilizing the history pipeline signals after the preprocessing operation to obtain a target pipeline anomaly perception LSTM model; Acquiring a real-time pipeline signal and executing the preprocessing operation; Inputting the real-time pipeline signal after the preprocessing operation into the target pipeline anomaly perception LSTM model to obtain the judgment parameters of the real-time pipeline signal, wherein the judgment parameters comprise a predicted signal, a residual sequence and a confidence coefficient; Determining an abnormal signal according to the judging parameter, wherein the abnormal signal comprises an invalid signal, a weak signal and a jump signal; and determining the pipeline space position corresponding to the abnormal signal based on the mapping information. In some embodiments, the pipeline anomaly aware LSTM model is improved by an LSTM network model,