CN-120236135-B - Multi-mode feature fusion-based multi-bridge monitoring data anomaly identification method and system

CN120236135BCN 120236135 BCN120236135 BCN 120236135BCN-120236135-B

Abstract

The invention discloses a multi-mode feature fusion-based multi-bridge monitoring data anomaly identification method and system, which relate to the field of data anomaly identification and comprise the following steps of firstly collecting SHM time sequence data of a plurality of bridges, carrying out normalization processing and segmentation, secondly visualizing the normalized time sequence data, simultaneously extracting statistical features in the data, manually marking an image and the statistical features, and constructing a dataset, thirdly establishing a multi-mode convolutional neural network model based on input of gray images and the statistical features, using the dataset from the bridges, simultaneously inputting the gray images and the corresponding statistical features into the network model for training, and fourthly using the trained convolutional neural network model for carrying out anomaly identification on the SHM data of a target bridge. The method realizes automatic identification of bridge SHM data anomalies, improves the identification precision of the anomalies under the conditions of unbalanced classification and limited data volume of bridge monitoring data sets, and provides basis for bridge operation state evaluation and early warning.

Inventors

HONG YU
ZHAO ZHONGKUI
PU QIANHUI
ZHOU TONG
LI SHENGYU

Assignees

西南交通大学

Dates

Publication Date: 20260512
Application Date: 20250320

Claims (9)

1. The multi-bridge monitoring data anomaly automatic identification method based on multi-mode feature fusion is characterized by comprising the following steps: S1, carrying out normalization processing on an original time sequence data set, and dividing the original time sequence data set into a series of segments with fixed length to obtain normalized time sequence data segments; s2, converting the obtained normalized time sequence data fragment into a gray image, extracting statistical features in the time sequence data, manually marking the image and the statistical features, and outputting a gray image dataset, a statistical feature dataset and marked normal/abnormal tags; S3, based on the output result of the step S2, establishing a multi-modal convolutional neural network model, and simultaneously inputting gray images and corresponding statistical features into the input end of the network model for training by using the data sets from a plurality of bridges to obtain a trained multi-modal deep learning model; s4, performing anomaly identification on the target bridge SHM data by using the trained multi-mode deep learning model; the step S3 specifically includes: The multi-mode convolutional neural network model comprises a convolutional layer, a pooling layer and a full-connection layer, wherein the overall structure is divided into three parts of an image encoder, a statistical feature encoder and a fusion classifier, the image encoder is based on a ResNet neural network, removes the last full-connection layer, receives gray image input with the size of 224 multiplied by 224, extracts image features in the gray image input, and outputs 512 image feature vectors; The training step of the multi-modal convolutional neural network model comprises the following steps: S301, manually marking abnormal mode labels on SHM data sets of a plurality of bridges, dividing the data sets into a training set and a verification set according to the proportion of 8:2, wherein the training set is mainly used for bridge data samples to be tested, and other bridge data samples are used for supplementing abnormal modes with fewer numbers; S302, training network weight parameters suitable for abnormal data identification based on the designed multi-mode convolutional neural network model, and adjusting super parameters according to training results to obtain an optimal training model.
2. The method for automatically identifying abnormal multi-bridge monitoring data based on multi-modal feature fusion according to claim 1, wherein before the step S1, the method further comprises: and collecting SHM time sequence data of a plurality of bridges, including acceleration, deflection, stress and temperature data, to form an original time sequence data set.
3. The method for automatically identifying abnormal multi-bridge monitoring data based on multi-modal feature fusion according to claim 1, wherein the step S2 comprises: S201, reading the segmented time sequence data, drawing each sample data into 224 multiplied by 224 gray level images, extracting statistical characteristics, and respectively storing the images and the statistical characteristics in png and npy formats; s202, manually marking the generated data image and the statistical feature sample with an abnormal mode label.
4. The method for automatically identifying abnormal multi-bridge monitoring data based on multi-mode feature fusion according to claim 1, wherein the statistical features comprise peak intensity, standard deviation, linearity and linear fitting slope; The peak intensity formula is: (1) in formula (1), D 0.9 is the distance between the upper and lower bounds of the data value where 90% of the data points in the sample are located, x max is the maximum value in the data, and x min is the minimum value in the data; The standard deviation formula is: (2) In equation (2), x i is the value of each data point, N is the total number of data points in the sample; the linearity formula is: (3) In the formula (3), max (|Δx i |) is the maximum value of the absolute value of the deviation between the smooth curve and the fitting straight line of all data points in the sample, and x max and x min are the maximum value and the minimum value in the sample respectively; The linear fit slope is the slope of the fit line obtained using least squares linear regression on all data points in the sample, and represents the extent of baseline shift, a characteristic index that is used to identify trend anomalies present in the data.
5. The multi-bridge monitoring data anomaly automatic identification method based on multi-mode feature fusion according to claim 1, wherein the ResNet neural network introduces a residual block structure, wherein the residual block consists of two convolution layers, and each convolution layer is followed by a batch normalization layer and a ReLU activation function; The purpose of batch normalization is to make the data distribution of each layer of input in the network more stable, accelerate the learning speed of the model, and the batch normalization formula is as follows: (4) In the formula (4), x i is an input sample, mu is a sample mean value, sigma is a sample variance, epsilon is a minimum value for preventing denominator from being zero, gamma and beta are scaling parameters and offset parameters, and the normalized data are subjected to linear transformation; the ReLU activation function has the characteristic of low calculation complexity, and can effectively alleviate the problems of gradient explosion and gradient disappearance possibly occurring in the neural network training, and the formula is as follows: (5)
6. The method for automatically identifying abnormal multi-bridge monitoring data based on multi-modal feature fusion according to claim 1, wherein the step S4 comprises: and converting the SHM data of the bridge to be detected into gray level images in a segmented mode, extracting 4 statistical features, inputting the data sample of the bridge to be detected into a trained multi-mode convolutional neural network model for identification and classification, and obtaining the abnormal mode to which the data sample belongs.
7. A multi-modal feature fusion-based multi-bridge monitoring data anomaly automatic identification system, characterized in that the system is applied to the method of any one of claims 1-6, the system comprising: The data collection module is used for collecting SHM time sequence data of a plurality of bridges, including acceleration, deflection, stress and temperature data, and forming an original time sequence data set; The data preprocessing module is used for carrying out normalization processing on the original time sequence data set and dividing the original time sequence data set into a series of segments with fixed length to obtain normalized time sequence data segments; The feature extraction and labeling module is used for converting the obtained normalized time sequence data fragments into gray images, extracting statistical features in the time sequence data, manually labeling the images and the statistical features, and outputting a gray image dataset, a statistical feature dataset and labeled normal/abnormal labels; The deep learning training module is used for establishing a multi-modal convolutional neural network model based on the output result of the characteristic extraction and labeling module, and inputting gray images and corresponding statistical characteristics into the network model for training by using a data set from a plurality of bridges to obtain a trained multi-modal deep learning model; And the anomaly detection module is used for carrying out anomaly identification on the target bridge SHM data by utilizing the trained multi-mode deep learning model.
8. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the multi-modal feature fusion-based multi-bridge monitoring data anomaly automatic identification method of any one of claims 1 to 6 when the program is executed by the processor.
9. A computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the program is executed by a processor, the method for automatically identifying abnormal multi-bridge monitoring data based on multi-mode feature fusion according to any one of claims 1 to 6 is implemented.

Description

Multi-mode feature fusion-based multi-bridge monitoring data anomaly identification method and system Technical Field The invention belongs to the technical field of data anomaly identification, and particularly relates to a multi-bridge monitoring data anomaly identification method and system based on multi-mode feature fusion. Background With the development of advanced sensing, internet of things, big data, artificial intelligence and other new generation information technologies, the structural health monitoring technology is widely applied to the structural monitoring and maintenance of civil engineering at present. The long-term monitoring will generate massive SHM data, and the monitoring data records rich performance evolution information in the bridge operation process, including structural load and response behavior rules in the conventional operation state, and rare load and response in special events such as typhoons, ship collisions, earthquakes, fires, traffic accidents and the like. By mining and analyzing the mass monitoring data collected by the SHM system, the safety and applicability of the bridge structure during the service life of the bridge structure can be evaluated. However, due to faults of the monitoring system, such as improper installation of sensors, quality damage, long-term use and the like, various abnormal interferences can occur to the monitoring data collected by the SHM system, and the data analysis result is seriously affected. Therefore, how to quickly and effectively implement the identification detection of abnormal data is a great challenge in the field of bridge health monitoring. Aiming at a plurality of data abnormal modes existing in bridge SHM data, training a convolutional neural network model to automatically identify and classify abnormal data is an effective method. The convolutional neural network has strong feature extraction capability, the prior art usually visualizes time sequence data and inputs the time sequence data into the convolutional neural network for training, and the problem of abnormal identification of the time sequence data is converted into the problem of image classification, however, the method is limited by the pixel size of an image, and the spatial information of the data can be lost or disturbed. In order to overcome the limitation, the fusion of multi-modal information is proposed as an effective solution, and the intrinsic characteristics of the data are more comprehensively represented by integrating the data from different modalities, so that the accuracy and the robustness of anomaly identification are remarkably improved. Meanwhile, the problem of serious data unbalance often exists in the SHM data of a single bridge, namely the number of abnormal categories in the data set is huge, the problem is reflected in that the sample size of normal data and abnormal data is unbalanced, the normal monitoring data is far more than that of the abnormal data, and meanwhile, the sample size of each abnormal mode in the abnormal data is unbalanced, and the problem of unbalance of the single bridge monitoring data often leads to confusion of recognition results among part of categories and influences the final recognition accuracy of the abnormal data. Disclosure of Invention The invention aims to solve the technical problems in the background art, and aims to provide a multi-bridge monitoring data anomaly identification method and a system based on multi-mode feature fusion, which aim to solve the problem of low partial anomaly data identification precision caused by unbalanced image-based anomaly data classification method and single-bridge monitoring data at present. In order to solve the technical problems, the technical scheme of the invention is as follows: A multi-bridge monitoring data anomaly automatic identification method based on multi-modal feature fusion, the method comprising: S1, carrying out normalization processing on an original time sequence data set, and dividing the original time sequence data set into a series of segments with fixed length to obtain normalized time sequence data segments; s2, converting the obtained normalized time sequence data fragment into a gray image, extracting statistical features in the time sequence data, manually marking the image and the statistical features, and outputting a gray image dataset, a statistical feature dataset and marked normal/abnormal tags; S3, based on the output result of the step S2, establishing a multi-modal convolutional neural network model, and simultaneously inputting gray images and corresponding statistical features into the network model for training by using a data set from a plurality of bridges to obtain a trained multi-modal deep learning model; S4, performing anomaly identification on the target bridge SHM data by using the trained multi-mode deep learning model. It can be understood that the phenomenon of unbalance exists in the SHM data of a single bridge