CN-122023893-A - Structural health monitoring abnormal data diagnosis method based on multi-task mixed expert vision transducer

CN122023893ACN 122023893 ACN122023893 ACN 122023893ACN-122023893-A

Abstract

The invention discloses a structural health monitoring abnormal data diagnosis method based on a multi-task mixed expert vision transducer. The method comprises the steps of converting a one-dimensional monitoring time sequence of a plurality of bridges into two-dimensional images, obtaining a Token sequence through patch embedding, superposing position codes, constructing an MT-MoE ViT shared trunk, introducing a task perception dynamic gating route into a mixed expert attention module, embedding and fusing Token tokens and bridge tasks, realizing expert dynamic allocation by adopting Gumbel-Softmax and Top-k sparse activation, completing expert feature extraction and weighted fusion, constructing a combined target comprising classification Loss, load balance Loss and Z-Loss in a training stage, adopting GradNorm self-adaptive balance multitask gradient conflict and training imbalance, and outputting an abnormal class judgment result corresponding to the bridge through a task specific classification head in an reasoning stage. According to the invention, multi-bridge collaborative anomaly diagnosis is realized under a single model frame, the bridge crossing sharing and bridge difference are considered, the multi-task training stability and expert load balancing capability are improved, the engineering deployment and maintenance cost is reduced, and the engineering applicability is good.

Inventors

BAO YUEQUAN
PAN QIUYUE
Long Feiyuan

Assignees

哈尔滨工业大学

Dates

Publication Date: 20260512
Application Date: 20260123

Claims (10)

1. A structural health monitoring anomaly data diagnosis method based on a multitasking hybrid expert vision transducer, characterized in that the diagnosis method comprises the following steps: Firstly, acquiring one-dimensional structure health monitoring time sequence data of a plurality of bridges, converting the time sequence data into two-dimensional image samples, defining each bridge as an independent subtask by taking 'bridge' as task granularity, and constructing a multi-task data set; step two, constructing a shared backbone network based on a visual transducer, introducing a mixed expert attention module and a feedforward neural network module into a transducer block to form an MT-MoEViT network, and extracting common characteristics and difference characteristics of a bridge-crossing task in a shared characteristic space; step three, in the mixed expert attention module introduced in the step two, embedding and vector fusion of Token characterization and bridge task numbers, calculating expert route scores through a gate control network, and realizing micro sparse sampling by adopting Gumbel-Softmax strategy to obtain expert selection probability distribution; Step four, according to the probability distribution selected by the expert in the step three, only the first k most relevant expert sub-networks are activated by adopting a Top-k strategy to generate Query vector Query expression, and Key vector Key and Value vector Value are shared among all the experts; step five, constructing a joint optimization objective function comprising classification Loss, load balancing Loss and auxiliary regularization Loss Z-Loss, introducing GradNorm algorithm to dynamically adjust the Loss weight of each bridge task so as to balance the optimization rate among different tasks, and simultaneously restraining the expert activation frequency by using the load balancing Loss so as to prevent the expert from collapsing; Step six, setting task specific classification heads for each bridge subtask, inputting the characteristics extracted by the deep network into the task specific classification heads matched with the corresponding bridge tasks, outputting the abnormal class prediction probability of the bridge tasks, giving out an abnormal class judgment result, and realizing multi-bridge collaborative abnormality diagnosis.
2. The structural health monitoring anomaly data diagnosis method based on a multi-task hybrid expert vision transducer according to claim 1, wherein the image embedding module in the step one satisfies the following definition: Representing an input image as a tensor Using patch dimensions The patch number is obtained: First, the The patches are flattened into The Token is embedded as follows: superimposing a learnable position code The input Token is obtained as: In the formula, As the height of the image is to be taken, For the width of the image to be the same, The number of image channels; The number of patches; in the form of a linear projection matrix, Is a bias term; is the first Position coding vectors corresponding to the patches; for adding a position-coded Token representation.
3. The structural health monitoring anomaly data diagnosis method based on a multi-task hybrid expert vision transducer according to claim 1, wherein the task aware gating route in the third step satisfies the following definition: gating network pair Number of Token Outputting expert score vectors and task numbers Task bias vectors are obtained through a task embedded network, and the task bias vectors are fused to obtain a route score And adopts Gumbel-Softmax to obtain expert selection probability In the formula, Is the first Expert routing logits for each Token; gating mapping for Token to expert scores; is the first Task embedding vectors of the individual tasks; biasing the mapping for the task; is the first The Token selects the th Probability of individual expert; Is the expert number; random noise sampled from a standard gummel distribution; Is a temperature coefficient.
4. The structural health monitoring abnormal data diagnosis method based on the multi-task mixed expert vision transducer according to claim 1, wherein the attention calculation of "expert Query independent, key/Value sharing" in the fourth step satisfies the following: First, the The Query generated by each expert is: the shared Key and Value are: In the formula, Inputting a Token sequence matrix; is the first A Query mapping matrix independent of each expert; key mapping weights shared for all experts; Value mapping weights shared for all experts; is expert number.
5. The structural health monitoring abnormal data diagnosis method based on the multi-task mixed expert vision transducer according to claim 4, wherein the Top-k sparse activation and weighted fusion in the fourth step satisfies the following definition: For the first The Token only selects the front with the highest probability The individual experts form an active set And normalize intra-set probabilities to Fusion Query is: and calculating the attention weight and output by using multiple head attention In the formula, Is the first Expert pair 1 A Query vector generated by the Token; the matrix is composed of Token fusion Query; feature dimensions for a single attention header; Is a transposition operation; is an attention weight matrix; is the attention output.
6. The structural health monitoring anomaly data diagnosis method based on a multi-task hybrid expert vision transducer according to claim 1, wherein the multi-task joint loss function in the fifth step satisfies the following definition: First, the The cross entropy classification loss of the individual tasks is: GradNorm calculate the first Weighted gradient norms for individual tasks: The load balancing loss is as follows: Z-Loss is: The total loss is: In the formula, Is the first A number of task samples; is the first A number of task anomaly categories; is a real one-hot label; Is a predictive probability; is a shared parameter set; is a dynamic task weight; Is the L2 norm; is the first The desired activation frequency of the individual expert; is the first The actual activation frequency of the individual expert; And (3) with Is a super parameter weight coefficient.
7. The method for diagnosing structural health monitoring abnormal data based on a multi-task mixed expert vision transducer as set forth in claim 1, wherein the task specific classification head in the step six satisfies the following conditions The classified prediction output of each bridge task is as follows: And by combining Obtaining an abnormal category judgment result; In the formula, Is the first A classification head for each task; Task feature vectors input to the classification head; is the first Category probability distribution for individual tasks; Is a predictive category.
8. A structural health monitoring anomaly data diagnostic system based on a multiplexed hybrid expert visual transducer, wherein the diagnostic system uses the structural health monitoring anomaly data diagnostic method based on a multiplexed hybrid expert visual transducer according to any one of claims 1 to 7, the diagnostic system comprising the following six steps: the data preprocessing and sequence embedding module is used for acquiring one-dimensional structure health monitoring time sequence data of a plurality of bridges, converting the time sequence data into two-dimensional image samples, defining each bridge as an independent subtask by taking 'bridge' as task granularity, and constructing a multi-task data set; the multi-task mixed expert network architecture construction module is used for constructing a shared backbone network based on a visual transducer, introducing a mixed expert attention module and a feedforward neural network module into the transducer block to form an MT-MoEViT network, and extracting common characteristics and difference characteristics of a cross-bridge task in a shared characteristic space; In the mixed expert attention module introduced in the step two, token characterization and bridge task number embedding vectors are fused, expert route scores are calculated through a gating network, and micro sparse sampling is realized by adopting Gumbel-Softmax strategy to obtain expert selection probability distribution; The expert feature extraction and weighted fusion module is used for activating only the first k most relevant expert sub-networks to generate Query vector Query representation by adopting a Top-k strategy according to probability distribution selected by the expert in the step three, and sharing Key vector Key and Value vector Value among all the experts; Constructing a joint optimization objective function comprising classification Loss, load balancing Loss and auxiliary regularization Loss Z-Loss, introducing GradNorm algorithm to dynamically adjust the Loss weight of each bridge task so as to balance the optimization rate among different tasks, and restraining the expert activation frequency by using the load balancing Loss so as to prevent the expert from collapsing; And the abnormality diagnosis decision output module is used for setting a task specific classification head for each bridge subtask, inputting the characteristics extracted by the deep network into the task specific classification head matched with the corresponding bridge task, outputting the abnormality class prediction probability of the bridge task, and giving an abnormality class judgment result to realize multi-bridge collaborative abnormality diagnosis.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method according to any of claims 1-7 when executing the computer program.
10. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a computer program which, when executed by a processor, implements the method according to any of claims 1-7.

Description

Structural health monitoring abnormal data diagnosis method based on multi-task mixed expert vision transducer Technical Field The invention belongs to the technical field of deep learning, abnormal data diagnosis and structural health monitoring, and particularly relates to a structural health monitoring abnormal data diagnosis method based on a multi-task mixed expert vision transducer. Background With the rapid development of traffic infrastructure construction, centralized management of regional or urban-level bridge clusters has become an important trend in the field of structural health monitoring. The monitoring center often needs to monitor tens or even hundreds of bridges at the same time to ensure the operation safety of the traffic network. The structural health monitoring system diagnoses and evaluates the abnormal state of the bridge by collecting sensor data such as acceleration, displacement, strain and the like for a long time. However, existing anomaly data diagnostic methods face serious challenges when faced with a collaborative monitoring scenario for large-scale bridge clusters. In the long-term service process of civil engineering infrastructures such as large bridges, a monitoring system is usually required to continuously collect multi-source response data to support tasks such as structural state evaluation, damage early warning and long-term performance evaluation. However, due to the influence of factors such as sensor aging and drift, acquisition equipment failure, packet loss and jitter of a communication link, external electromagnetic interference, strong time variability of environment and traffic load and the like, noise pollution, abnormal paragraphs, distribution drift, label confusion and the like are inevitably generated in a monitoring sequence, so that the reliability of a subsequent diagnosis conclusion is weakened. Aiming at the structural health monitoring abnormal data diagnosis problem, the prior art generally adopts ideas such as threshold judgment, statistical feature analysis, traditional machine learning classification method and the like, and also has research and introduction of a deep learning model to convert one-dimensional time sequence signals into two-dimensional images and then carry out end-to-end feature learning and abnormal recognition. The method has certain effectiveness in a scene of relatively stable single bridge or data distribution, but under the background of collaborative monitoring of bridge groups, different bridges have obvious differences in structural system, dynamic characteristics, sensor layout and operation working conditions, so that the monitored data distribution and an abnormal mode have obvious heterogeneity. Meanwhile, abnormal class spaces of all bridges are often inconsistent, and unified output spaces are difficult to directly share. If independent models are respectively trained on each bridge, the number of models is large, the training and maintenance cost is high, the engineering deployment is complex, and if migration learning or unified single models are adopted to be generalized forcefully, the phenomenon of unstable migration or negative migration is easy to occur, and stable recognition performance is difficult to maintain on multi-bridge tasks. The multi-task learning can display sharing and differences among modeling tasks, extract cross-bridge commonality features through sharing representation, and reserve bridge differences by means of task specific modules. However, in the multi-bridge anomaly diagnosis, the key bottleneck still exists that firstly, the sample size, the class distribution and the difficulty difference of each task are obvious, gradient conflict and training unbalance are easily caused by direct joint optimization, so that a model is biased to a task with larger data quantity, and secondly, after a dynamic routing mechanism such as a mixed expert is introduced, expert load unevenness and route collapse can occur, so that model capacity waste and performance degradation are caused. Therefore, there is a need for a method for diagnosing abnormal data for collaborative monitoring of bridge clusters, which combines the cross-bridge sharing and individuation differences under a unified framework, adapts to inconsistent class space, and improves the availability and deployment efficiency of engineering. Disclosure of Invention The invention aims to provide a structural health monitoring abnormal data diagnosis method based on a multi-task mixed expert vision transducer, which is used for solving the problems that the existing single-bridge/single-task abnormal diagnosis model is difficult to consider the structural heterogeneity, the abnormal type space inconsistency, the high cross-bridge deployment cost and the like of a plurality of bridges, and aims to realize collaborative learning and high-precision recognition of multi-bridge abnormal modes under a unified network frame and simultaneously i