Search

CN-121997193-A - Electromagnetic interference fault prediction method integrating mixed loss and gating LoRA

CN121997193ACN 121997193 ACN121997193 ACN 121997193ACN-121997193-A

Abstract

The invention provides an electromagnetic interference fault prediction method for fusing mixed loss and a gating LoRA. According to the method, a gating vector is dynamically generated by constructing a double feedback signal fusing the internal uncertainty and the external knowledge correctness of the model, and the vector performs fine adjustment on rank dimension level on a low rank matrix of LoRA. The mechanism changes the knowledge injection process from global and indiscriminate updating to an adaptive process which is independently controlled according to knowledge importance and model cognitive state, so that key fault characteristics can be subjected to key learning, and the original capability of the model is prevented from being damaged. It can be seen that the invention integrates the advantages of model uncertainty evaluation and parameter efficient fine adjustment, and forms a new electromagnetic interference fault prediction model capable of remarkably improving prediction accuracy, robustness and interpretability.

Inventors

  • YANG SHUNKUN
  • ZHANG YAOXING

Assignees

  • 北京航空航天大学

Dates

Publication Date
20260508
Application Date
20260106

Claims (10)

  1. 1. The electromagnetic interference fault prediction method integrating the mixed loss and the gating LoRA is characterized by comprising the following steps of: Step 1, constructing an electromagnetic interference fault data set and carrying out linguistic preprocessing; step 2, initializing a large language model and integrating a low-rank adaptation LoRA module; step 3, constructing a mixed loss function of uncertainty and knowledge correctness of the fusion model ; Step 4, designing a rank dimension gating vector generation mechanism based on mixing loss; Step 5, performing self-adaptive fine tuning training to optimize model parameters; and 6, performing electromagnetic interference fault prediction by using the trimmed model.
  2. 2. The method for predicting the electromagnetic interference fault of the fused hybrid loss and gating LoRA according to claim 1 is characterized by comprising the steps of converting multisource heterogeneous system operation data into a natural language text format which can be understood and processed by a large language model, wherein raw data comprise a timestamp, a bus message, a sensor voltage/current reading, an error frame count and a system log text, converting the structured or semi-structured data items into descriptive sentences with definite semantics according to a preset template in a preprocessing process, and finally, pairing the processed text sequences with corresponding expert labeling fault labels to form a < promt, label > dataset for fine adjustment of the model.
  3. 3. The method for predicting electromagnetic interference failure by integrating mixed loss and gating LoRA as set forth in claim 1 or 2, wherein in step 2, a pre-trained large language model is selected as a basic model, a low-rank adaptation LoRA module is integrated in a specific layer of the model for realizing efficient fine tuning of parameters, and the specific layer includes a query of a self-attention mechanism And key Matrix, module in original pre-training weight matrix By-pass, a low-rank decomposition matrix is connected in parallel And Is a trainable low rank matrix, wherein, Is of the dimension of , Is of the dimension of , Is of the dimension of Rank of Far smaller than When the model propagates forward, the output of the model corresponding layer Calculated from the following formula, wherein, Is the input feature vector of the model corresponding layer: ; in the fine tuning process, the original weights Keep frozen and only update the parameters far less Is a matrix of (a) And 。
  4. 4. The method for predicting the electromagnetic interference failure of the hybrid loss and gating LoRA as recited in claim 1, wherein the knowledge accuracy loss Adopts standard cross entropy loss function to measure model prediction result The consistency with the real label y is calculated by the following formula: ; Wherein, the Is the total number of fault categories; one-hot encoding vector for real label The bit element(s), Predicting samples for model as belonging to A probability value for a class, wherein, The loss term ensures that the model is optimized towards learning the correct domain knowledge; Model uncertainty loss The method comprises the steps of quantifying uncertainty of a model to current input by adopting entropy of predictive probability distribution, judging the possibility of multiple categories by a high entropy representation model to be more average, namely, high uncertainty, and calculating the uncertainty according to the following formula: ; Wherein, the Is the total number of fault categories; predicting samples for model as belonging to A probability value for a class, wherein, The uncertainty loss term reflects the cognitive state of the model on the knowledge system.
  5. 5. The method for predicting an electromagnetic interference failure in combination with a hybrid loss and gating LoRA as set forth in claim 4, wherein the hybrid loss function is Is a weighted sum of model uncertainty and knowledge correctness: ; Wherein the super parameter For balancing the relative importance of the two penalty terms.
  6. 6. The method for predicting electromagnetic interference failure of fusion mixed loss and gating LoRA as set forth in claim 1, wherein in step 4, the gating vector is The generation process of (a) is as follows: ; Wherein, the Is a small multi-layer perceptron which loses scalar values Map to one A dimension vector; is a Sigmoid activation function that outputs each element of the vector Is constrained within the interval (0, 1), wherein, For an index of LoRA module rank dimensions, Each element of (3) The value of the rank corresponding to the LoRA bypass matrix represents the importance of the dimension in the current learning task or the necessity of updating, the value of the rank being close to 1 indicates that the dimension needs to be fully utilized and updated, the value of the rank being close to 0 indicates that the dimension should be restrained, and the original capability of the model is protected or invalid updating is avoided.
  7. 7. The method for predicting an electromagnetic interference failure with mixed loss and gating LoRA as set forth in claim 1, wherein in step 5, the generated gating vector is Applied to the LoRA updating process, the self-adaptive fine adjustment is realized, in particular, the original LoRA forward propagation formula is modified, and the gating vector is controlled Embedded in a diagonal matrix: 。
  8. 8. The method for predicting the electromagnetic interference failure of the fusion hybrid loss and gating LoRA according to claim 7, wherein the method comprises the following steps: Is to vector Conversion to principal diagonal elements Is a diagonal matrix of (a); this operation corresponds to pair LoRA The individual rank dimensions are independently scaled or switch controlled, and the gradient will be based on the back propagation and parameter update Calculation is performed but propagation path and pair of gradients The influence of the matrix is affected by the diagonal matrix The model generates a gating vector which allows more information flow to pass through when facing samples with high uncertainty and wrong prediction, thereby enhancing the learning of new knowledge, otherwise, the model can inhibit updating and keep stable.
  9. 9. The electromagnetic interference fault prediction method based on the fusion mixing loss and gating LoRA of claim 1 is characterized in that in step 6, after multiple rounds of adaptive fine tuning training, models containing optimized LoRA parameter matrixes A and B are deployed, in actual application, real-time linguistic processing is carried out on system operation data collected on site to generate input text, the text is input into the fine-tuned models, and the models directly output fault type prediction results on the current system state.
  10. 10. The method for predicting an electromagnetic interference failure of a hybrid loss and gating LoRA as recited in claim 9, wherein the adaptive trimming process includes the sub-steps of, for each batch of training: (a) Forward propagation and loss calculation, namely, for each sample in a batch, the model firstly carries out standard forward propagation once to calculate the prediction probability And then according to And a genuine label Calculating the mixing loss ; (B) Gating vector generation, will Inputting the data into an MLP network to generate gating vectors corresponding to batches or samples ; (C) Gradient calculation with gating using the gating forward propagation formula applied to LoRA: ; taking a standard training target as an optimization target, performing back propagation, and calculating the gradient of the matrix A and the matrix B And Due to The calculation of (2) depends on While Itself again depending on the model output, to simplify the computation, the gradients are passed back to A and B Regarding as a fixed constant, making it not participate in the calculation of the gradient chain, thus being only used as a weight factor for adjusting the gradient magnitude; (d) Parameter updating based on calculated gradients using an optimizer And Parameters a and B of the LoRA module are updated.

Description

Electromagnetic interference fault prediction method integrating mixed loss and gating LoRA Technical Field The invention belongs to the field of artificial intelligence and electromagnetic compatibility fault prediction intersection, and particularly relates to an electromagnetic interference fault prediction method integrating mixed loss and gating LoRA. According to the method, the mixed loss of the internal uncertainty and the external correctness signals of the fusion model is constructed, so that the low-rank adaptive weight is driven to carry out the gating adjustment of the rank dimension level, and the capability of the model for accurately predicting and explaining potential faults in a complex electromagnetic environment is improved. Background As electronic devices become more complex in functionality and more integrated, electromagnetic interference (EMI) has become one of the major constraints affecting their reliability and safety. In critical systems such as aerospace, rail transit, smart grids, autopilot and the like, external electromagnetic environments or internal circuit crosstalk may invade electronic devices through conduction and radiation paths, resulting in data transmission errors, logic dysfunction and even hardware damage and other faults. Therefore, developing a technology capable of accurately predicting potential faults caused by electromagnetic interference has important engineering value for guaranteeing stable operation of a key system. Currently, the technical solutions for electromagnetic interference fault prediction mainly include the following three types: The first is a simulation method based on a physical model. The method needs to establish an accurate electromagnetic model of electronic components, circuits and the whole machine in the system, and then carries out simulation analysis through numerical calculation tools such as a finite element method and the like. The advantage of this approach is that the physical mechanism of the fault can be revealed. The method has the defects that 1) the modeling process is complex, the modeling process is particularly difficult for a large system with a complex structure, 2) the calculation resource consumption is huge, the time consumption is too long, the method is not suitable for real-time prediction, and 3) the simulation result is highly dependent on the accuracy of model parameters, and the influence of the change of working conditions and the uncertainty of environment in actual operation is difficult to reflect. The second category is prediction methods based on traditional machine learning. The method utilizes algorithms such as a Support Vector Machine (SVM), random forest and the like to analyze collected electromagnetic signals, bus data and the like so as to identify a fault mode. Compared with physical simulation, the method can process actual operation data, and is higher in calculation efficiency. The method has the main limitations that 1) the performance is seriously dependent on the characteristics of manual design, the quality of characteristic extraction determines the upper limit of the performance of the model, and 2) the generalization capability of the model is limited, and when the running environment or the fault mode is changed, the prediction performance is obviously reduced. The third class is prediction methods based on deep learning. The method adopts a depth model such as a cyclic neural network (RNN) and a long and short time memory network (LSTM) and the like, and can automatically extract the characteristics from the original time sequence data. The method has the advantage of being capable of processing high-dimensional and nonlinear data without complex characteristic engineering. However, these methods have inherent drawbacks in that 1) the model is poorly interpretable, its prediction result is often regarded as a "black box" and cannot provide basis for fault tracing, and 2) the model performs poorly when dealing with weak correlation signals with high randomness and burstiness caused by electromagnetic interference, and it is difficult to capture implicit causality, resulting in insufficient prediction accuracy in the face of new or rare disturbances. To overcome the limitations of the above approach, researchers have begun exploring the application of large language models for fault prediction. The large language model has strong context understanding, logic reasoning and knowledge integration capability, and can theoretically realize deeper fault prediction by analyzing text data such as system logs, monitoring reports and the like and fusing domain knowledge. However, it is a core technical challenge to efficiently and accurately integrate the expertise of the electromagnetic field into the generic LLM. Existing efficient fine-tuning techniques (e.g., low-rank adaptation, loRA) typically employ a globally uniform update strategy when performing knowledge injection, i.e., applyin