CN-121997141-A - Method and system for detecting quality of resistance spot welding based on physical guidance multi-mode

CN121997141ACN 121997141 ACN121997141 ACN 121997141ACN-121997141-A

Abstract

The invention relates to a resistance spot welding quality detection method and system based on physical guidance multi-mode. The method comprises the steps of firstly obtaining visual mode and time sequence mode multi-sensor data of the resistance spot welding and completing preprocessing, then constructing a multi-view image encoder set to extract visual mode data features, fusing to obtain unified visual features, meanwhile, processing the preprocessed time sequence mode data through a time sequence encoder to generate a time sequence feature sequence, constructing a hierarchical attention aggregation network, taking the unified visual features as query vectors, taking the time sequence feature sequence as key vectors and value vectors to input the key vectors and the value vectors to obtain physically guided multi-mode fusion features, finally analyzing the fusion features through a multi-task prediction network, and synchronously outputting welding quality classification results and physical performance regression prediction values of the resistance spot welding. Compared with the prior art, the invention has the advantages of obviously improving the accuracy and reliability of the resistance spot welding quality detection and the like.

Inventors

FENG QIAOBO
Xie Detian

Assignees

上海电力大学

Dates

Publication Date: 20260508
Application Date: 20260130

Claims (10)

1. A resistance spot welding quality detection method based on physical guidance multi-mode is characterized by comprising the following steps: s1, acquiring and preprocessing multi-mode sensor data of resistance spot welding to be detected, wherein the multi-mode sensor data comprises visual mode data and time sequence mode data; S2, constructing a multi-view image encoder set, inputting visual mode data into the multi-view image encoder set, outputting a feature extraction result, and fusing the feature extraction result so as to generate unified visual features; s3, constructing a time sequence encoder, inputting the preprocessed time sequence modal data into the time sequence encoder, and outputting a time sequence characteristic sequence; S4, constructing a hierarchical attention aggregation network, namely taking the unified visual features as query vectors, taking the time sequence feature sequences as key vectors and value vectors, inputting the hierarchical attention aggregation network, and outputting physically guided multi-mode fusion features; S5, processing the multi-mode fusion characteristics of the physical guidance by utilizing a multi-task prediction network, and outputting welding quality classification results and physical property regression prediction values of the resistance spot welding to serve as resistance spot welding quality detection results.
2. The method for detecting the quality of resistance spot welding based on the physical guidance multi-mode according to claim 1, wherein the multi-view image encoder set, the time sequence encoder, the hierarchical attention aggregation network and the multi-task prediction network together form a prediction model; the method further comprises a model training process, wherein during training, a training data set is established by adopting sample multi-mode sensor data, the training data set is used as input to execute steps S1-S5, the end-to-end training is carried out on the prediction model by utilizing a mixed loss function, a trained prediction model is obtained, during prediction, data to be detected are input into the trained prediction model, and a welding quality classification result and a physical performance regression prediction value are output.
3. The method for detecting the quality of the resistance spot welding based on the physical guidance multi-mode according to claim 2, wherein the mixed loss function comprises classification loss, regression loss and invariance regularization loss; The classification loss adopts focus loss, the weight of a sample easy to classify is reduced through a focusing parameter gamma, the difficult-to-separate sample including cold welding or splashing is focused, and the regression loss adopts mean square error loss and is used for monitoring the prediction of nugget diameter and tensile shear force.
4. The method for detecting quality of resistance spot welding based on physical guidance multi-mode according to claim 3, wherein when the training data set performs step S1, the preprocessing further comprises data balance processing of the training data set based on a mixed resampling strategy, and the specific process is that the number of samples of each category in the training data set is counted by using a synthetic minority oversampling technology of a self-adaptive neighborhood parameter, the number of samples of the least category is determined, a K nearest neighbor parameter is set according to the number of samples of the least category, and then a new synthetic sample is generated in a feature space through linear interpolation according to the K nearest neighbor parameter.
5. The method for detecting the quality of the resistance spot welding based on the physical guidance multi-mode according to claim 1, wherein the visual mode data comprise an infrared thermal imaging image, a front RGB image of a welding spot and a back RGB image of the welding spot, and the time sequence mode data comprise current, voltage, pressure, electrode displacement, welding time and plate thickness parameters in the welding process.
6. The method of claim 5, wherein the preprocessing includes resizing and normalizing the image data and Z-Score normalizing the time-series parameters.
7. The method for detecting the quality of the resistance spot welding based on the physical guidance multi-mode according to claim 6, wherein the multi-view image encoder set in the step S2 comprises three pre-training multi-view image encoders with independent parameters, wherein the three pre-training multi-view image encoders are respectively used for extracting an infrared thermal imaging image, a welding spot front RGB image and a welding spot back RGB image; The feature extraction result in S2 comprises an infrared feature, a front feature and a back feature; the step S2 of fusing the feature extraction results to generate unified visual features comprises the following specific processes: processing the infrared features, the front features and the back features using a three-way parallel inter-view cyclic cross-attention mechanism, the three ways comprising: The infrared main guide way takes infrared characteristics as query vectors, and the concatenation of front characteristics and back characteristics is taken as key vectors and value vectors; The front main guide way takes the front characteristics as a query vector, and the concatenation of the infrared characteristics and the back characteristics as a key vector and a value vector; The back main guide way takes the back characteristics as a query vector, and the concatenation of the front characteristics and the infrared characteristics as a key vector and a value vector; and finally, splicing the three paths of outputs, and fusing the spliced results through a full-connection layer to generate unified visual characteristics containing heat distribution and surface morphology complementary information.
8. The method for detecting quality of resistance spot welding based on physical guidance multi-mode according to claim 1, wherein in the step S3, the specific process of generating the time sequence feature sequence comprises the steps of extracting bidirectional dynamic features of time sequence mode data in a time sequence encoder by utilizing a bidirectional long-short-term memory network, injecting sine position codes into an extraction result as time position information, and obtaining the injected sequence as the time sequence feature sequence.
9. The method for detecting the quality of the resistance spot welding based on the physical guidance multi-mode according to claim 1, wherein the hierarchical attention aggregation network in the step S4 specifically performs the following operations of constructing an asymmetric attention mechanism, taking the unified visual feature as a query vector, taking the time sequence feature sequence as a key vector and a value vector, mapping the query vector, the key vector and the value vector to a plurality of attention subspaces respectively through a linear projection matrix, calculating the attention weight of the unified visual feature and each time feature in the time sequence in each subspace, carrying out weighted aggregation on the time sequence feature according to the attention weight, and splicing output results of a plurality of attention heads to obtain the multi-mode fusion feature of the physical guidance.
10. A resistance spot welding quality detection system based on physical guidance multi-mode, which is characterized in that the system works by applying the resistance spot welding quality detection method based on physical guidance multi-mode according to any one of claims 1-9, and comprises a data acquisition module, a data preprocessing module, a visual characteristic processing module, a time sequence characteristic processing module, a physical guidance fusion module and a multi-task prediction module; The data acquisition module is used for acquiring multi-mode sensor data of the resistance spot welding; The data preprocessing module is used for preprocessing the multi-mode sensor data; The visual characteristic processing module comprises a multi-view image encoder group and is used for processing visual mode data, outputting characteristic extraction results and fusing the characteristic extraction results to generate unified visual characteristics; The time sequence feature processing module comprises a time sequence encoder which is used for processing the preprocessed time sequence modal data and outputting a time sequence feature sequence; The physical guidance fusion module comprises a hierarchical attention aggregation network, a multi-mode fusion module and a physical guidance fusion module, wherein the hierarchical attention aggregation network is used for processing a unified visual characteristic as a query vector, a time sequence characteristic sequence as a key vector and a value vector and outputting a multi-mode fusion characteristic of physical guidance; The multi-task prediction module comprises a multi-task prediction network and is used for processing the multi-mode fusion characteristics of the physical guidance and outputting welding quality classification results and physical performance regression prediction values of the resistance spot welding.

Description

Method and system for detecting quality of resistance spot welding based on physical guidance multi-mode Technical Field The invention relates to the technical field of welding quality detection, in particular to a method and a system for detecting the quality of resistance spot welding based on physical guidance multi-mode. Background Resistance spot welding (RESISTANCE SPOT WELDING, RSW) is widely used in the automotive and aerospace fields. With the development of industry 4.0, various sensors are integrated on the production line to monitor welding quality. The existing quality detection methods are mainly divided into image-based detection and process signal-based detection. However, the prior art has the following limitations: 1. The limitation of a single mode is that surface defects such as splashing and the like are difficult to detect only by depending on process parameters, and the dynamic mechanism of nugget formation cannot be revealed only by depending on images. 2. Simple multi-mode splicing, wherein the existing multi-mode fusion method (such as simple splicing or symmetrical attention) omits the physical causal relationship that a welding process determines a welding result and a welding image is the final embodiment of the process, and has no interpretability. 3. The serious class imbalance is that the industrial field qualified samples are far more than the defect samples, so that the recognition capability of the model on rare defects (such as cold joint and burn-through) is poor, and the regression prediction precision of physical properties (such as shearing force) is difficult to cooperatively improve by a simple classification task. For example, an invention patent with publication number CN119927394a discloses a resistance spot welding quality detection method based on edge calculation, which constructs a dynamic resistance curve by acquiring voltage and current data, and performs quality prediction by using an embedded neural network. However, the method has the obvious limitations that the scheme only depends on single-mode time sequence signals such as voltage, current and the like, key visual information such as surface morphology, heat distribution and the like of welding spots cannot be obtained, surface defects such as splashing, burning-through and the like are difficult to detect, the scheme adopts a simple neural network structure, modeling capability of complex association relation among multi-mode data is lacking, physical guidance diagnosis logic caused by fruit trace cannot be realized, the scheme only carries out single quality classification task, cooperative optimization with physical performance regression prediction is not considered, a special processing mechanism for the problem of industrial data category imbalance is lacking, the system architecture is relatively simple, and the scheme lacks modularized design, so that the scheme is difficult to adapt to diversified requirements of complex industrial environments. In summary, the current resistance spot welding quality detection technology has the problems of single-mode information limitation, lack of a physically guided multi-mode fusion mechanism, insufficient class imbalance processing capacity and lack of multi-task collaborative prediction capacity. Disclosure of Invention The invention aims to overcome the defects of the prior art and provide a method and a system for detecting the quality of resistance spot welding based on physical guidance multi-mode. The aim of the invention can be achieved by the following technical scheme: According to one aspect of the invention, there is provided a method for detecting quality of resistance spot welding based on physical guidance multi-mode, comprising the steps of: S1, acquiring and preprocessing multi-mode sensor data of resistance spot welding to be detected, wherein the multi-mode sensor data comprises visual mode data and time sequence mode data; S2, constructing a multi-view image encoder set, inputting visual mode data into the multi-view image encoder set, outputting a feature extraction result, and fusing the feature extraction result so as to generate unified visual features; s3, constructing a time sequence encoder, inputting the preprocessed time sequence modal data into the time sequence encoder, and outputting a time sequence characteristic sequence; S4, constructing a hierarchical attention aggregation network, namely taking the unified visual features as query vectors, taking the time sequence feature sequences as key vectors and value vectors, inputting the hierarchical attention aggregation network, and outputting physically guided multi-mode fusion features; S5, processing the physically guided multi-mode fusion characteristics by utilizing a multi-task prediction network, and outputting a welding quality classification result and a physical property regression prediction value of the resistance spot welding to serve as a resistance spot weldi