CN-121972847-A - Intelligent welding method based on deep learning and three-dimensional reconstruction

CN121972847ACN 121972847 ACN121972847 ACN 121972847ACN-121972847-A

Abstract

The invention relates to the technical field of intelligent manufacturing and industrial robots, in particular to an intelligent welding method based on deep learning and three-dimensional reconstruction. The method comprises the steps of obtaining point cloud data of a workpiece to be welded, identifying a welding seam area through preprocessing and a point cloud segmentation network, obtaining welding seam track information comprising welding seam space coordinates and welding gun postures, optimizing a robot motion track through a global optimization algorithm and a local obstacle avoidance algorithm, generating a planning track, obtaining the welding seam position information in real time in the welding process, filtering through a state estimation algorithm, obtaining a real-time welding seam position, comparing the real-time welding seam position with the planning track, obtaining position deviation, dynamically correcting, carrying out defect detection on a welding seam image, generating a quality assessment result through combining process parameter analysis and case matching, and carrying out closed-loop self-adaptive control based on the real-time welding seam position and the quality assessment result. The invention can realize high-precision automatic extraction of the weld joint track, optimal planning of the motion track, real-time accurate tracking of the weld joint position and intelligent evaluation of the welding quality.

Inventors

DAN BINBIN
XIONG LING
RONG ZHIJUN
DU LIPING
Feng Pengyun

Assignees

武汉科技大学

Dates

Publication Date: 20260505
Application Date: 20251229

Claims (10)

1. An intelligent welding method based on deep learning and three-dimensional reconstruction is characterized by comprising the following steps: Acquiring point cloud data of a workpiece to be welded, preprocessing the point cloud data, identifying a welding seam region through a point cloud segmentation network, and acquiring welding seam track information through a feature extraction network based on the welding seam region, wherein the welding seam track information comprises welding seam space coordinates and welding gun postures; optimizing the motion track of the welding robot through a global optimization algorithm and a local obstacle avoidance algorithm based on the weld track information, and generating a planning track through track smoothing; acquiring the position information of a welding line in real time in the welding process, filtering the position information through a state estimation algorithm to obtain the position of the welding line in real time, comparing the position of the welding line in real time with the planned track to obtain a position deviation, and dynamically correcting the motion track of the robot according to the position deviation; Performing defect detection on the weld image through an image recognition network, and generating a quality assessment result by combining process parameter analysis and case matching; and performing closed-loop self-adaptive control based on the real-time weld joint position and the quality evaluation result, and automatically adjusting welding parameters to form a complete closed loop with parameter adjustment from point cloud data acquisition, track planning, real-time tracking and quality detection.
2. The intelligent welding method based on deep learning and three-dimensional reconstruction according to claim 1, wherein obtaining point cloud data of a workpiece to be welded, preprocessing the point cloud data, comprises: Acquiring the point cloud data in a laser three-dimensional scanning and/or structured light projection mode; Removing noise points by adopting a statistical outlier filtering algorithm, setting a neighborhood point number and a standard deviation multiple threshold, judging points deviating from the mean value by more than the standard deviation as outliers, and removing the outliers; downsampling the point cloud by adopting voxel grid filtering, and setting voxel side length parameters; and estimating normal vectors of each point by adopting a principal component analysis method, and setting search radius parameters.
3. The intelligent welding method based on deep learning and three-dimensional reconstruction according to claim 2, wherein the identifying of the weld region by the point cloud segmentation network, the obtaining of the weld trajectory information by the feature extraction network based on the weld region, comprises: the point cloud segmentation network adopts a layering point cloud processing network comprising a multi-scale feature extraction module and a feature propagation module and is used for segmenting the point cloud data into a welding seam area and a non-welding seam area; The characteristic extraction network adopts a graph convolution neural network to model point cloud data of a welding line area into a graph structure, nodes represent sampling points, and edges represent a space adjacent relation; the graph convolution neural network comprises an edge convolution layer, a graph annotation force layer and a global pooling layer, wherein the edge convolution layer is used for aggregating geometric features of neighborhood points, the graph annotation force layer is used for adaptively learning weight coefficients of the neighborhood points, and the global pooling layer is used for integrating local features into global descriptors; The point cloud segmentation network outputs a weld type classification result while segmenting a weld region, wherein the weld type at least comprises a butt weld, a fillet weld, a lap weld and a plug weld, and the weld track information comprises weld space coordinates, welding gun postures and the weld type classification result.
4. The intelligent welding method based on deep learning and three-dimensional reconstruction according to claim 1, wherein optimizing the motion trajectory of the welding robot by a global optimization algorithm and a local obstacle avoidance algorithm comprises: the global optimization algorithm adopts a self-adaptive genetic algorithm and optimizes based on a multi-objective fitness function, wherein the multi-objective fitness function is as follows: wherein P represents a track path, L (P) represents a path length index, S (P) represents a path smoothness index defined as an accumulated sum of tangential vector angles of adjacent track segments, E (P) represents a joint energy consumption index defined as an integral of square of angular velocity of each joint, D (P) represents an obstacle avoidance safety margin index defined as an inverse of a minimum value of distances between each point on the track and a nearest obstacle, 、、、 Is a weight coefficient and satisfies ; The local obstacle avoidance algorithm adopts an artificial potential field method, calculates the resultant force direction of attractive force generated by a target point and repulsive force generated by an obstacle as a local optimization direction, and the cross probability and variation probability of the adaptive genetic algorithm are dynamically adjusted according to population fitness.
5. The intelligent welding method based on deep learning and three-dimensional reconstruction according to claim 4, wherein the dynamic adjustment calculation formula of the crossover probability Pc and the mutation probability Pm is: When (when) When in use; When (when) When in use; When (when) Time of day When (when) When in use; Wherein fmax is the maximum fitness value of the population, favg is the average fitness value of the population, f' is the larger fitness value of two individuals participating in intersection, f is the fitness value of the individual to be mutated, pc1 and Pc2 are the upper and lower limits of the intersection probability respectively, and Pm1 and Pm2 are the upper and lower limits of the mutation probability respectively; The trajectory smoothing process uses a polynomial interpolation of five times while constraining boundary conditions of position, velocity and acceleration.
6. The intelligent welding method based on deep learning and three-dimensional reconstruction according to claim 1, wherein the acquiring of the position information of the weld in real time during the welding process includes: Ray lasers are projected to a welding line area, images are synchronously collected, and Gaussian filtering, self-adaptive threshold segmentation, morphological processing and laser stripe central line extraction are sequentially carried out on the images; based on the laser triangulation ranging principle, calculating the three-dimensional space coordinates of the welding line, wherein the calculation formula is as follows: Wherein Z is the distance from the weld surface point to the optical center of the camera, f is the focal length of the camera lens, B is the baseline distance between the laser and the camera, d is the offset of the laser stripe on the image sensor, and θ is the included angle between the laser plane and the optical axis of the camera; and obtaining an internal reference matrix and a distortion coefficient through camera calibration, and obtaining a transformation relation between a sensor coordinate system and a robot tail end coordinate system through hand-eye calibration.
7. The intelligent welding method based on deep learning and three-dimensional reconstruction according to claim 6, wherein the filtering processing of the position information by a state estimation algorithm to obtain a real-time weld position, and the dynamic correction of the robot motion track according to the position deviation comprises: The state estimation algorithm adopts a Kalman filtering algorithm, and the state equation is that The observation equation is Wherein Xk is a state vector at k time and comprises the positions of the welding lines in the directions of three coordinate axes and the change rate of the positions; A is a state transition matrix, B is a control matrix, uk-1 is a control vector, wk-1 is a process noise, zk is an observation vector at k moment, H is an observation matrix, and Vk is an observation noise; and the dynamic correction adopts a PID controller, the position deviation is decomposed into a high-low direction deviation and a left-right direction deviation to be respectively compensated, and the compensation result is overlapped on the planned track to generate a corrected track instruction.
8. The intelligent welding method based on deep learning and three-dimensional reconstruction according to claim 1, wherein the defect detection of the weld image through the image recognition network comprises: the image recognition network adopts a convolutional neural network and comprises a plurality of convolutional blocks which are sequentially connected, each convolutional block comprises a convolutional layer, a batch normalization layer, an activation function and a pooling layer, and the convolutional features are pooled through global average and then sent into a full-connection layer to output probability distribution of each defect type; The targets for the defect detection include at least air holes, cracks, slag inclusions, unfused, undercut, and surface forming defects.
9. The intelligent welding method based on deep learning and three-dimensional reconstruction according to claim 8, wherein generating quality assessment results in combination with process parameter analysis and case matching comprises: The process parameter analysis adopts a cyclic neural network to carry out abnormality detection on time sequence data of welding current, arc voltage and wire feeding speed; The case matching is based on a welding quality knowledge graph and a case library, a neural network optimized similarity calculation method is adopted to search matching cases, and a similarity calculation formula is as follows: wherein C is the current case to be detected, ci is the i-th case in the case base, And (3) with The j-th feature attributes of C and Ci respectively, For the characteristic weight coefficients learned by the neural network, As a function of local similarity.
10. The intelligent welding method based on deep learning and three-dimensional reconstruction according to claim 1, wherein closed-loop adaptive control is performed based on the real-time weld position and the quality evaluation result, and automatically adjusting welding parameters comprises: Acquiring welding process parameters in real time, and establishing a welding quality prediction model by combining the real-time welding seam position and the quality evaluation result; When welding deviation or quality abnormality is detected, automatically adjusting welding parameters based on a reinforcement learning algorithm, including incremental adjustment of welding current, arc voltage and welding speed, wherein the adjustment strategy is to mainly adjust arc voltage and dry elongation of welding wires for high and low deviation and mainly adjust welding current and wire feeding speed for gap change; And storing point cloud data, planning tracks, real-time tracking data, quality assessment results and adjusted welding parameters in a welding database in a correlated manner in the welding process, periodically analyzing and labeling the accumulated welding data, adding high-quality welding cases into a training data set to perform incremental training update on the point cloud segmentation network, the feature extraction network and the image recognition network, and realizing complete closed loop of parameter adjustment from point cloud data acquisition, track planning, real-time tracking and quality detection and continuous iterative optimization based on data accumulation.

Description

Intelligent welding method based on deep learning and three-dimensional reconstruction Technical Field The invention relates to the technical field of intelligent manufacturing and industrial robots, in particular to an intelligent welding method based on deep learning and three-dimensional reconstruction. Background The existing welding automation technology depends on a manual teaching reproduction mode or a traditional image processing algorithm, and a single fixed track planning strategy is adopted. The manual teaching mode has the defects of low programming efficiency, poor track precision, lack of self-adaptive capacity and the like, the traditional image processing algorithm is not sensitive to illumination and has insufficient robustness, and a single track planning strategy is difficult to adapt to complex and changeable welding environments. Meanwhile, the existing scheme lacks a complete closed loop from three-dimensional sensing, track planning and real-time tracking of workpieces to quality detection, and the data flow of each link is broken, so that full-automatic intelligent control of the welding process cannot be realized. Disclosure of Invention Aiming at the problems of lack of complete data flow closed loop, low track extraction precision, single planning strategy, poor tracking self-adaption capability and the like in the prior art, the invention provides an intelligent welding method based on deep learning and three-dimensional reconstruction. The technical scheme for solving the technical problems is as follows: The invention provides an intelligent welding method based on deep learning and three-dimensional reconstruction, which comprises the following steps: Acquiring point cloud data of a workpiece to be welded, preprocessing the point cloud data, identifying a welding seam region through a point cloud segmentation network, and acquiring welding seam track information through a feature extraction network based on the welding seam region, wherein the welding seam track information comprises welding seam space coordinates and welding gun postures; optimizing the motion track of the welding robot through a global optimization algorithm and a local obstacle avoidance algorithm based on the weld track information, and generating a planning track through track smoothing; acquiring the position information of a welding line in real time in the welding process, filtering the position information through a state estimation algorithm to obtain the position of the welding line in real time, comparing the position of the welding line in real time with the planned track to obtain a position deviation, and dynamically correcting the motion track of the robot according to the position deviation; Performing defect detection on the weld image through an image recognition network, and generating a quality assessment result by combining process parameter analysis and case matching; and performing closed-loop self-adaptive control based on the real-time weld joint position and the quality evaluation result, and automatically adjusting welding parameters to form a complete closed loop with parameter adjustment from point cloud data acquisition, track planning, real-time tracking and quality detection. Optionally, acquiring point cloud data of the workpiece to be welded, and preprocessing the point cloud data, including: Acquiring the point cloud data in a laser three-dimensional scanning and/or structured light projection mode; Removing noise points by adopting a statistical outlier filtering algorithm, setting a neighborhood point k and a standard deviation multiple threshold sigma, judging points with deviation mean values exceeding sigma times standard deviation as outliers, and removing; downsampling the point cloud by adopting voxel grid filtering, and setting voxel side length parameters; and estimating normal vectors of each point by adopting a principal component analysis method, and setting search radius parameters. Optionally, identifying the weld region through a point cloud segmentation network, and acquiring weld track information through a feature extraction network based on the weld region, including: the point cloud segmentation network adopts a layering point cloud processing network comprising a multi-scale feature extraction module and a feature propagation module and is used for segmenting the point cloud data into a welding seam area and a non-welding seam area; The characteristic extraction network adopts a graph convolution neural network to model point cloud data of a welding line area into a graph structure, nodes represent sampling points, and edges represent a space adjacent relation; the graph convolution neural network comprises an edge convolution layer, a graph annotation force layer and a global pooling layer, wherein the edge convolution layer is used for aggregating geometric features of neighborhood points, the graph annotation force layer is used for adaptively learn