CN-122023283-A - Three-dimensional object weld joint identification method and device based on point cloud and image fusion

CN122023283ACN 122023283 ACN122023283 ACN 122023283ACN-122023283-A

Abstract

The invention discloses a three-dimensional object weld joint identification method and device based on point cloud and image fusion, and relates to the technical field of image processing. A three-dimensional object weld joint identification method based on point cloud and image fusion comprises the steps of obtaining three-dimensional point cloud data and corresponding two-dimensional image data of the surface of a workpiece to be welded, extracting a suspected weld joint point set from the three-dimensional point cloud data, fitting to generate a suspected weld joint skeleton line, projecting the suspected weld joint skeleton line into a pixel coordinate system of the two-dimensional image data based on a hand-eye calibration conversion matrix to generate an ROI mask distributed along a weld joint track, extracting features in the ROI mask, obtaining two-dimensional central line pixel coordinates of the weld joint, reversely mapping back to a three-dimensional space to obtain a fine candidate point set, calculating space geometric consistency between the fine candidate point set and the suspected weld joint skeleton line, retaining real weld joint points and fitting curves to obtain three-dimensional weld joint track coordinates. The invention is suitable for V-shaped grooves, lap joints and even plane butt welds with very small gaps.

Inventors

CHANG HAONAN
HUA TIANYU
YAN HONGKUN
WEI JINYU
LI FENG
LI DONG

Assignees

苏州灵视视觉科技有限公司

Dates

Publication Date: 20260512
Application Date: 20260109

Claims (10)

1. A three-dimensional object weld joint identification method based on point cloud and image fusion is characterized by comprising the following steps: Acquiring three-dimensional point cloud data and corresponding two-dimensional image data of the surface of a workpiece to be welded; preprocessing the three-dimensional point cloud data, extracting a suspected weld point set according to local geometric features, and fitting to generate a suspected weld skeleton line in a three-dimensional space; Based on a hand-eye calibration conversion matrix from a three-dimensional vision sensor coordinate system to a two-dimensional camera coordinate system, projecting the suspected weld skeleton line into a pixel coordinate system of two-dimensional image data, and generating an ROI mask of the two-dimensional image data distributed along a weld track; Extracting features in the ROI mask of the two-dimensional image data, obtaining two-dimensional central line pixel coordinates of a welding line, reversely mapping back to a three-dimensional space through a hand-eye calibration conversion matrix to obtain a fine candidate point set; And performing curve fitting on the real weld joint points to obtain three-dimensional weld joint track coordinates.
2. The three-dimensional object weld joint identification method based on point cloud and image fusion according to claim 1, wherein the preprocessing is used for removing spatial noise points by performing through filtering and statistical outlier removal on three-dimensional point cloud data.
3. The three-dimensional object weld joint identification method based on point cloud and image fusion according to claim 1, wherein the process of extracting the suspected weld joint point set according to the local geometric features is as follows: for each target point in the three-dimensional point cloud data, search for it All points in the neighborhood; calculating the main component analysis method Covariance matrix of neighborhood point and corresponding eigenvalue Wherein Using the formula Calculating the surface curvature change of the target point ; When the surface curvature changes When the target point is larger than a preset geometric feature threshold value, marking the target point as a suspected weld joint point; and connecting the suspected weld points by using a DBSCAN clustering algorithm, and fitting out a suspected weld skeleton line in a three-dimensional space.
4. The method for recognizing a weld joint of a three-dimensional object based on point cloud and image fusion according to claim 1, wherein the process of generating the ROI mask distributed along the trajectory of the weld joint is as follows: using pinhole camera model and external reference matrix Three-dimensional points in the suspected weld skeleton line Projected to a two-dimensional image pixel plane : ; In the formula, As an internal reference matrix of the camera, Is a scale factor; In projection points Centered on Generating rectangular windows for side length, and combining all the rectangular windows to form the final rectangular ROI mask, wherein the rectangular ROI size With depth And (3) self-adaptive adjustment: ; In the formula, For the focal length of the camera, For the preset scale factor to be a preset one, Is the base redundancy width.
5. The three-dimensional object weld joint identification method based on point cloud and image fusion according to claim 1, wherein the feature extraction is used for calculating the normal direction of the gray gradient of an image by using a Steger algorithm, and extracting a light bar center or a texture edge center with sub-pixel precision; Or inputting the image in the ROI mask into a pre-trained yolov semantic segmentation neural network model, outputting a pixel classification result of a welding line region, and extracting a central axis; And reversely mapping the extracted two-dimensional center line pixel coordinates into a three-dimensional space through the corresponding depth values of the hand-eye calibration conversion matrix and the pixel coordinates in the three-dimensional point cloud data by index matching or interpolation to obtain a fine candidate point set.
6. The three-dimensional object weld joint identification method based on point cloud and image fusion according to claim 1, wherein the method for eliminating the concentrated artifact points of fine candidate points by calculating the geometric consistency of a space is characterized by comprising the following specific steps: Traversing each point in the fine candidate point set Calculate the point Shortest Euclidean distance to the suspected weld skeleton line And obtaining a reference point on the suspected weld skeleton line ; When (when) Is greater than a preset distance checking threshold Determining the point Artifacts created by scratches or stains on the two-dimensional image and removed from the point set; When (when) Less than or equal to the distance verification threshold The point is preserved.
7. The method for three-dimensional object weld joint identification based on point cloud and image fusion as defined in claim 6, wherein said calculation of spatial geometric consistency further comprises a normal vector consistency check for each point in said fine candidate point set Searching for distance in original three-dimensional point cloud data Recently, the method of the present invention Calculating the neighborhood points The characteristic value decomposition is carried out on the covariance matrix, and the characteristic vector corresponding to the minimum characteristic value is selected as the point Is a local surface normal vector of (2) Obtaining a reference point by adopting the same calculation method Is a local surface normal vector of (2) Calculating by using vector dot product formula And (3) with Included angle between : ; In the formula of " "Representing a vector dot product" "Represents the length of the vector modulus when calculating the included angle Greater than a preset angle threshold Then determine the point And (5) eliminating the non-welding line interference points.
8. The three-dimensional object weld joint identification method based on point cloud and image fusion according to claim 1, wherein the curve fitting adopts a cubic B spline interpolation algorithm to carry out smoothing treatment on the reserved real weld joint points, resamples according to a fixed step length and outputs welding track data comprising position coordinates and welding gun posture vectors.
9. The three-dimensional object weld joint recognition device based on point cloud and image fusion is characterized by comprising: The data acquisition module is used for acquiring three-dimensional point cloud data and corresponding two-dimensional image data of the surface of the workpiece to be welded; preprocessing the three-dimensional point cloud data, extracting a suspected weld point set according to local geometric features, and fitting to generate a suspected weld skeleton line in a three-dimensional space; the weld joint identification module is used for projecting the suspected weld joint skeleton line into a pixel coordinate system of two-dimensional image data based on a hand-eye calibration conversion matrix from a three-dimensional vision sensor coordinate system to a two-dimensional camera coordinate system to generate an ROI mask of the two-dimensional image data distributed along a weld joint track; Extracting features in the ROI mask of the two-dimensional image data, obtaining two-dimensional central line pixel coordinates of a welding line, reversely mapping back to a three-dimensional space through a hand-eye calibration conversion matrix to obtain a fine candidate point set; And performing curve fitting on the real weld joint points to obtain three-dimensional weld joint track coordinates.
10. A computer readable storage medium having stored thereon a computer program/instructions, which when executed by a processor, performs the steps of the method for identifying a weld of a three-dimensional object based on point cloud and image fusion as claimed in any one of claims 1 to 8.

Description

Three-dimensional object weld joint identification method and device based on point cloud and image fusion Technical Field The invention relates to a three-dimensional object weld joint identification method and device based on point cloud and image fusion, and belongs to the technical field of image processing. Background In the field of industrial manufacturing, welds are important connecting structures for metal components, the spatial position and morphology of which directly affect the accuracy of post-weld quality assessment, weld grinding, and automated process path planning. Therefore, the method has important significance in accurately and stably identifying the welding seam on the surface of the three-dimensional object. The existing weld joint identification method mainly comprises the following steps: 1. Weld joint recognition methods based on two-dimensional images such methods typically utilize an industrial camera to capture weld joint images, and recognize weld joint positions through gray level analysis, edge detection, or deep learning models. The method has the advantages of simple realization, easy illumination change, weld surface reflection, splash and background texture interference in the actual industrial environment, poor recognition stability and difficult direct acquisition of three-dimensional space information of the weld. 2. The method is used for acquiring point cloud data of the surface of a workpiece in a mode of structured light, laser scanning and the like, and extracting a welding line region by utilizing geometric features such as curvature change, normal abrupt change or height difference. Although the point cloud method is insensitive to illumination change, false detection or omission is easy to occur under the conditions of small welding lines, continuous curved surfaces or noise, and the detail of the welding lines is difficult to accurately express when the resolution of the point cloud is limited. 3. In the existing point cloud and image fusion method, although the point cloud and image information are acquired simultaneously in the prior art, the result superposition or weighted fusion is carried out after independent processing, an effective cooperative constraint mechanism is not formed, and the problems of low calculation efficiency and high false detection rate still exist. In view of the problems that in the existing weld joint identification technology, a method based on a two-dimensional image is easy to be interfered by illumination change, reflection and splashes on the surface of a weld joint, the identification stability is insufficient, and three-dimensional space information of the weld joint is difficult to directly obtain, while a method based on a three-dimensional point cloud is easy to be subjected to false detection and omission under the conditions that the weld joint is geometrically continuous with a base metal, the size of the weld joint is smaller or the resolution of the point cloud is limited, and the positioning precision of the boundary of the weld joint is limited, so that the method is difficult to meet the actual requirements of accurate and stable identification of the weld joint in a complex industrial scene. Meanwhile, the existing part adopts a welding seam recognition scheme of point cloud and image, mostly adopts independent processing or result layer superposition of point cloud and image information, lacks an effective cooperative constraint mechanism, fails to fully exert the complementary advantages of point cloud geometric information and image texture information in welding seam recognition, and still has the problems of low calculation efficiency, insufficient anti-interference capability and poor engineering adaptability. Disclosure of Invention The invention aims to provide a three-dimensional object weld joint identification method and device based on point cloud and image fusion, which are used for defining a weld joint candidate region in space by utilizing the geometric characteristics of the three-dimensional point cloud, carrying out two-dimensional image weld joint characteristic identification in the defined region, and carrying out reverse correction on the point cloud weld joint region by combining an image identification result so as to realize reliable identification and accurate positioning of a weld joint. In order to achieve the above purpose, the invention is realized by adopting the following technical scheme. In one aspect, the invention provides a three-dimensional object weld joint identification method based on point cloud and image fusion, which comprises the following steps: Acquiring three-dimensional point cloud data and corresponding two-dimensional image data of the surface of a workpiece to be welded; preprocessing the three-dimensional point cloud data, extracting a suspected weld point set according to local geometric features, and fitting to generate a suspected weld skeleton line in a