Search

CN-121998913-A - Electronic element welding defect visual detection method and system based on multispectral imaging

CN121998913ACN 121998913 ACN121998913 ACN 121998913ACN-121998913-A

Abstract

The application relates to a visual inspection method and a visual inspection system for welding defects of electronic elements based on multispectral imaging, wherein the visual inspection method comprises the steps of acquiring multiband sequence images of a welding area to be tested, which are acquired by a preset multispectral imaging terminal, when a data acquisition instruction is received; the method comprises the steps of constructing a three-dimensional morphology point cloud of a welding area to be detected based on a multiband sequence image, extracting associated feature vectors related to welding quality in the multiband sequence image, carrying out weighted fusion on associated features in different wavebands to generate a fusion feature image, carrying out space matching alignment on the fusion feature image and the three-dimensional morphology point cloud, outputting geometric position information of the associated feature vectors in the three-dimensional morphology point cloud, identifying and judging whether defect features are formed, screening defect types corresponding to the defect features when the defect features are judged to exist, and calculating quantization information corresponding to the defect features. The application has the advantages of more comprehensive detection of the welding point and realization of accurate analysis of the defect of the welding point.

Inventors

  • SUN WEI
  • Shi Jiarun

Assignees

  • 锦瑞信息科技(南通)有限公司

Dates

Publication Date
20260508
Application Date
20251231

Claims (10)

  1. 1. A visual inspection method for welding defects of electronic elements based on multispectral imaging is characterized by comprising the following steps: When a data acquisition instruction is received, acquiring a multiband sequence image of a welding area to be detected, wherein the multiband sequence image comprises a visible light wave band, a near infrared wave band and a short wave infrared wave band, and the multiband sequence image is acquired by a preset multispectral imaging terminal; constructing a three-dimensional morphology point cloud of a welding area to be detected based on the multiband sequence image; Extracting associated feature vectors related to welding quality in the multiband sequence image based on a preset feature recognition rule, and carrying out weighted fusion on the associated features in different wave bands to generate a fusion feature map; Matching and aligning the fusion feature map and the three-dimensional morphology point cloud in space through coordinate transformation, and outputting geometric position information of the associated feature vector in the three-dimensional morphology point cloud; identifying and judging whether defect features are formed or not based on the distribution conditions of the associated feature vectors and the corresponding geometric position information; When judging that the defect characteristics exist, screening out defect types corresponding to the defect characteristics based on a preset defect characteristic database, calculating quantization information corresponding to the defect characteristics, and sending the defect types and the quantization information to a user side.
  2. 2. The visual inspection method of welding defects of electronic components based on multispectral imaging of claim 1, wherein the step of constructing a three-dimensional morphology point cloud of a welding area to be inspected based on the multiband sequence image comprises: The spectrum imaging terminal acquires a static plane image of a welding area to be detected through a visible light wave band and a near infrared wave band in advance; The spectrum imaging terminal performs three-dimensional scanning on a to-be-detected welding area through a visible light wave band, and performs phase constraint on the static plane image to generate a preliminary morphology point cloud; the spectrum imaging terminal scans a welding area to be detected in a near infrared band or a short wave infrared band, and judges whether an abnormal point group exists in the preliminary morphology point cloud; if the abnormal point group exists, acquiring a point group image of a position corresponding to the abnormal point group in a near infrared band or a short wave infrared band, and mapping the point group image to the position of the abnormal point group so as to cover the abnormal point group.
  3. 3. The visual inspection method of welding defects of electronic components based on multispectral imaging according to claim 1, wherein the step of extracting the associated feature vectors related to welding quality in the multiband sequence image based on the preset feature recognition rule, and performing weighted fusion on the associated features in different wavebands to generate a fused feature map comprises the following steps: denoising the multiband sequence image and registering phases of wave bands; Extracting common feature vectors of different wavebands in the multiband sequence image in advance through a preset feature extraction model; the feature extraction model respectively extracts specific feature vectors corresponding to the visible light wave band, the near infrared wave band and the short wave infrared wave band, respectively binds the common feature vector with a plurality of specific feature vectors, and obtains associated feature vectors in different wave bands; and giving different weight values to the associated feature vectors in different wave bands through an attention mechanism and a preset mark, and carrying out weighted fusion on the associated feature vectors in different wave bands.
  4. 4. The visual inspection method for welding defects of electronic components based on multispectral imaging according to claim 3, wherein in the step of assigning different weight values to the associated feature vectors in different wavebands through an attention mechanism and a preset mark, the preset mark comprises a welding spot type vector, the weight value ratios of the corresponding visible light wavebands, near infrared wavebands and short infrared wavebands are different, and the feature extraction model identifies the welding spot type of the welding area to be inspected through the attention mechanism so as to adjust the weight value ratios of the visible light wavebands, the near infrared wavebands and the short infrared wavebands.
  5. 5. The visual inspection method of soldering defects of electronic components based on multispectral imaging according to claim 1, wherein the step of identifying and judging whether to form a defect feature based on the distribution of the associated feature vector and the corresponding geometric position information comprises: judging whether the associated feature vector at the position is abnormal or not based on the geometric position information, and if so, acquiring a neighboring point group of the geometric position information in the three-dimensional morphology point cloud; identifying an abnormal point group range in which the associated feature vector is abnormal in the adjacent point groups; And when the abnormal point group range exceeds a preset range parameter, judging the defect characteristics of the current welding spot formation.
  6. 6. The visual inspection method for welding defects of electronic components based on multispectral imaging according to claim 5, wherein the step of determining whether the associated feature vector at the location is abnormal based on geometric location information, if so, obtaining the neighboring point group of the geometric location information in the three-dimensional morphology point cloud comprises: Judging welding spots to which the geometric position information belongs in the three-dimensional morphology point cloud, and identifying the welding spot type of the welding spots; Judging the distribution condition of the associated feature vectors of all geometric position information in the welding spots based on the welding spot types of the welding spots; Judging whether points with abnormal associated feature vectors exist in the welding spots or not based on the distribution condition of the associated feature vectors of all geometric position information in the current welding spots; if the point set exists, a point set associated with the abnormal point is acquired from the three-dimensional morphology point cloud, and the point associated with the coordinates is taken as a neighboring point group.
  7. 7. The visual inspection method of electronic component bonding defects based on multispectral imaging according to claim 6, wherein when judging that the defect features exist, the step of screening out defect types corresponding to the defect features and calculating quantization information corresponding to the defect features based on a preset defect feature database comprises the steps of: when judging that the characteristic defects exist, identifying an abnormal geometric area formed by the abnormal point group range; Based on the combination of the abnormal geometric area and the corresponding associated feature vector, the defect type of the current abnormal feature is matched from a preset defect feature database; And invoking a defect calculation method matched with the defect type, and calculating quantization information corresponding to the defect type based on the abnormal geometric region and the corresponding associated feature vector.
  8. 8. A visual inspection system for welding defects of electronic components based on multispectral imaging is characterized in that: the acquisition module is used for acquiring multiband sequence images of a welding area to be detected, which are acquired by a preset multispectral imaging terminal when a data acquisition instruction is received, wherein the multiband sequence images comprise visible light bands, near infrared bands and short wave infrared bands; The point cloud construction module is used for constructing a three-dimensional morphology point cloud of the welding area to be detected based on the multiband sequence image; The feature fusion module is used for extracting associated feature vectors related to welding quality in the multiband sequence image based on a preset feature recognition rule, and carrying out weighted fusion on the associated features in different wave bands to generate a fusion feature map; the spectrum and morphology combining module is used for matching and aligning the fusion feature map with the three-dimensional morphology point cloud in space through coordinate transformation and outputting the geometric position information of the association feature vector in the three-dimensional morphology point cloud; the defect judging module is used for identifying and judging whether defect features are formed or not based on the distribution conditions of the associated feature vectors and the corresponding geometric position information; and the defect analysis module is used for screening out defect types corresponding to the defect characteristics and calculating quantization information corresponding to the defect characteristics based on a preset defect characteristic database when judging that the defect characteristics exist, and sending the defect types and the quantization information to a user side.
  9. 9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the multispectral imaging-based visual inspection method of electronic component soldering defects as claimed in any one of claims 1 to 7 when the computer program is executed.
  10. 10. A computer-readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the steps of the multispectral imaging-based electronic component soldering defect visual inspection method of any one of claims 1 to 7.

Description

Electronic element welding defect visual detection method and system based on multispectral imaging Technical Field The application relates to the technical field of visual detection, in particular to a visual detection method and a visual detection system for welding defects of electronic elements based on multispectral imaging. Background With the development of miniaturization and high density of electronic elements, automatic visual detection of welding quality faces serious challenges, the currently mainstream inspection technology mainly obtains morphology and color information of the surface of a welding spot through two-dimensional visual sense of visible light, and for defects of inner or subsurface layers such as false welding, inner cavities, microcracks and the like, the condition of missing inspection is very easy to occur due to lack of penetrability or insufficient characteristic contrast, meanwhile, the current two-dimensional visual detection is only stopped at a judging level with defects, and the capability of quantifying and analyzing the defects of the welding spot is insufficient, so that effective closed loop feedback is difficult to provide for subsequent process parameter optimization, and the detection precision and analysis depth of the welding defect need to be improved. Disclosure of Invention In order to enable welding point detection to be more comprehensive and achieve accurate analysis of welding point defects, an effective and reliable improvement strategy is provided for welding process parameters, and the application provides a visual detection method and a visual detection system for the welding point defects of an electronic element based on multispectral imaging. The first object of the present application is achieved by the following technical solutions: a visual inspection method for welding defects of electronic components based on multispectral imaging comprises the following steps: When a data acquisition instruction is received, acquiring a multiband sequence image of a welding area to be detected, wherein the multiband sequence image comprises a visible light wave band, a near infrared wave band and a short wave infrared wave band, and the multiband sequence image is acquired by a preset multispectral imaging terminal; constructing a three-dimensional morphology point cloud of a welding area to be detected based on the multiband sequence image; Extracting associated feature vectors related to welding quality in the multiband sequence image based on a preset feature recognition rule, and carrying out weighted fusion on the associated features in different wave bands to generate a fusion feature map; Matching and aligning the fusion feature map and the three-dimensional morphology point cloud in space through coordinate transformation, and outputting geometric position information of the associated feature vector in the three-dimensional morphology point cloud; identifying and judging whether defect features are formed or not based on the distribution conditions of the associated feature vectors and the corresponding geometric position information; When judging that the defect characteristics exist, screening out defect types corresponding to the defect characteristics based on a preset defect characteristic database, calculating quantization information corresponding to the defect characteristics, and sending the defect types and the quantization information to a user side. By adopting the technical scheme, the multispectral imaging terminal acquires images of welding spots of a welding area to be detected under different wave band lights for a plurality of times through the visible light wave band, the near infrared wave band and the short wave infrared wave band, wherein the visible light wave band can acquire surface textures and color characteristics of the welding spots, the near infrared wave band can acquire internal structural transmission characteristics of the welding spots, the short wave infrared wave band can acquire thermal distribution characteristics of the welding spots, the image acquisition under different wave bands can comprehensively acquire multidimensional characteristics of the welding spots, the high-sensitivity detection of traditional blind area defects such as virtual welding, internal cavities, microcracks and the like is realized, the detection range is extended from the surface to the inside and the interface, further, the judgment accuracy of defects such as bridging, warping, tin-less and the like is obviously improved by constructing three-dimensional shape point clouds, the error rate is reduced, and the characteristic fusion of the welding spots and the spectrum imaging of the welding spots is fused in the point clouds, namely, the geometric position information of each welding spot is fused with the relevant characteristic vector, the defect identification is more attractive, the defect detection is improved, the defect detec