CN-121982029-A - Circuit board defect detection method and system based on cross-light source mode feature blending

CN121982029ACN 121982029 ACN121982029 ACN 121982029ACN-121982029-A

Abstract

The invention provides a circuit board defect detection method and a system based on cross-light source mode feature blending, and relates to the defect detection field, comprising the following steps of obtaining multi-mode images of the same region to be detected under coaxial bright field light, low-angle dark field light, near infrared light and RGB light; generating candidate defect areas based on response differences under different illumination modes, inputting a multi-mode image corresponding to the candidate defect areas into a plurality of parallel feature extraction branches to obtain a plurality of feature tensors, generating self-adaptive weights based on the feature tensors and obtaining a blended feature map, inputting the blended feature map into a defect prediction model, and outputting defect positions, defect categories and defect confidence. By adopting the method, the physical perception boundary of the traditional two-dimensional machine vision is expanded, and the detection capability of the complex PCB in the defect scene is improved. And the invalid region participation reasoning is reduced, so that the detection precision and the online detection efficiency are both considered.

Inventors

LI YAO
CHEN GUO
LI YANG

Assignees

苏州深视信息科技有限公司

Dates

Publication Date: 20260505
Application Date: 20260408

Claims (10)

1. The circuit board defect detection method based on cross-light source mode feature blending is characterized by comprising the following steps of: s1, acquiring a multi-mode image through a multi-light source camera module; S2, calculating candidate scores of each pixel point under each local area according to the physical-optical response difference of the multi-mode image and a candidate algorithm, generating a dynamic candidate threshold value, and generating a candidate defect area according to the candidate scores and the dynamic candidate threshold value; s3, inputting the candidate defect areas and the multi-mode images into a plurality of parallel feature extraction branches to obtain a plurality of feature tensors; S4, splicing all the feature tensors in the channel dimension to obtain a global feature tensor, carrying out space global average pooling and multi-layer perceptron processing on the global feature tensor to obtain fusion features, generating self-adaptive weights W corresponding to all modes according to the fusion features, and carrying out recalibration on the global feature tensor through the self-adaptive weights W to generate a fusion feature map; and S5, inputting the blending feature map into a defect prediction model obtained through joint loss function training, and outputting defect positions, defect types and defect confidence.
2. The method for detecting defects of a circuit board based on cross-light source mode feature blending according to claim 1, wherein in S1, the multi-light source camera module comprises a multi-channel light source array and a camera assembly, the multi-channel light source array comprises coaxial bright field light, low angle dark field light, near infrared light and RGB light, and the multi-mode image comprises bright field image, dark field image, near infrared image and RGB image.
3. The method for detecting defects of a circuit board based on cross-light source mode feature blending according to claim 2, wherein in S2, the specific step of calculating a candidate score of each pixel point under each local area includes: s21, converting the RGB image into a single-channel gray image or forming a visible light image by adopting a bright field image; s22, calculating pixel gray values of each pixel point of the bright field image and the dark field image under the corresponding local area to respectively obtain a corresponding bright field pixel gray matrix and a corresponding dark field pixel gray matrix, calculating gradient amplitude of each pixel point, and carrying out pixel level difference and normalization calculation according to the gradient amplitude to obtain a morphology gradient response component of the corresponding local area; S23, calculating a pixel gray value of each pixel point of the near infrared image under the corresponding local area to obtain a near infrared pixel gray matrix, calculating a pixel gray average value of the visible light image under the corresponding local area, and performing cross-spectrum residual error calculation and normalization calculation according to the near infrared pixel gray matrix and the pixel gray average value to obtain a spectrum contrast response component of the corresponding local area; S24, calculating pixel gray standard deviation of the visible light map in each local area, combining the pixel gray mean value to obtain local texture stability components, and summing the morphology gradient response components, the spectrum contrast response components and the local texture stability components in the same local area according to weights to obtain candidate scores of all pixel points in the corresponding local area.
4. The method for detecting defects of a circuit board based on cross-light source mode feature blending according to claim 3, wherein in S2, the specific step of generating a dynamic candidate threshold value, and generating a candidate defect region according to the candidate score and the dynamic candidate threshold value comprises: S25, adaptively calculating a gray segmentation threshold value according to a visible light image by adopting an Ojin method, performing binarization segmentation on the visible light image according to the gray segmentation threshold value so as to distinguish a low gray substrate candidate region from a high gray conductive line region, calculating a candidate score average value and a candidate score standard deviation of the substrate candidate region in the multi-mode image, and calculating according to the candidate score average value and the candidate score standard deviation to obtain a dynamic candidate threshold value; S26, traversing all local areas of the multi-mode image, and judging the pixel points to be suspected defective pixels if the candidate scores of the pixel points in the local areas are larger than the dynamic candidate threshold; S27, carrying out morphological opening and closing operation on the multi-mode image to eliminate isolated noise points, and extracting the minimum circumscribed rectangle through connected domain analysis to generate a candidate defect region.
5. The method for detecting defects in a circuit board based on cross-light source mode feature blending according to claim 3, wherein an expression of the candidate score is calculated: , Wherein, the Is a candidate score that is a score of the candidate, 、 And Is the score adjustment weight factor and, Is a response component of the topography gradient, Is the spectral contrast response component of the spectrum, Is a local texture stability component; , Wherein, the Is a dark field pixel gray matrix of a dark field map, Is a bright field pixel gray matrix of a bright field map, Is used for extracting the operation symbol of the two-dimensional space edge physical gradient to obtain the gradient amplitude, Is a very small positive constant for ensuring calculation stability; , Wherein, the Is a near-infrared pixel gray matrix, Is a view of the light in the visible light, Is a pixel gray level average value calculation function, the I is an operation symbol of an absolute value, Is a minimum maximum normalization function; , Wherein, the Is the standard deviation of the pixel gray scale of the visible light map, Is the pixel gray-scale average of the visible light map.
6. The method for detecting defects of a circuit board based on cross-light source mode feature blending according to claim 4, wherein the expression of the dynamic candidate threshold is: , Wherein, the Is a dynamic candidate threshold value that is selected, A candidate score average for a non-conductive substrate region, The candidate standard deviation of scoring for areas of the non-conductive substrate, Is the sensitivity coefficient.
7. The method for detecting the circuit board defect based on cross-light source mode feature fusion according to claim 2, wherein the step S3 specifically comprises the steps of inputting the candidate defect region and corresponding bright field diagram, dark field diagram, near infrared diagram and RGB diagram into four parallel feature extraction branches of a CI-FFN network respectively, wherein the four parallel feature extraction branches perform independent physical feature extraction through independent weights in a shallow convolutional layer, and perform shared physical feature extraction through shared weights in a deep convolutional layer so as to obtain four feature tensors with the same spatial dimension and C channel number.
8. The circuit board defect detection method based on cross-light source modal feature blending according to claim 1 is characterized in that in S4, the specific steps of recalibrating global feature tensors through the self-adaptive weights W and generating a blended feature map comprise the steps of multiplying the self-adaptive weights W and the global feature tensors element by element along a channel dimension to obtain recalibrated features, and carrying out channel dimension reduction and information fusion on the recalibrated features through a convolution layer of 1x1 to obtain the blended feature map.
9. The method for detecting defects of a circuit board based on cross-light source mode feature blending according to claim 2, wherein in S5, the expression of the joint loss function is as follows: , , Wherein, the Is the joint loss function of the data set, And The classification weight and the regression weight are respectively, Is a class-loss function that is a function of the class, Is the CIoU regression loss function of the model, Is a class balancing factor that is used to balance the classes, Is a dynamic scaling factor that is used to scale the image, Is the confidence level of the defect, Is the focus parameter.
10. The circuit board defect detection system based on cross-light source mode feature fusion is characterized in that the circuit board defect detection method based on cross-light source mode feature fusion as claimed in any one of claims 1-9 is adopted, and the circuit board defect detection system comprises: The multi-light source camera module is used for acquiring multi-mode images; The detection control module is used for calculating candidate scores of each pixel point under each local area according to physical-optical response differences of the multi-modal image and a candidate algorithm, generating a dynamic candidate threshold, generating a candidate defect area according to the candidate scores and the dynamic candidate threshold, inputting the candidate defect area and the multi-modal image into a plurality of parallel feature extraction branches to obtain a plurality of feature tensors, splicing all the feature tensors in channel dimensions to obtain a global feature tensor, carrying out space global pooling and multi-layer perceptron processing on the global feature tensor to obtain a self-adaptive weight W, generating a blended feature map according to the self-adaptive weight W and the global feature tensor, inputting the blended feature map into a defect prediction model, and outputting defect positions, defect categories and defect confidence.

Description

Circuit board defect detection method and system based on cross-light source mode feature blending Technical Field The invention relates to circuit board defect detection, in particular to a circuit board defect detection method and system based on cross-light source mode feature blending. Background At present, an Automatic Optical Inspection (AOI) device generally adopts a single light source or adopts a bright field light and dark field light alternative lighting mode to collect images of a PCB to be inspected, and in an algorithm level, the images collected by different light sources are input into a general inspection network after being simply stacked in a channel mode so as to realize defect identification. However, the prior art has the common problems of (1) high false defect false alarm rate. Unstructured interference such as dust, water stain, scaling powder residue and the like on the surface of the PCB is easy to be misjudged as a real defect under a single light source or a simple splicing scheme. (2) multiple types of confounding defects are difficult. Different defects correspond to different optical response mechanisms, and the prior art has difficulty in simultaneously considering surface defects and subsurface related anomalies. (3) the consumption of computing resources is large, and the online beats are difficult to meet. The direct input of multiple light source images into the depth network can significantly increase the amount of computation, and the network has difficulty in effectively utilizing the complementary relationship between the light sources. How to fully utilize the physical response difference corresponding to different illumination modes, reduce false defect false alarm and invalid calculation cost while ensuring detection precision, and become a technical problem to be solved in the field of online defect detection of printed circuit boards. Disclosure of Invention In order to overcome the problems in the prior art, the invention provides a circuit board defect detection method and a system based on cross-light source mode feature blending, and the method and the system can effectively reduce false defect false alarm and invalid calculation overhead. The invention adopts the following technical scheme. A circuit board defect detection method based on cross-light source mode feature blending comprises the following steps: s1, acquiring a multi-mode image through a multi-light source camera module; S2, calculating candidate scores of each pixel point under each local area according to the physical-optical response difference of the multi-mode image and a candidate algorithm, generating a dynamic candidate threshold value, and generating a candidate defect area according to the candidate scores and the dynamic candidate threshold value; s3, inputting the candidate defect areas and the multi-mode images into a plurality of parallel feature extraction branches to obtain a plurality of feature tensors; S4, splicing all feature tensors in channel dimensions to obtain global feature tensors, carrying out space global average pooling and multi-layer perceptron processing on the global feature tensors to obtain fusion features, generating self-adaptive weights W corresponding to all modes according to the fusion features, and carrying out recalibration on the global feature tensors through the self-adaptive weights W to obtain a fusion feature map; S5, inputting the blending feature map into a defect prediction model obtained through joint loss function training, and outputting defect positions, defect types and defect confidence. As a further improvement of the present invention, in S1, the multi-light source camera module includes a multi-channel light source array including on-axis bright field light, low-angle dark field light, near infrared light, and RGB light, and a camera assembly, and the multi-mode image includes a bright field image, a dark field image, a near infrared image, and an RGB image. As a further improvement of the present invention, in S2, the specific step of calculating a candidate score for each pixel under each local area includes: s21, converting the RGB image into a single-channel gray image or forming a visible light image by adopting a bright field image; S22, calculating pixel gray values of each pixel point of the bright field image and the dark field image under the corresponding local area to respectively obtain a corresponding bright field pixel gray matrix and a corresponding dark field pixel gray matrix, calculating gradient amplitude of each pixel point, and carrying out pixel level difference and normalization calculation according to the gradient amplitude to obtain morphology gradient response components of the corresponding local area; s23, calculating a pixel gray value of each pixel point of the near-infrared image in the corresponding local area to obtain a near-infrared pixel gray matrix, calculating a pixel gray average value of th