CN-121998894-A - Method, system and storage medium for detecting foreign matters in whole PCB (printed circuit board) based on twin network

CN121998894ACN 121998894 ACN121998894 ACN 121998894ACN-121998894-A

Abstract

The invention discloses a method, a system and a storage medium for detecting foreign matters of a whole PCB based on a twin network, wherein the method comprises the steps of generating a fake foreign matter defect image on a real PCB image and constructing an expansion data set for model training; the method comprises the steps of synchronously inputting a detection board image of a PCB to be detected and a gold board image in a standard state into a twin neural network with a shared weight sub-network structure, enabling two branches to respectively extract corresponding high-level semantic feature images, comparing the feature images of the gold board and the detection board pixel by pixel and carrying out difference enhancement processing through a visual feature difference extraction module, identifying an abnormal region which is obviously deviated from a normal mode, sending difference features into a path aggregation feature pyramid network to carry out multi-scale feature fusion, and judging whether foreign matters exist or not according to a preset threshold value by combining classification and positioning logic of a target detection head. The invention realizes high-precision and high-efficiency detection of the whole plate foreign matters such as flying parts, span parts, tin balls, stickers, greasy dirt and the like in the PCB in the SMT industry.

Inventors

Ru Binxin
LIU XINING
Tu Chuyu
JI CONG
SHANG BAOZHU
ZHONG JIAJIE
CHEN HONGFEI

Assignees

深圳市识渊科技有限公司
北京识渊科技有限公司
上海识渊科技有限公司

Dates

Publication Date: 20260508
Application Date: 20251203

Claims (10)

1. The method for detecting the foreign matters of the whole PCB based on the twin network is characterized by comprising the following steps: Simulating foreign body characteristics of flying pieces, span pieces, tin connection, stickers, tin beads and oil stain types by using a traditional vision processing algorithm, generating a representative fake foreign body defect image on a real PCB image, and constructing an expansion data set for model training; synchronously inputting a detection board image of the PCB to be detected and a gold board image in a standard state thereof into a twin neural network with a shared weight sub-network structure, so that two branches respectively extract corresponding high-level semantic feature images; The feature images of the gold plate and the detection plate are subjected to pixel-by-pixel comparison and difference enhancement processing through a visual feature difference extraction module, and an abnormal region which is obviously deviated from a normal mode is identified; Sending the difference features into a path aggregation feature pyramid network to perform multi-scale feature fusion, and strengthening the spatial expression capability of the fine foreign matters; and combining classification and positioning logic of the target detection head, judging whether foreign matters exist according to a preset threshold value, and outputting a detection result containing the type and the position information of the foreign matters.
2. The method for detecting the foreign matter on the whole PCB based on the twin network according to claim 1, wherein the generation process of the image of the counterfeit foreign matter defect comprises implanting a simulated foreign matter pattern into an original non-defective PCB image according to the geometric form, the size distribution, the spatial position rule and the gray texture characteristic of the foreign matter commonly found in the actual production, and applying the illumination variation and the noise disturbance to improve the data diversity.
3. The method for detecting the foreign matter on the whole PCB based on the twin network according to claim 1, wherein the golden board image is a standard reference image of the same type of PCB under the condition of confirming that no manufacturing defect exists, the detection board image is a PCB image to be detected which is actually acquired by the current production line, and the two images need to be matched with each other by keeping the same imaging angle and resolution.
4. The method for detecting foreign matter on a PCB based on a twin network according to claim 1, wherein the two sub-networks of the twin network adopt identical convolution architecture and share all learnable parameters, ensuring that a consistent feature mapping operation is performed on the golden board and the detection board, thereby highlighting structural differences therebetween.
5. The method for detecting the foreign matter on the whole PCB based on the twin network according to claim 1, wherein the visual characteristic difference extraction module generates a difference activation map by calculating an absolute difference or Euclidean distance of a characteristic map output by a gold plate and a detection plate, and enhances weak abnormal response through nonlinear transformation.
6. The method for detecting the foreign matter on the PCB based on the twin network according to claim 1, wherein the path aggregation feature pyramid network is connected with different layers of feature graphs through double paths from top to bottom and from bottom to top, so that efficient fusion of shallow details and deep semantic information is realized, and sensitivity to small-size foreign matters is improved.
7. The method for detecting the foreign matter on the whole PCB based on the twin network according to claim 1, wherein the target detection head completes candidate frame generation based on a region suggestion mechanism, judges the foreign matter category through a classification branch, positions the branch to determine the boundary frame coordinates, and finally outputs a foreign matter detection report with a confidence score.
8. Whole board foreign matter detecting system of PCB based on twin network, its characterized in that includes: The image acquisition module is used for acquiring a detection image of the PCB to be detected and a corresponding standard gold plate reference image thereof; The feature extraction module is used for extracting high-level semantic features of the detection image and the golden plate reference image respectively by adopting a double-branch convolutional neural network structure with shared weight, and generating a corresponding feature map; The visual characteristic difference extraction module is in communication connection with two output ends of the characteristic extraction module and is configured to carry out pixel-by-pixel difference comparison on the high-level semantic characteristic images of the gold plate reference image and the detection image, and an abnormal area which is obviously deviated from a normal mode is identified through difference enhancement processing; The multi-scale fusion module is coupled with the output end of the visual characteristic difference extraction module and is used for sending the characteristics subjected to the difference enhancement treatment into a characteristic pyramid network of the path aggregation structure, and performing cross-level characteristic fusion so as to strengthen the space expression capacity of the fine foreign matters; The detection output module is integrated with a target detection head, judges whether foreign matters exist or not under the preset threshold value condition based on the classification score and the positioning prediction result of the characteristics after multi-scale fusion, and outputs a detection result containing the foreign matter type and the spatial position information.
9. The system of claim 8, further comprising a model training module, which operates independently of the image acquisition module, and is configured to synthesize a sample of counterfeit foreign materials of the fly, span, sticker, tin bead and oil stain types on the historical real PCB image using conventional vision algorithms, construct an extended training dataset, and use the parameters of the training twin neural network model, and the trained model parameters are solidified and loaded into the feature extraction module for use in online detection.
10. A computer readable storage medium having stored thereon computer program instructions, which when executed by a processor, cause the processor to implement the twinning network based PCB whole board foreign object detection method of any one of claims 1 to 7.

Description

Method, system and storage medium for detecting foreign matters in whole PCB (printed circuit board) based on twin network Technical Field The invention relates to the technical field of intelligent detection of foreign matters on a whole PCB (printed circuit board) enabled by a twin network, in particular to a method, a system and a storage medium for detecting foreign matters on the whole PCB based on the twin network. Background The surface mount technology (Surface Mount Technology, SMT) is used as a core technology in the field of electronic manufacturing, and by directly mounting electronic components on the surface of a printed circuit board (Printed Circuit Board, PCB), the goals of high density, high performance and low cost of electronic product assembly are achieved. Compared with the traditional through hole technology, the SMT obviously improves the integration level of the circuit board, reduces the product volume and the manufacturing cost, and is widely applied to the manufacturing of modern electronic products such as smart phones, tablet personal computers, automobile electronics and the like. In SMT patch production, automated optical inspection (Automatic Optic Inspection, AOI) techniques are critical. The quality control method can be used for rapidly and accurately detecting the quality of the PCB after the surface mounting by virtue of high precision and high efficiency, timely finding and correcting the defects of component dislocation, surface leakage, polarity reversal and the like, and providing solid guarantee for the quality control of the SMT production line. In recent years, with the rise of artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) and machine learning technologies, the electronics manufacturing industry has been deeply revolutionized. In the field of SMT manufacturing printed circuit board assembly (Printed Circuit Board Assembly, PCBA), AI and AOI fusion becomes a key to drive intelligent transformation. The AI solution brings positive influence to SMT industry by means of the progress of hardware, machine vision and algorithm, namely, the hardware level, advanced processor and image sensor provide powerful computing and processing capability, the machine vision technology improves the image recognition capability, and the algorithm optimization, such as the application of deep learning algorithm, strengthens the automatic detection capability, reduces false alarm and continuously optimizes the production flow. In the high-reliability demand industry, such as the fields of automobiles and aerospace, the introduction of AI fills the defect of automatic inspection, meets strict production demands, timely discovers and corrects production problems through real-time monitoring and data analysis, improves the flow, optimizes process parameters, and improves the product quality and production efficiency. The "foreign object detection" in the SMT industry is a critical visual inspection task to ensure product quality. The presence of foreign materials such as flyers, span tin, tin beads, etc., can severely impact circuit board performance and reliability. Four major classes of foreign matter are detected: 1. flying parts, namely, components and parts are not accurately attached to a PCB in the SMT (surface mount technology) paster process, so that flying or serious deviation of positions occurs, short circuit, open circuit or damage of the components and parts can be caused, and the yield and reliability of a production line are threatened. Common types include completely unapplied, off-set flyers, drop stuck, and multi-suction, misplug flyers. 2. And the span is connected with tin, namely, the tin between two or more adjacent devices is connected accidentally, so that short circuit, signal interference or functional failure is caused, and the phenomenon that the mounting density is high, the printing of the soldering paste is excessive or the reflow process is abnormal is common. 3. And the solder beads are tiny solder particles separated from welding spots and appear around the welding pads or devices after welding, so that the appearance of the product is affected, and short circuit and reliability reduction can be caused. 4. Other foreign matters, including stickers, copper scraps, tool fragments, scaling powder residues, greasy dirt and the like, are detected, so that the packaging logic accuracy is improved, and the problems of short circuit, poor contact and the like are avoided. At present, a plurality of methods for detecting foreign matters such as flying pieces in SMT industry, such as traditional AOI+template matching, SPI (solder paste detection), AOI+traditional AI detection, X-ray detection (AXI), electrical performance test (ICT), functional test (FCT), chip mounter real-time monitoring and the like, are available. However, these methods have the defects of precision and stability problems, complex operation, high learning cost, detection limitation, lo