CN-122023243-A - SMT patch multi-defect real-time detection system and method

CN122023243ACN 122023243 ACN122023243 ACN 122023243ACN-122023243-A

Abstract

The invention discloses an SMT patch multi-defect real-time detection system and method. The algorithm reasoning unit adopts a double-layer distillation model, wherein a main network receives SMT process semantic features and control related semantic features, and a sub-network synchronously outputs defect detection results and pre-calculated control parameters through knowledge distillation. The soft-hard cooperative module comprises a cooperative triggering unit and a reverse calibration unit, realizes parallel processing of algorithm reasoning and hardware control, and corrects control parameters through real-time working condition data feedback. The invention solves the technical problems of insufficient precision and high system delay of the traditional lightweight model and meets the real-time detection requirement of a high-speed SMT production line through a model training and reasoning-pre-computing parallel mechanism guided by process semantics.

Inventors

GOU PENGFEI
LONG XIANGLI

Assignees

宁波双龙光学技术有限公司

Dates

Publication Date: 20260512
Application Date: 20251222

Claims (10)

1. An SMT patch multi-defect real-time detection system, comprising: the image acquisition unit is used for acquiring RGB images of the SMT patch; the algorithm reasoning unit is connected with the image acquisition unit and is used for receiving and processing the RGB image, and the algorithm reasoning unit comprises: The main network is used for receiving the SMT process semantic features and control related semantic features, and training the SMT process semantic features and the control related semantic features through a composite loss function to output trained model parameters, wherein the SMT process semantic features comprise a pad topological structure, element packaging gray level distribution and standard welding spot morphology, and the control related semantic features comprise a center coordinate offset range of a defect area and a defect area threshold value; the sub-network is connected with the main network, is used for receiving the RGB image through double-layer distillation migration data from the main network, and synchronously outputs a defect detection result and pre-calculation control parameters, wherein the defect detection result comprises a defect type, coordinates and confidence, and the pre-calculation control parameters comprise feeding coordinates, a mechanical arm correction angle or a laser marking range required by hardware execution; The soft and hard cooperation module is respectively connected with the algorithm reasoning unit and the hardware execution unit and comprises: The cooperative triggering unit is used for synchronously transmitting the pre-calculated control parameters to the hardware execution unit in a lightweight data frame format when the sub-network outputs the defect detection result; The reverse calibration unit is used for receiving the real-time working condition data fed back by the hardware execution unit, and correcting the pre-calculated control parameters in real time based on the SMT process semantic features migrated in the sub-network so as to generate corrected pre-calculated control parameters; and the hardware execution unit is connected with the soft and hard coordination module and is used for executing sorting or marking operation according to the pre-calculated control parameters or the corrected pre-calculated control parameters.
2. The SMT patch multi-defect real-time detection system of claim 1, wherein the composite loss function comprises a feature alignment loss term, a context consistency loss term, and a control semantic consistency loss term; The characteristic alignment loss term is used for restraining the distribution consistency of the characteristics output by the main network and the standard characteristics in a multidimensional space; the context consistency loss term is used for maintaining the spatial relevance between adjacent feature points in the feature extraction process; The control semantic consistency loss item is used for ensuring that the defect geometric features output by the main network are matched with hardware control parameter thresholds in the control-associated semantic features; And the composite loss function integrates the three losses in a weighted summation mode, wherein the weight coefficient is dynamically adjusted according to the process requirement of the SMT production line.
3. The SMT patch multi-defect real-time detection system of claim 2, wherein the composite loss function is expressed as: ; Wherein, the As a value of the total loss, The weight coefficients for the feature alignment penalty, Weight coefficients for the context consistency penalty, To control the weight coefficient of semantic consistency loss, As a result of the normalization factor, As a function of the integral variable, Is a shape parameter of the Gamma function, As a function of the Gamma of the light source, Is a natural constant which is used for the production of the high-temperature-resistant ceramic material, As a function of the error, Is the position of the model The characteristic value of the output is used for generating a characteristic value, To mark the position Is used for the characteristic value of the (c), As a scaling parameter for the feature differences, In order to find the sum of the indices, As the total number of dimensions of the contextual feature, As a bezier function of the first kind, As a scaling factor for the context feature, Is the first The feature value of the context is maintained, As a parameter of the Gamma function, In order to control the attenuation coefficient of the parameter differences, Is the control parameter difference value.
4. The SMT patch multi-defect real-time detection system of claim 1, wherein the SMT process semantic features are obtained by: extracting characteristic information of a pad topological structure, element packaging gray distribution and standard welding spot morphology from the SMT patch image based on the image segmentation model; The control related semantic features are obtained by importing SMT production line process files and comprise a center coordinate offset range of a defect area and a defect area threshold value; the master network receives the SMT process semantic feature and the control-associated semantic feature simultaneously as inputs.
5. The SMT patch multi-defect real-time detection system of claim 1, wherein the sub-network is constructed based on a lightweight convolutional neural network; The sub-network is configured to synchronously output defect detection results including defect types, coordinates and confidence and pre-calculated control parameters including feed coordinates, mechanical arm correction angles or laser marking ranges in a preset time period after receiving the RGB image.
6. The SMT patch multi-defect real-time detection system of claim 1, wherein the co-trigger unit is configured to perform the following parallel operations: when the sub-network starts to output the defect detection result, immediately pushing the defect detection result to a system display end; meanwhile, the pre-calculated control parameters are packaged into lightweight data frames in a JSON format; And sending the lightweight data frame to the hardware execution unit in real time through an industrial Ethernet.
7. The SMT patch multi-defect real-time detection system of claim 1, wherein the inverse calibration unit is configured to: receiving real-time working condition data fed back by the hardware execution unit, wherein the real-time working condition data comprises the current position error of the mechanical arm and the actual power of the laser mark; Invoking SMT process semantic features migrated in the sub-network, including mechanical compatibility threshold values of element encapsulation and laser tolerance power of welding spots; And when the real-time working condition data exceeds the fault tolerance range defined by the SMT process semantic features, dynamically correcting the pre-calculated control parameters.
8. The SMT patch multi-defect real-time detection system of claim 1, wherein the hardware execution unit comprises: The six-axis sorting mechanical arm is configured to execute sorting or correcting operation according to the feeding coordinates or the mechanical arm correcting angle in the pre-calculated control parameters; And the ultraviolet laser marking device is configured to perform marking operation according to the laser marking range in the pre-calculated control parameters.
9. The SMT patch multi-defect real-time detection system of claim 1, wherein the image acquisition unit comprises an industrial camera and a telecentric lens configured to acquire the RGB images at a 120fps frame rate; The algorithm reasoning unit is realized based on edge computing equipment; The image acquisition unit, the algorithm reasoning unit and the hardware execution unit are connected through an EtherCAT industrial Ethernet.
10. An SMT patch multi-defect real-time detection method applied to the SMT patch multi-defect real-time detection system of any one of claims 1-9, comprising: Step S1, an image acquisition unit acquires RGB images of an SMT patch; Step S2, a main network receives SMT process semantic features and control associated semantic features, and trains the SMT process semantic features and the control associated semantic features through a composite loss function to output trained model parameters, wherein the SMT process semantic features comprise a pad topological structure, element packaging gray level distribution and standard welding spot morphology, and the control associated semantic features comprise a center coordinate offset range of a defect area and a defect area threshold value; Step S3, a sub-network migrates data from the main network through double-layer distillation, receives the RGB image, and synchronously outputs a defect detection result and pre-calculation control parameters, wherein the defect detection result comprises a defect type, coordinates and confidence level, and the pre-calculation control parameters comprise feeding coordinates, a mechanical arm correction angle or a laser marking range required by hardware execution; step S4, when the sub-network outputs the defect detection result, the cooperative triggering unit synchronously sends the pre-calculated control parameters to the hardware execution unit in a lightweight data frame format; S5, a reverse calibration unit receives real-time working condition data fed back by the hardware execution unit, and real-time corrects the pre-calculated control parameters based on the SMT process semantic features migrated in the sub-network to generate corrected pre-calculated control parameters; And S6, the hardware execution unit executes sorting or marking operation according to the pre-calculated control parameters or the corrected pre-calculated control parameters.

Description

SMT patch multi-defect real-time detection system and method Technical Field The invention relates to the technical field of quality detection and control in a Surface Mount Technology (SMT) production line, in particular to a SMT chip multi-defect real-time detection system and method. Background Surface Mount Technology (SMT) is widely used in electronic manufacturing, and the production process involves multiple links such as component mounting, welding, and defect detection. Along with the improvement of the production automation degree, the real-time detection and defect processing in the surface mounting process become key links for guaranteeing the product quality and the production efficiency. In actual production, SMT chip mounting devices typically detect solder joints and components through an image acquisition system to identify soldering defects or mounting offsets. However, due to the diversity of different process parameters, element packaging types and pad topologies, the conventional detection system has insufficient generalization performance under a complex scene and often needs frequent manual calibration or relies on experience parameters, so that the detection precision is unstable and the automatic control efficiency is affected. In addition, in the patch detection and control link, the problems of low coupling degree and high feedback delay often exist between the algorithm model and the hardware execution unit. The detection result output by the algorithm cannot be converted into a hardware control instruction in time, or a real-time correction mechanism is lacking after the detection result is executed, so that error accumulation caused by working condition change in a dynamic production environment is difficult to deal with. Therefore, how to realize high-precision image detection and control parameter collaborative generation in an SMT patch scene, and improve the linkage and self-adaptive capacity between an algorithm reasoning result and hardware execution actions becomes a technical problem to be solved urgently. Disclosure of Invention Aiming at the defects existing in the prior art, the invention aims to provide the SMT patch multi-defect real-time detection system and the SMT patch multi-defect real-time detection method, which remarkably improve the SMT patch detection precision, control response speed and system self-adaption capability and realize an efficient, intelligent and movable patch defect detection and control integrated scheme. In order to achieve the above purpose, the invention provides a technical scheme that the SMT patch multi-defect real-time detection system comprises: the image acquisition unit is used for acquiring RGB images of the SMT patch; the algorithm reasoning unit is connected with the image acquisition unit and is used for receiving and processing the RGB image, and the algorithm reasoning unit comprises: The main network is used for receiving the SMT process semantic features and control related semantic features, and training the SMT process semantic features and the control related semantic features through a composite loss function to output trained model parameters, wherein the SMT process semantic features comprise a pad topological structure, element packaging gray level distribution and standard welding spot morphology, and the control related semantic features comprise a center coordinate offset range of a defect area and a defect area threshold value; the sub-network is connected with the main network, is used for receiving the RGB image through double-layer distillation migration data from the main network, and synchronously outputs a defect detection result and pre-calculation control parameters, wherein the defect detection result comprises a defect type, coordinates and confidence, and the pre-calculation control parameters comprise feeding coordinates, a mechanical arm correction angle or a laser marking range required by hardware execution; The soft and hard cooperation module is respectively connected with the algorithm reasoning unit and the hardware execution unit and comprises: The cooperative triggering unit is used for synchronously transmitting the pre-calculated control parameters to the hardware execution unit in a lightweight data frame format when the sub-network outputs the defect detection result; The reverse calibration unit is used for receiving the real-time working condition data fed back by the hardware execution unit, and correcting the pre-calculated control parameters in real time based on the SMT process semantic features migrated in the sub-network so as to generate corrected pre-calculated control parameters; and the hardware execution unit is connected with the soft and hard coordination module and is used for executing sorting or marking operation according to the pre-calculated control parameters or the corrected pre-calculated control parameters. Further, the composite loss function includes a featur