CN-121998959-A - Intelligent detection method and system for pin defects of connector based on machine vision

CN121998959ACN 121998959 ACN121998959 ACN 121998959ACN-121998959-A

Abstract

The invention provides a connector pin defect intelligent detection method and system based on machine vision, which relate to the technical field of machine vision and comprise the following steps: obtaining multi-view images under the multi-light source illumination condition, calculating three-dimensional morphology data, extracting stitch areas, establishing associated edges, combining surface normal vectors, boundary contours and other characteristics, optimizing node characteristics based on a graph neural network, extracting stitch examples and classifying defects, and iteratively updating defect labels through defect propagation probability. The invention can accurately identify various defects of the connector pins, improve the detection precision and efficiency, and reduce the omission ratio and the false detection rate.

Inventors

HU HAIYANG
MA JUNXIA

Assignees

深圳智航精密科技有限公司

Dates

Publication Date: 20260508
Application Date: 20260130

Claims (10)

1. The intelligent detection method for the pin defects of the connector based on machine vision is characterized by comprising the following steps of: acquiring a multi-view image sequence of a connector to be detected under a multi-light source illumination condition, solving pixel gray values of the same spatial position under different illumination angles to obtain three-dimensional morphology data, extracting edge pixel points with obvious surface curvature change from the three-dimensional morphology data, grouping according to generality to obtain candidate stitch areas, performing spatial position verification on the candidate stitch areas, reserving stitch areas meeting preset design specifications, distributing node identifiers for each stitch area, and establishing a correlation edge between stitch areas which are spatially adjacent and have collaborative postures; Taking the surface normal vector and boundary contour corresponding to the stitch region as node characteristics, transmitting the node characteristics to a neighborhood node along the associated edge, aggregating the neighborhood transmission characteristics to obtain optimized node characteristics, determining a stitch instance set based on the optimized node characteristics, extracting the surface morphology characteristics of each stitch instance in the stitch instance set from the three-dimensional morphology data, splicing the surface morphology characteristics with the optimized node characteristics to obtain a high-dimensional characteristic representation, performing multi-scale convolution operation on the high-dimensional characteristic representation, and then decoding to obtain a preliminary detection result; And calculating defect propagation probability between each pair of stitch examples adjacent to each other in the stitch example set, and carrying out iterative updating on the defect type label of each stitch in the preliminary detection result based on the defect propagation probability to obtain a defect detection result.
2. The method of claim 1, wherein obtaining a multi-view image sequence of the connector to be detected under the multi-light source illumination condition, solving to obtain three-dimensional morphology data based on pixel gray values of the same spatial position under different illumination angles, extracting edge pixel points with obvious surface curvature change from the three-dimensional morphology data, and obtaining candidate stitch areas according to general grouping comprises: sequentially lighting a plurality of light sources according to a preset time sequence, collecting a connector image to be detected by a camera when each light source is lighted, obtaining the connector images to be detected, the number of which is the same as that of the light sources, and combining the connector images to be detected to obtain a multi-view image sequence; extracting a plurality of pixel gray values corresponding to the pixel positions when different light sources are lightened according to each pixel position in the multi-view image sequence, substituting the incident angles of the pixel gray values and the corresponding light sources into a preset luminosity three-dimensional constraint equation to solve, obtaining a surface normal vector and reflectivity of a space point corresponding to each pixel position, and traversing all the pixel positions to obtain three-dimensional morphology data; calculating the direction change gradient in the horizontal direction and the vertical direction for the surface normal vector of each space point in the three-dimensional morphology data, and solving to obtain a surface curvature value; And marking the space points with the surface curvature value larger than the curvature threshold as candidate edge pixel points, calculating the space distance between any two candidate edge pixel points, classifying the candidate edge pixel points with the space distance smaller than the preset communication distance into the same communication group, extracting the space coordinates of all the candidate edge pixel points in the communication group for each communication group, performing convex hull calculation to obtain a minimum closed contour surrounding the communication group, taking the minimum closed contour as a boundary contour, and determining a candidate stitch region based on the boundary contour.
3. The method of claim 1, wherein the step of reserving stitch areas that meet a preset design specification after performing the spatial location verification on the candidate stitch areas, and wherein the step of assigning node identifiers to each stitch area and establishing a correlation edge between stitch areas that are spatially adjacent and in a coordinated posture comprises: calculating centroids corresponding to three-dimensional coordinates of all space points in the candidate stitch region to obtain center coordinates of the candidate stitch region, and carrying out principal component decomposition on boundary contours of the candidate stitch region to obtain principal direction vectors which are used as gesture directions of the candidate stitch region; Projecting the central coordinate to a horizontal reference plane of a space coordinate system to obtain projection coordinates, gridding the candidate stitch areas according to row and column positions of the projection coordinates to obtain two-dimensional array distribution, when the number of the candidate stitch areas in the grid positions is greater than one for each grid position in the two-dimensional array distribution, reserving the candidate stitch area with the nearest boundary contour area to the standard stitch area and the highest boundary contour integrity, removing other candidate stitch areas in the grid positions to obtain redundancy-removed stitch areas, calculating the difference absolute value between the central coordinate distance of the redundancy-removed stitch areas corresponding to adjacent grid positions in the two-dimensional array distribution and the preset stitch design distance as central distance deviation, and reserving the redundancy-removed stitch areas with the central distance deviation smaller than a position verification threshold as stitch areas conforming to design specifications; Generating node identifications for each stitch region according to row index and column index codes in two-dimensional array distribution, calculating an inner product value of gesture direction vectors of stitch regions corresponding to adjacent grid positions as gesture cooperative measurement, and establishing a correlation edge between adjacent stitch regions with gesture cooperative measurement larger than a cooperative threshold.
4. The method of claim 1, wherein propagating the node feature to a neighborhood node along the associated edge using the surface normal vector and the boundary contour corresponding to the stitch region as node features and aggregating neighborhood propagation features to obtain an optimized node feature comprises: Acquiring surface normal vectors of all space points in the boundary contour corresponding to the stitch region, performing statistical analysis to obtain a mean value vector and a variance value of the surface normal vectors, and splicing the mean value vector and the variance value to form a surface normal vector feature; Carrying out contour sampling on the boundary contour of the stitch region to obtain a contour key point sequence, calculating the space distance and angle change between adjacent key points in the contour key point sequence, splicing to obtain boundary contour features, and splicing the surface normal vector features and the boundary contour features to obtain node features corresponding to the stitch region; For each stitch region, searching node identifiers corresponding to all stitch regions connected with the current stitch region through an associated edge according to the node identifiers of the stitch regions to obtain a neighborhood node identifier, extracting node characteristics of the stitch regions corresponding to all neighborhood node identifiers to obtain a neighborhood node characteristic set, determining a weight value based on the associated edge, and carrying out weighted summation on the node characteristics in the neighborhood node characteristic set according to the weight value to obtain neighborhood propagation characteristics; And calculating the feature similarity between the node features and the neighborhood propagation features, determining an adaptive weight coefficient based on the feature similarity, and carrying out feature fusion on the node features corresponding to the stitch region and the neighborhood propagation features according to the adaptive weight coefficient to obtain optimized node features.
5. The method of claim 1, wherein determining a set of pin instances based on the optimized node features, extracting surface topography features of each pin instance in the set of pin instances from the three-dimensional topography data and stitching the surface topography features with the optimized node features to obtain a high-dimensional feature representation, and decoding the high-dimensional feature representation after performing a multi-scale convolution operation to obtain a preliminary detection result comprises: Calculating feature distances between every two optimized node features through a hierarchical clustering algorithm based on the feature similarity between the optimized node features, merging the optimized node features with the feature distances smaller than a preset initial clustering threshold into initial clusters, repeatedly merging different initial clusters with minimum inter-cluster distances until all inter-cluster distances are larger than the preset merging distance threshold, obtaining optimized clusters, and taking stitch areas corresponding to the optimized clusters as stitch examples; Three-dimensional morphology data in a space range surrounded by boundary contours in the stitch example are obtained to be used as morphology data blocks, direction distribution statistics is carried out on surface normal vectors in the morphology data blocks to obtain normal vector distribution histograms, height distribution statistics is carried out on space points in the morphology data blocks to obtain height distribution histograms, and the height distribution histograms and the normal vector distribution histograms are spliced to obtain surface morphology features; performing feature averaging on the optimized node features corresponding to the stitch region included in the stitch instance to obtain an aggregate optimized node feature, and splicing the aggregate optimized node feature with the surface morphology feature to obtain a high-dimensional feature representation; and performing multi-scale cavity convolution operation on the high-dimensional characteristic representation to obtain a plurality of adaptive characteristic diagrams with receptive field scales, performing channel splicing and decoding on the adaptive characteristic diagrams to obtain defect type prediction probability distribution and defect position prediction coordinates, and combining to obtain a preliminary detection result.
6. The method of claim 5, wherein performing a multi-scale hole convolution operation on the high-dimensional feature representation to obtain a plurality of adaptive feature maps with receptive field scales, performing channel stitching and decoding on the adaptive feature maps, and combining defect type prediction probability distribution and defect position prediction coordinates to obtain a preliminary detection result comprises: Inputting the high-dimensional characteristic representation into a plurality of preset hole convolution layers with increasing hole ratios to perform hole convolution operation to obtain initial characteristic diagrams with a plurality of receptive field scales, learning the offset of the pixel positions of each initial characteristic diagram through convolution operation, performing self-adaptive adjustment on the sampling positions of a convolution kernel according to the offset, and performing convolution operation again to obtain self-adaptive characteristic diagrams with a plurality of receptive field scales; Splicing the self-adaptive feature images in the channel dimension to obtain a spliced feature image, carrying out global average pooling and maximum pooling of the space dimension on the spliced feature image, calculating through a convolution layer to obtain a space attention weight image, and multiplying the space attention weight image with the spliced feature image pixel by pixel to obtain a space weighting feature image; And performing feature mapping on the space weighted feature map through the full-connection layer and the normalization layer to obtain defect type prediction probability distribution, performing feature mapping on the space weighted feature map through the full-connection layer and the regression activation layer to obtain defect position prediction coordinates, selecting the category with the maximum probability value from the defect type prediction probability distribution as a prediction defect type, and combining the prediction defect type and the defect position prediction coordinates to obtain a preliminary detection result.
7. The method of claim 1, wherein calculating a probability of defect propagation between each pair of spatially adjacent pin instances in the set of pin instances, and iteratively updating a defect type label for each pin in the preliminary detection result based on the probability of defect propagation comprises: For each stitch instance in the stitch instance set, calculating node association degree between the current stitch instance and other stitch instances in the stitch instance set according to aggregation optimization node characteristics corresponding to the current stitch instance, taking stitch instance pairs with node association degree larger than a preset association degree threshold as stitch instance pairs adjacent in space, acquiring center coordinates of stitch instances adjacent in space, calculating a space Euclidean distance, calculating a space attenuation factor based on the space Euclidean distance, and solving by combining the node association degree to obtain defect propagation probability between the stitch instances adjacent in space; Acquiring all the space adjacent stitch examples corresponding to the stitch examples and the defect propagation probability corresponding to the space adjacent stitch examples, calculating to obtain an external propagation confidence coefficient vector, taking the defect type prediction probability distribution corresponding to the current stitch example as an internal detection confidence coefficient vector, and carrying out weighted fusion on the external propagation confidence coefficient vector according to a preset fusion proportion to obtain a fusion confidence coefficient distribution, and updating defect type labels in the preliminary detection result based on the fusion confidence coefficient distribution; Repeating updating the defect type labels of all the stitch examples in the stitch example set until the defect type labels of all the stitch examples are kept unchanged in continuous iteration or reach preset iteration times, and combining the defect type labels obtained in the last iteration with the defect position prediction coordinates to obtain a defect detection result.
8. A machine vision based connector pin defect intelligent detection system for implementing the method of any of the preceding claims 1-7, comprising: The stitch region construction unit is used for acquiring a multi-view image sequence of the connector to be detected under the multi-light source illumination condition, solving and obtaining three-dimensional morphology data based on pixel gray values of the same spatial position under different illumination angles, extracting edge pixel points with obvious surface curvature change from the three-dimensional morphology data, obtaining candidate stitch regions according to general grouping, checking the spatial position of the candidate stitch regions, reserving stitch regions which meet preset design specifications, distributing node identifiers for each stitch region, and establishing association edges between stitch regions which are spatially adjacent and have collaborative gestures; The feature optimization detection unit is used for taking the surface normal vector and the boundary contour corresponding to the stitch region as node features, transmitting the node features to a neighborhood node along the associated edge, aggregating the neighborhood transmission features to obtain optimized node features, determining a stitch instance set based on the optimized node features, extracting the surface topography features of each stitch instance in the stitch instance set from the three-dimensional topography data, splicing the surface topography features with the optimized node features to obtain a high-dimensional feature representation, and decoding the high-dimensional feature representation after multi-scale convolution operation to obtain a primary detection result; The defect label propagation unit is used for calculating defect propagation probability between each pair of stitch examples which are adjacent in space in the stitch example set, and carrying out iterative updating on the defect type label of each stitch in the preliminary detection result based on the defect propagation probability to obtain a defect detection result.
9. An electronic device, comprising: A processor; A memory for storing processor-executable instructions; Wherein the processor is configured to invoke the instructions stored in the memory to perform the method of any of claims 1 to 7.
10. A computer readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the method of any of claims 1 to 7.

Description

Intelligent detection method and system for pin defects of connector based on machine vision Technical Field The invention relates to the technical field of machine vision, in particular to an intelligent detection method and system for pin defects of a connector based on machine vision. Background As an important component in electronic devices, the quality directly affects the reliability and the lifetime of the electronic device. Connector pins are key components in the connector and are responsible for transmission of electrical signals, and defects such as deformation, fracture, deflection and the like may occur in the manufacturing process. With the development of miniaturization and high integration of electronic products, the size of pins of a connector is smaller and smaller, the spacing is denser and denser, and higher requirements are put forward on the pin defect detection technology. Currently, connector pin defect detection relies primarily on manual visual inspection and conventional machine vision techniques. The manual detection efficiency is low, and the requirement of mass production is difficult to meet. The traditional machine vision technology is mainly based on two-dimensional image analysis, and generally adopts an image acquired under a single light source illumination condition to process, and the prior art still has the problems that the acquired image cannot completely reflect the three-dimensional geometric characteristics of pins, the defects at certain specific angles are easy to cause missed detection, the consideration of the spatial correlation between the pins is lacking, the composite defects caused by the mutual influence of adjacent pins are difficult to effectively identify, the sensitivity to the tiny defects on the surfaces of the pins is insufficient, and particularly in a high-density connector, the tiny pin bending or surface scratches are difficult to accurately detect. Disclosure of Invention The embodiment of the invention provides a connector pin defect intelligent detection method and system based on machine vision, which at least can solve part of problems in the prior art. In a first aspect of the embodiment of the present invention, an intelligent detection method for pin defects of a connector based on machine vision is provided, including: acquiring a multi-view image sequence of a connector to be detected under a multi-light source illumination condition, solving pixel gray values of the same spatial position under different illumination angles to obtain three-dimensional morphology data, extracting edge pixel points with obvious surface curvature change from the three-dimensional morphology data, grouping according to generality to obtain candidate stitch areas, performing spatial position verification on the candidate stitch areas, reserving stitch areas meeting preset design specifications, distributing node identifiers for each stitch area, and establishing a correlation edge between stitch areas which are spatially adjacent and have collaborative postures; Taking the surface normal vector and boundary contour corresponding to the stitch region as node characteristics, transmitting the node characteristics to a neighborhood node along the associated edge, aggregating the neighborhood transmission characteristics to obtain optimized node characteristics, determining a stitch instance set based on the optimized node characteristics, extracting the surface morphology characteristics of each stitch instance in the stitch instance set from the three-dimensional morphology data, splicing the surface morphology characteristics with the optimized node characteristics to obtain a high-dimensional characteristic representation, performing multi-scale convolution operation on the high-dimensional characteristic representation, and then decoding to obtain a preliminary detection result; And calculating defect propagation probability between each pair of stitch examples adjacent to each other in the stitch example set, and carrying out iterative updating on the defect type label of each stitch in the preliminary detection result based on the defect propagation probability to obtain a defect detection result. In an alternative embodiment of the present invention, Acquiring a multi-view image sequence of a connector to be detected under a multi-light source illumination condition, solving based on pixel gray values of the same spatial position under different illumination angles to obtain three-dimensional morphology data, extracting edge pixel points with obvious surface curvature change from the three-dimensional morphology data, and grouping according to the universality to obtain candidate stitch areas, wherein the method comprises the following steps: sequentially lighting a plurality of light sources according to a preset time sequence, collecting a connector image to be detected by a camera when each light source is lighted, obtaining the connector images t