Search

CN-121010811-B - High-precision image detection method for micro-drilling cutting edge surface based on deep learning

CN121010811BCN 121010811 BCN121010811 BCN 121010811BCN-121010811-B

Abstract

The invention discloses a high-precision image detection method of a micro-drilling cutting edge surface based on deep learning, which comprises the following steps of S1, collecting a visible light image and a structural light image of the micro-drilling cutting edge surface, finishing image preprocessing, S2, performing space alignment on the images, performing cross-modal fusion to generate a feature fusion tensor, S3, inputting the feature fusion tensor into a multi-scale residual trunk network, extracting a layered semantic feature set, S4, inputting the semantic feature set into three task branches of defect detection, region segmentation and type classification, outputting a corresponding prediction result, S5, calculating a multi-task loss function, dynamically adjusting task branch weight, optimizing a feature sharing structure, S6, generating a detection report according to the prediction result, and outputting a defect coordinate, a boundary contour, a type label and a confidence value. The invention realizes the multi-mode fusion, high-precision identification and structured output of the micro-drill blade surface, and remarkably improves the accuracy, efficiency and automation level of defect detection.

Inventors

  • LI MINHUA
  • LI GANGQIN

Assignees

  • 深圳宏友金科技有限公司

Dates

Publication Date
20260512
Application Date
20250806

Claims (10)

  1. 1. The high-precision image detection method of the micro-drill cutting edge surface based on deep learning is characterized by comprising the following steps of: S1, collecting visible light images and structured light images of the micro-drill blade surface, and preprocessing; S2, performing spatial alignment on the preprocessed visible light image and the preprocessed structured light image, performing cross-mode fusion operation based on channel feature matching, and performing channel level alignment on the information of the corresponding pixel region to obtain a feature fusion tensor; S3, inputting the feature fusion tensor into a backbone neural network, wherein the backbone neural network adopts a multi-scale residual error structure to perform feature extraction on the feature tensor of the fusion image, and outputs a layered semantic feature set; S4, inputting the layered semantic feature set into three task branch networks, wherein the three task branch networks are respectively used for executing edge face defect detection, edge face region segmentation and defect type classification, and each task branch network extracts a feature subset and outputs a corresponding prediction result; S5, jointly calculating a multi-task loss function according to the feature subsets, adjusting weight parameters of each task branch network, and optimizing feature sharing structures between the backbone neural network and each task branch network; and S6, generating a detection report of the micro-drilling cutting edge surface according to the prediction result, and marking the defect position coordinates, the regional boundary outline, the defect type label and the corresponding confidence value.
  2. 2. The high-precision image detection method of the micro-drilling blade surface based on the deep learning according to claim 1, wherein the visible light image is a two-dimensional image of the micro-drilling blade surface acquired under a natural illumination condition by an industrial camera, and the structured light image is a three-dimensional morphology image of the micro-drilling blade surface acquired under a specific coding illumination condition by using a structured light projection device.
  3. 3. The method of claim 1, wherein the preprocessing comprises image distortion correction, gray scale normalization, pixel level alignment, boundary cropping, contrast stretching, edge sharpening, and noise filtering.
  4. 4. The high-precision image detection method of the micro-drilling blade face based on deep learning according to claim 1, wherein the step S2 specifically comprises: S21, setting the preprocessed visible light image as a two-dimensional image, and setting the structured light image as a three-dimensional image containing depth gray information, wherein the image coordinates contain a horizontal pixel position and a longitudinal pixel position, and the structured light image also contains a gray value of each pixel; S22, carrying out space coordinate registration on the two-dimensional image and the three-dimensional image by adopting an affine transformation mode, respectively executing transformation, ensuring that corresponding pixels are consistent in space position, and outputting a pair of images with aligned pixels; s23, respectively extracting a plurality of channel feature vectors in the two-dimensional image and the three-dimensional image, and determining an optimal channel corresponding relation by calculating feature similarity between different channels, wherein the feature similarity is obtained by comprehensively calculating according to directions and module lengths between channel features; And S24, performing splicing fusion operation on the channel characteristics with high matching degree according to the channel dimension, and generating a characteristic fusion tensor containing multi-source image information.
  5. 5. The high-precision image detection method of the micro-drill cutting edge surface based on deep learning according to claim 4, wherein the optimal channel corresponding relation is a channel matching relation established based on a similarity matrix between a visible light image channel feature vector and a structured light image channel feature vector, a channel matching score matrix is constructed by adopting a similarity calculation result, a maximum matching strategy is executed in the channel matching score matrix, a channel pair with the highest similarity score is selected as a matching result, a channel index pair set is output, and channel dimensions of a visible light image and a structured light image are subjected to fusion arrangement according to the channel index pair set.
  6. 6. The high-precision image detection method of the micro-drilling blade face based on deep learning according to claim 1, wherein the step S3 specifically comprises: s31, inputting a characteristic fusion tensor into a backbone neural network, wherein three dimensions of the characteristic tensor correspond to a transverse pixel position, a longitudinal pixel position and a fusion channel dimension of an image respectively; S32, setting a multi-scale structure formed by a plurality of residual units in a backbone neural network, sequentially receiving the characteristic output of the previous unit by each residual unit, performing convolution transformation, batch normalization and nonlinear activation function combination operation, and adding the input characteristic into the output characteristic through identical jump connection to form a short connection structure; s33, performing downsampling processing on the feature images output by each residual unit to extract semantic features under different receptive field scales, so as to form a multi-level feature image set; and S34, splicing the feature graphs output by all residual units in the channel dimension to form a layered semantic feature set, and transmitting the layered semantic feature set to a task branch network.
  7. 7. The high-precision image detection method of the micro-drill cutting edge surface based on deep learning according to claim 6, wherein the layered semantic feature set refers to a tensor set formed by sequentially arranging feature images output by a plurality of scale levels according to semantic granularity, and each layer of feature images corresponds to image semantic information under different receptive fields.
  8. 8. The high-precision image detection method of the micro-drilling blade face based on deep learning according to claim 1, wherein the step S4 specifically comprises: s41, representing the layered semantic feature set as a three-dimensional tensor, and inputting the three-dimensional tensor to a defect detection branch network, a cutting edge area segmentation branch network and a defect type classification branch network in parallel along a channel dimension, wherein each branch extracts an independent feature subset to perform task calculation; s42, extracting a defect feature subset from the three-dimensional tensor by the defect detection branch network, obtaining a low-dimensional feature tensor through a channel compression convolution structure, and outputting a probability value of whether each pixel is a defect or not according to channel weighted summation; S43, extracting a blade face feature subset from a three-dimensional tensor by a blade face region segmentation branch network, performing multi-scale up-sampling operation on the blade face feature subset, generating up-sampling feature images under three scales, splicing all scale feature images in channel dimensions to form a fusion tensor, and outputting region category distribution of each pixel through a classification convolution layer; S44, extracting a type feature subset from the three-dimensional tensor by the defect type classification branch network, carrying out full-graph average pooling operation on the type feature subset, extracting a channel-level global feature vector, inputting the channel-level global feature vector into the full-connection neural network, calculating probability distribution of each defect type, and outputting a defect classification prediction result.
  9. 9. The high-precision image detection method of the micro-drilling blade face based on deep learning according to claim 1, wherein the step S5 specifically comprises: S51, respectively calculating loss functions of corresponding tasks according to output results of three task branches including defect detection, edge face region segmentation and defect type classification, wherein the defect detection tasks adopt pixel-level binary cross entropy loss, the region segmentation tasks adopt multi-class cross entropy loss, the defect classification tasks adopt single-label multi-class cross entropy loss, and each loss function is calculated pixel by pixel or class by class according to real labels and prediction results; S52, distributing an adjustable weight coefficient for each task branch, constructing a joint multi-task loss function by weighting and combining the loss functions of three tasks, wherein the sum 1 of all weight coefficients is self-adaptively adjusted according to task gradient distribution in the training process, and ensuring the balance of learning effects of each task; S53, carrying out back propagation operation based on the joint multitask loss function, simultaneously optimizing the parameters of the backbone neural network and the parameters of the task branch network, maintaining the bottom layer universality in the feature transfer structure, realizing the separation sharing strategy of the task specific features at a middle-high layer, and optimizing the learning efficiency of the whole network structure through an end-to-end parameter updating mechanism.
  10. 10. The high-precision image detection method of the micro-drilling blade face based on deep learning according to claim 1, wherein the step S6 specifically comprises: S61, binarizing pixel-level defect probability outputted by a defect detection task branch, extracting a high-confidence pixel region as a defect region mask by combining a set threshold value, acquiring a boundary minimum bounding rectangle of all independent defect regions through connected region analysis, and extracting two-dimensional coordinate information of a corresponding position as a defect position coordinate; S62, intersecting the segmentation probability predicted value output by the cutting edge region segmentation task branch with the defect region mask, further extracting structural category information of the defect region, performing polygon fitting operation on the outer contour of each defect region by adopting a contour tracking algorithm, and outputting a closed region boundary contour formed by a continuous boundary point set; S63, selecting a maximum probability index value to correspond to a defect label set as a defect type label according to a defect type prediction vector output by a defect type classification task branch, and combining the maximum probability index value as a confidence value with a defect position coordinate and a boundary contour to form a structured description vector; S64, writing the structured description vectors of all defects into a detection report text file according to a preset format, wherein the report content comprises defect numbers, position coordinates, attribution of structural areas, boundary contour point sets, defect type labels and corresponding confidence values, and carrying out index sorting according to image numbers.

Description

High-precision image detection method for micro-drilling cutting edge surface based on deep learning Technical Field The invention relates to the technical field of image detection, in particular to a high-precision image detection method of a micro-drill cutting edge surface based on deep learning. Background In the fields of high-density circuit board processing, precise hole processing, micro part manufacturing and the like, a micro drill is used as a key cutting tool, and the cutting face integrity directly determines the processing quality and the cutter service life. At present, the industrial site mainly relies on manual visual inspection or a traditional image algorithm to detect defects on the edge surface of the micro drill, and the manual method is limited by experience of operators, so that the problems of strong subjectivity, low efficiency, high false detection rate and the like exist, and the traditional visual algorithm based on edge detection and gray level analysis is difficult to deal with accurate identification of complex background, micro cracks and high reflective surfaces. In the prior art, an attempt is made to automatically classify micro-drilling images by using a convolutional neural network, but most of the prior art is based on a single visible light image, and ignores the three-dimensional structural characteristics of the surface morphology of the cutting edge, so that the recognition performance is obviously reduced under the conditions of crack blurring, insignificant cutting edge damage and the like. Meanwhile, most of the existing models only realize defect detection functions, cannot jointly realize cutting edge face segmentation and defect type identification, lack a multi-task learning framework with complete structure, and cannot meet comprehensive requirements of industrial on-line detection on detection precision, detection efficiency and detection information integrity. In addition, the micro-drill blade image is easily affected by illumination change, metal reflection and micro-texture interference in the acquisition process, an image preprocessing algorithm usually adopts a fixed parameter model, and the original characteristic information is lost due to the lack of a self-adaptive preprocessing flow aiming at the micro-drill working condition, so that the model discrimination capability is affected. Meanwhile, the multi-task deep learning network has the problem of inter-task interference in the aspect of feature sharing structure design, and a negative optimization phenomenon that the precision of one task is improved and the performance of the other task is reduced is easy to occur. Therefore, how to provide a high-precision image detection method of a micro-drilling blade surface based on deep learning is a problem to be solved by those skilled in the art. Disclosure of Invention The invention aims to provide a high-precision image detection method of a micro-drill cutting edge surface based on deep learning, which fully fuses visible light image and structured light image information, constructs a multi-scale residual error trunk network and a multi-task branch structure, and details the whole process from image preprocessing, cross-mode fusion, feature extraction, defect positioning to result output, and has the advantages of high detection precision, full identification type and strong capability of adapting to industrial complex scenes. The high-precision image detection method of the micro-drilling cutting edge surface based on the deep learning comprises the following steps of: S1, collecting visible light images and structured light images of the micro-drill blade surface, and preprocessing; S2, performing spatial alignment on the preprocessed visible light image and the preprocessed structured light image, performing cross-mode fusion operation based on channel feature matching, and performing channel level alignment on the information of the corresponding pixel region to obtain a feature fusion tensor; S3, inputting the feature fusion tensor into a backbone neural network, wherein the backbone neural network adopts a multi-scale residual error structure to perform feature extraction on the feature tensor of the fusion image, and outputs a layered semantic feature set; S4, inputting the layered semantic feature set into three task branch networks, wherein the three task branch networks are respectively used for executing edge face defect detection, edge face region segmentation and defect type classification, and each task branch network extracts a feature subset and outputs a corresponding prediction result; S5, jointly calculating a multi-task loss function according to the feature subsets, adjusting weight parameters of each task branch network, and optimizing feature sharing structures between the backbone neural network and each task branch network; and S6, generating a detection report of the micro-drilling cutting edge surface according to the