Search

CN-122023318-A - Metal product surface defect detection method and system based on machine vision

CN122023318ACN 122023318 ACN122023318 ACN 122023318ACN-122023318-A

Abstract

The invention discloses a method and a system for detecting surface defects of metal products based on machine vision, which are characterized in that three-dimensional geometric and two-dimensional texture information is synchronously acquired through a structured light and multispectral composite sensor, clear initial defect data is obtained through a reflection separation technology based on polarization degree, precise positioning of sub-pixel level defect boundaries is realized through multi-modal data alignment and a contour enhancement technology based on an attention mechanism, multitask intelligent classification is further carried out by fusing contour, geometric and spectral characteristics, fine differentiation and quantitative description of defects are completed, a high-fidelity three-dimensional digital model of key defects is constructed through a nerve radiation field technology, self-optimization of the model is realized by combining view difference and generative complementation, multidimensional quantitative indexes and a process knowledge rule base are automatically matched, a structured detection report is generated, full-flow intelligentization from perception, analysis and reconstruction to decision is realized, and the accuracy, efficiency and automation level of industrial quality control are remarkably improved.

Inventors

  • ZHANG YUMIN
  • LIU MEIQI

Assignees

  • 广州市和友模塑科技有限公司

Dates

Publication Date
20260512
Application Date
20260127

Claims (10)

  1. 1. A method for detecting surface defects of a metal product based on machine vision is characterized by comprising the following steps: s1, synchronously acquiring three-dimensional point cloud and multiband images through a composite sensor, and obtaining preliminary defect space position and suspected category information through multitasking neural network processing; S2, decomposing the image data in the preliminary positioning area into a diffuse reflection component and a specular reflection component, decomposing the image data into a diffuse reflection component and a specular reflection component, and respectively performing targeted filtering processing to obtain image data for inhibiting specular interference; S3, performing space coordinate alignment on the filtered diffuse reflection component image and the three-dimensional point cloud data, calculating the consistency measurement of the features of the diffuse reflection component image and the three-dimensional point cloud data, and if the consistency measurement is lower than a first set threshold, starting a contour enhancement network based on a coordinate attention mechanism, and outputting enhanced defective sub-pixel level contour data; s4, inputting the enhanced defect sub-pixel level profile data, the corresponding three-dimensional local point cloud and the spectral feature vector extracted from the diffuse reflection component image into a defect classification network for processing to obtain a classified defect type label; S5, constructing a local nerve radiation field model by taking the multispectral image and the corresponding three-dimensional point cloud under the multi-view angle as training data for the classified defect type labels, and generating a high-resolution three-dimensional defect model fused with spectral reflection characteristics; And S6, performing differential comparison on the new view generated by rendering the local nerve radiation field model and the original view, triggering an image complement network to complement if the differential value exceeds a second set threshold value, and reversely optimizing and updating the local nerve radiation field model to obtain complete defect multi-mode description data.
  2. 2. The method for detecting surface defects of a metal product based on machine vision as set forth in claim 1, wherein the step S1 comprises the following steps: The three-dimensional point cloud is subjected to denoising and voxelization treatment to obtain a voxel grid, and the multiband image is subjected to radiation correction and registration to form a standardized multichannel image; Extracting geometric structural features from the voxel grid input point cloud coding branches, extracting weighted texture spectrum features from the multichannel image input image coding branches, and generating a shared feature representation by fusing the two features through a cross attention module; based on the shared features, generating a defect probability voxel heat map through three-dimensional convolution decoding head regression to determine a space voxel region of the defect, and outputting suspected defect types through a classification decoding head; and mapping the space voxel region to an actual workpiece coordinate system according to the sensor calibration parameters, associating the three-dimensional space position with the suspected defect category, and outputting a preliminary defect description.
  3. 3. The method for detecting surface defects of a metal product based on machine vision as set forth in claim 1, wherein the step S2 comprises the following steps: calculating the polarization degree of each pixel point from the corresponding multispectral image aiming at the defect area of preliminary positioning, and decomposing the area pixels into diffuse reflection components and specular reflection components by using a Gaussian mixture model according to the statistical distribution of the polarization degree; Aiming at the separated specular reflection component image, calculating pixel variances of local areas of the specular reflection component image, dynamically adjusting non-local mean filtering parameters according to the variances, and carrying out key smooth suppression on areas with high variances; Applying anisotropic diffusion filtering to the separated diffuse reflection component image, controlling diffusion intensity according to image gradient to smooth noise and protect edge information; and carrying out pixel weighted fusion on the two groups of processed images, carrying out self-adaptive histogram equalization processing on the fused images, and outputting clear multi-mode image data with suppressed reflection interference.
  4. 4. The method for detecting surface defects of a metal product based on machine vision as set forth in claim 1, wherein the step S3 comprises the following steps: carrying out strict spatial coordinate alignment on the diffuse reflection image and the three-dimensional point cloud by utilizing the composite sensor parameters to form a pixel-point cloud aligned data pair; In the preliminary defect area, texture gradient features are extracted from the aligned images respectively, surface normal or curvature features are extracted from the point cloud, and cosine similarity of two feature vectors is calculated to be used as a consistency measurement value; Comparing the consistency metric value with a preset threshold value, triggering a contour enhancement network based on a coordinate attention mechanism if the consistency metric value is lower than the threshold value, taking the aligned image block and local point cloud characteristics as input, and focusing on a characteristic conflict area through the attention mechanism; The contour enhancement network outputs a contour probability map of a sub-pixel level, and extracts accurate defect boundary pixel coordinates through non-maximum suppression and a sub-pixel interpolation technology to generate enhanced defect sub-pixel level contour data.
  5. 5. The method for detecting surface defects of a metal product based on machine vision as set forth in claim 1, wherein the step S4 comprises the following steps: Calculating shape descriptors for the outline data of the defect sub-pixels to generate outline feature vectors, calculating surface geometric statistical features for the three-dimensional local point cloud to generate geometric feature vectors, calculating multiband spectral statistical features for the corresponding areas of the diffuse reflection images to generate spectral feature vectors, and splicing the three to form comprehensive feature vectors; inputting the comprehensive feature vector into a shared depth feature extraction backbone network, and fusing advanced feature representation of multi-source information with the depth abstract output through nonlinear transformation; the defect classification network comprises three parallel classification heads, wherein the first classification head outputs probability distribution of macroscopic categories of defects, the second classification head outputs continuous numerical microscopic morphological attributes, and the third classification head outputs discrete size grades; and integrating output results of the three classification heads, determining final macroscopic categories, quantized morphological attributes and size intervals, and generating a structured multi-dimensional defect type label.
  6. 6. The method for detecting surface defects of a metal product based on machine vision as set forth in claim 1, wherein the step S5 comprises the following steps: according to the space position of the classified defects, a multispectral image subregion and a three-dimensional point cloud subset corresponding to the defects are extracted from the original multiview data, and a multimodal training data set centering on the defects is constructed through space registration and scale normalization; Defining a boundary cube by using a defect space position, initializing a nerve radiation field model, performing iterative training by using a training data set through a differential rendering technology, taking the difference between a minimum rendering pixel and a real pixel as a target, and simultaneously taking a point cloud geometric position as a space constraint; after training, taking the optimized model as a static query function, and synthesizing a high-resolution multispectral new view and a corresponding accurate depth map under any target visual angle by rendering and integrating the continuous fields; based on the synthesized multi-view depth map or directly extracting an isosurface from the density field, generating a triangular mesh model of the defect surface, carrying out association mapping on the vertex and the spectrum attribute predicted by the nerve radiation field, and finally outputting a three-dimensional defect digital model fused with the geometric and spectrum reflection characteristics.
  7. 7. The method for detecting surface defects of a metal product based on machine vision as set forth in claim 1, wherein the step S6 comprises the following steps: generating a synthesized view for each original acquisition view angle rendering by using a local nerve radiation field model, and calculating pixel-by-pixel absolute difference values of the synthesized view and the processed original view on each channel of RGB or multispectral; Carrying out connected domain analysis on the differential result image, marking the connected region with the pixel average differential value continuously exceeding a second threshold value as a feature missing region, and recording the position coordinates and the minimum circumscribed rectangular range of the connected region; inputting the feature loss area and the context image of the feature loss area into a pre-trained generation anti-complement model by taking the original view as a real reference, and generating coordinated complement content according to the context semantics and the texture information to obtain a repaired view; and taking the repaired view as a new training sample, carrying out back propagation optimization on the local nerve radiation field model together with the original data, and iteratively updating model parameters until the average difference between the rendered view and the original/repaired view meets the accuracy requirement, and finally integrating to obtain a complete defect multi-mode description data set.
  8. 8. The method for detecting surface defects of metal products based on machine vision as set forth in claim 1, wherein S7 is characterized in that multi-dimensional quantitative indexes are extracted according to the complete defect multi-mode description data and matched with a predefined rule base, and a structured detection report containing defect codes, severity levels and maintenance suggestions is automatically generated.
  9. 9. The method for detecting surface defects of a metal product based on machine vision as set forth in claim 8, wherein the step S7 comprises the following steps: Extracting and calculating quantization indexes of five dimensions from the complete defect multi-mode description data, wherein the quantization indexes comprise defect types, three-dimensional dimensions representing space occupation of the defect types, maximum depth reflecting severity of the defect types, contour sharpness describing edge definition of the defect types, and spectral reflection anomaly representing material anomaly of the defect types; Automatically matching and logically reasoning the quantization index set with a predefined rule base containing material standards and process knowledge; and automatically outputting a corresponding defect code, severity level and structural detection report of maintenance suggestion according to the matching result.
  10. 10. A machine vision-based metal product surface defect detection system, which is applied to the machine vision-based metal product surface defect detection method as set forth in any one of claims 1 to 9, and is characterized by comprising the following steps: the multi-mode data acquisition and positioning module is used for synchronously acquiring three-dimensional point cloud and multi-band texture images and carrying out preliminary defect positioning and classification through a multi-task neural network; the reflection interference suppression and image enhancement module is used for decomposing reflection components based on polarization information and performing targeted filtering to eliminate the highlight interference on the metal surface; The multi-mode contour refinement and enhancement module is used for obtaining a defect boundary with sub-pixel level precision through feature consistency verification and contour enhancement of a two-dimensional image and a three-dimensional point cloud; The multi-dimensional intelligent classification and quantification module is used for fusing multi-source characteristics and outputting a structured defect label containing macroscopic category, microscopic attribute and size grade through a multi-task classification network; the three-dimensional defect reconstruction module is used for constructing a defect three-dimensional model fusing geometric and spectral reflection characteristics based on a nerve radiation field technology; the model self-optimization and data complement module is used for obtaining complete defect multi-mode description data through view difference, feature complement and model reverse optimization; And the knowledge driving decision and report generating module is used for automatically matching the defect quantification index according to the rule base to generate a structured detection report.

Description

Metal product surface defect detection method and system based on machine vision Technical Field The invention relates to the technical field of industrial quality detection and intelligent manufacturing, in particular to a method and a system for detecting surface defects of a metal product based on machine vision. Background The detection of the surface defects of the metal products is an indispensable ring in the manufacturing quality control, the importance of the detection is that the detection is directly related to the safety and the service life of the products, especially in the fields of high requirements of automobiles, aerospace and the like, the serious consequences can be caused by the tiny defects, the research in the field has key significance for improving the industrial production efficiency and guaranteeing the product quality, however, when facing complex environments, many detection methods at present often have the defects, such as strong reflection plaques formed on the surface of a sheet metal part under strong light illumination on a white body detection line before the spraying of an automobile body, the real scratches or pits are easily covered, and especially under the interference of the metal surface caused by light reflection, the misjudgment easily occurs, and meanwhile, the differentiation capability for various defect types is insufficient, so that the detection method is difficult to adapt to diversified production requirements. In-depth analysis can find that the detection difficulty of the surface of the metal product mainly comes from two core technical factors, namely, the comprehensive defect of the acquired information is firstly caused, the reflection characteristic difference of the light rays of the metal surface is huge due to different materials and shapes, the fine texture change and the three-dimensional morphology information are difficult to capture by a single acquisition mode, for example, on the casting surface of an engine cylinder body, the deep-colored oxide subsurface cracks and the local protrusions which are blurred due to reflection are difficult to accurately acquire by a common camera, and the defect further causes another key problem that when the data are processed, the information from different sources such as the fact that the matching between a plane image and a depth profile often deviates, and the defect characteristic is lost or misread is difficult to fuse. Particularly, on an actual production line, the detection equipment may miss tiny cracks or scratches because the detection equipment cannot accurately adapt to the reflective characteristics of the metal surface, and meanwhile, even if various data are collected, the judgment on the defect type is fuzzy due to the lack of effective integration means, for example, the scratches are mistakenly regarded as pits, so that the subsequent repair decision is influenced. Therefore, how to realize the accurate collection of various information on the metal surface in a complex environment, effectively integrate different types of data to accurately distinguish various defects, and become a key problem for improving the detection effect, and the solution of the problem directly influences the quality control and the efficiency improvement in the production process. Disclosure of Invention The invention aims to provide a metal product surface defect detection method and system based on machine vision, which realize complete automatic closed loop from accurate detection to intelligent decision. The aim of the invention can be achieved by the following technical scheme: The application provides a metal product surface defect detection method based on machine vision, which comprises the following steps: s1, synchronously acquiring three-dimensional point cloud and multiband images through a composite sensor, and obtaining preliminary defect space position and suspected category information through multitasking neural network processing; S2, decomposing the image data in the preliminary positioning area into a diffuse reflection component and a specular reflection component, decomposing the image data into a diffuse reflection component and a specular reflection component, and respectively performing targeted filtering processing to obtain image data for inhibiting specular interference; S3, performing space coordinate alignment on the filtered diffuse reflection component image and the three-dimensional point cloud data, calculating the consistency measurement of the features of the diffuse reflection component image and the three-dimensional point cloud data, and if the consistency measurement is lower than a first set threshold, starting a contour enhancement network based on a coordinate attention mechanism, and outputting enhanced defective sub-pixel level contour data; s4, inputting the enhanced defect sub-pixel level profile data, the corresponding three-dimensional local point cloud and the