CN-121997279-A - Visual sense and touch sense fused sensing metal part surface crack detection method

CN121997279ACN 121997279 ACN121997279 ACN 121997279ACN-121997279-A

Abstract

The invention discloses a visual touch sense fusion-perceived metal part surface crack detection method which comprises the steps of collecting images of a metal part surface to be detected through an RGB-D camera, carrying out self-adaptive image enhancement and denoising pretreatment on the collected images to obtain pretreated images, extracting visual geometric feature vectors of cracks from the pretreated images, locating the images to a crack area through a touch sensor, collecting 3D morphology, displacement field and distribution force data of the cracks, calculating touch sense measurement depth of the cracks based on a mechanical model, constructing touch feature vectors based on the touch sense measurement depth, carrying out multi-scale alignment on the visual geometric feature vectors and the touch sense feature vectors through wavelet transformation, carrying out feature level fusion through a return neural network optimization algorithm to obtain fusion features, carrying out crack detection through a machine learning classifier based on the fusion features, and outputting crack size, position and confidence. The invention can improve the accuracy and reliability of detection.

Inventors

Giuseppe Karl Bona
FU YUANHANG
XIAO QIAN
Han Kunya
SHI TIANHAO
ZENG DEQUAN
HU YIMING
YANG JINWEN

Assignees

华东交通大学

Dates

Publication Date: 20260508
Application Date: 20260409

Claims (8)

1. The visual sense and touch sense fusion sensing method for detecting the surface cracks of the metal part is characterized by comprising the following steps of: step S1, acquiring an image of the surface of a metal part to be detected through an RGB-D camera, and carrying out self-adaptive image enhancement and denoising pretreatment on the acquired image to obtain a pretreated image; s2, extracting visual geometric feature vectors of the cracks from the preprocessed image, wherein the visual geometric feature vectors comprise the length, the width, the visual depth and the visual depth statistical distribution characteristics of the cracks; S3, positioning to a crack area through a touch sensor, collecting 3D morphology, displacement field and distribution force data of the crack, calculating the touch measurement depth of the crack based on a mechanical model, and constructing a touch feature vector based on the touch measurement depth; s4, performing multi-scale alignment on the visual geometric feature vector and the tactile feature vector by adopting wavelet transformation, and performing feature level fusion by using a return-to-zero neural network optimization algorithm to obtain fusion features; And S5, performing crack detection by using a machine learning classifier based on the fusion characteristics, and outputting the size, the position and the confidence of the crack.
2. The method for detecting surface cracks of a metal part according to claim 1, wherein in step S1, adaptive image enhancement uses a limited contrast adaptive histogram equalization algorithm, a contrast limiting threshold In order to take the value range of 2.0-4.0, the size of the blocking grid is Is that ; In step S1, denoising pretreatment is carried out by a median filtering algorithm, a filtering window of 5 multiplied by 5 is adopted, and the denoising is realized by endowing the center pixel with the median of the pixel gray value sequence in the window.
3. The method for detecting surface cracks of a metal part according to claim 2, wherein in step S2, the visual geometric feature vector is The method comprises the following steps: Wherein, the For the length of the crack to be a crack, For the width of the crack to be a crack width, For the visual depth of the crack, As the visual depth variance of the crack, As the standard deviation of the visual depth of the crack, Representing the transpose.
4. The visual sense and tactile sense fusion sensing method for detecting surface cracks of a metal part according to claim 3, wherein in the step S3, a calculation formula of a tactile measurement depth of the cracks is: Wherein, the The depth is measured for the sense of touch of the crack, As a total number of samples, Represent the first The height position at the time of sub-sampling, In order to achieve the initial contact position, In order for the force of the contact to be such, Is a calibrated system stiffness coefficient.
5. The method for detecting cracks on a surface of a metal part according to claim 4, wherein in step S3, the haptic feature vector is The expression of (2) is: Wherein, the 、、 The average values of contact forces in the x, y and z directions are respectively, 、、 The variances of the contact forces in the x, y and z directions are respectively, 、、 The peak values of the contact force in the x, y and z directions are respectively shown.
6. The visual sense and tactile sense fusion sensing method for detecting surface cracks of a metal part according to claim 5, wherein in the step S4, attention mechanism weighting is introduced when feature level fusion is performed through a return-to-zero neural network optimization algorithm, and a weight calculation formula is as follows: Wherein, the Is the first in the fusion characteristics Features of Is used for the concentration weight of the person, Is the first in the fusion characteristics Features of Is characterized by the feature score of (c), Is the first in the fusion characteristics Features of Is characterized by the feature score of (c), Is the total dimension of the feature vector before fusion.
7. The method for detecting surface cracks of a metal part through visual sense and touch fusion perception according to claim 5, wherein in the step S4, when a return-to-zero neural network optimization algorithm performs feature level fusion, the following objective function is dynamically solved: Wherein, the The representation takes the minimum value of the value, In order to regularize the super-parameters, Relates to fusion parameters to be optimized Is a regularization function of (2); Fusion parameters Satisfies the following formula: Wherein, the Is that Is a first-order derivative of (a), Indicating the deviation of the deflection of the beam, Is dependent on parameters Is used as a scalar target function of (a), Representing scalar objective functions Relative to parameters Is a matrix of gradients of (1), Representing the exponentiation of the gradient matrix, In order for the coefficient of convergence to be a function of, To act on scalar objective functions Is used to activate the non-linear activation function of (a).
8. The visual sense and tactile sense fusion sensing method for detecting surface cracks of a metal part according to claim 7, wherein in the step S5, a calculation formula of the confidence coefficient is: Wherein, the The degree of confidence is indicated and, For the visual discrimination confidence based on the visual geometry feature vector obtained in step S2, To determine the confidence level based on the mechanics of the haptic feature vector obtained in step S3, The confidence is judged for fusion based on the fusion characteristics obtained in the step S4; 、、 Respectively is 、、 Is not a negative weight coefficient of (a).

Description

Visual sense and touch sense fused sensing metal part surface crack detection method Technical Field The invention relates to the technical field of crack detection, in particular to a visual sense and tactile sense fusion perceived metal part surface crack detection method. Background The crack detection of the metal parts in the high-speed rail is a core technical link for guaranteeing the safe operation of the rail transit system, preventing major accidents and prolonging the service life of equipment. The crack is used as a main expression form of metal fatigue damage, and early accurate identification of the crack can effectively avoid fracture failure of key components such as a bogie, a wheel set, a brake disc and the like in high-speed operation. The traditional detection means mainly depend on manual visual inspection and nondestructive inspection technologies, and have obvious limitations that 1) manual inspection needs to be stopped for maintenance, detection efficiency is outstanding with high-frequency operation demands of trains, and is influenced by personnel experience and fatigue degree to have detection omission risk, 2) conventional nondestructive inspection such as ultrasonic inspection needs to be in contact coupling, on-line monitoring of moving parts is difficult to realize, 3) industrial endoscope detection is limited by narrow visual field and limited in recognition capability of cracks in complex structures. Along with the development of intelligent detection technology, a crack detection method based on computer vision is gradually applied to the field of rail transit, however, the prior art mainly relies on a single sensor for detection, multi-mode fusion cannot be realized, and the accuracy and reliability of detection are to be improved. Disclosure of Invention In view of the above, the invention provides a visual sense and touch sense fusion sensing metal part surface crack detection method, so as to fully exert the complementary advantages of cross-mode data and improve the accuracy and reliability of detection. A visual sense and touch sense fusion sensing method for detecting cracks on the surface of a metal part comprises the following steps: step S1, acquiring an image of the surface of a metal part to be detected through an RGB-D camera, and carrying out self-adaptive image enhancement and denoising pretreatment on the acquired image to obtain a pretreated image; s2, extracting visual geometric feature vectors of the cracks from the preprocessed image, wherein the visual geometric feature vectors comprise the length, the width, the visual depth and the visual depth statistical distribution characteristics of the cracks; S3, positioning to a crack area through a touch sensor, collecting 3D morphology, displacement field and distribution force data of the crack, calculating the touch measurement depth of the crack based on a mechanical model, and constructing a touch feature vector based on the touch measurement depth; s4, performing multi-scale alignment on the visual geometric feature vector and the tactile feature vector by adopting wavelet transformation, and performing feature level fusion by using a return-to-zero neural network optimization algorithm to obtain fusion features; And S5, performing crack detection by using a machine learning classifier based on the fusion characteristics, and outputting the size, the position and the confidence of the crack. The visual sense and touch sense fusion sensing metal part surface crack detection method provided by the invention has the following beneficial effects: (1) According to the invention, the detection precision is effectively improved through the depth cooperation of the visual sense and the touch sensor. Global crack morphology information is provided by the RGB-D camera, while the tactile sensor provides locally accurate mechanical parameters, which are effectively complementary. Experimental data show that in the test of the high-speed railway wheel sample, the F1 fraction of the invention reaches 95.2%, which is obviously superior to that of a single visual detection method. (2) While conventional visual inspection methods are susceptible to illumination variation and surface texture interference, the present invention effectively overcomes these limitations by a dual verification mechanism. The visual detection stage adopts a depth information guided image enhancement technology and combines a multi-scale filtering strategy, and the touch verification stage provides real data verification of a physical layer through a mechanical model to form a redundancy check mechanism. The tactile data provides reliable backup verification when the visual sensor is disturbed by surface reflection or shadow, whereas the visual coordinate calibration corrects errors in real time when the tactile sensor has positioning bias. The bidirectional fault tolerance mechanism ensures the stable operation of the system in a complex indu