CN-121994919-A - Ceramic tile hollowing identification method based on multi-mode data acquired by wall climbing robot

CN121994919ACN 121994919 ACN121994919 ACN 121994919ACN-121994919-A

Abstract

The invention discloses a tile hollowing identification method based on multi-mode data acquired by a wall climbing robot, which comprises the steps of synchronously acquiring the multi-mode data including ultrasonic mode data, acoustic mode data and visual image mode data by the wall climbing robot, preprocessing the multi-mode data, constructing a multi-mode data set and training a constructed multi-mode fusion model, wherein the multi-mode fusion model comprises an ultrasonic branch, an acoustic branch, a visual image branch and a transducer fusion module, and the transducer fusion module fuses the output of the ultrasonic branch, the acoustic branch and the visual image branch and then carries out hollowing/non-hollowing identification by a Softmax classifier. According to the method, ultrasonic, acoustic and visual multi-mode data are fused, and the cloud edge cooperative intelligent architecture is combined, so that high-precision and high-efficiency automatic empty detection is realized.

Inventors

CHU JIAQING
TANG SHENGHUA
XIA FENG
CHENG YUZHU
WANG YI
YU LE
ZHANG CHAOQUN

Assignees

杭州驭熵科技有限公司

Dates

Publication Date: 20260508
Application Date: 20260109

Claims (10)

1. The tile hollowing identification method based on the multi-modal data acquired by the wall climbing robot is characterized by comprising the steps of synchronously acquiring the multi-modal data including ultrasonic mode data, acoustic mode data and visual image mode data by utilizing the wall climbing robot, preprocessing the multi-modal data, constructing a multi-modal data set and training a constructed multi-modal fusion model, wherein the multi-modal fusion model comprises an ultrasonic branch, an acoustic branch, a visual image branch and a transducer fusion module, and the transducer fusion module fuses the output of the ultrasonic branch, the acoustic branch and the visual image branch and then carries out hollowing/non-hollowing identification by utilizing a Softmax classifier; The multi-mode data set ultrasonic mode sample structure comprises an ultrasonic echo signal-time spectrogram, the acoustic mode sample structure comprises a discrete time acoustic signal sequence-short time energy/frequency spectrum amplitude/spectral entropy/mel cepstrum coefficient characteristic, and the visual image mode sample structure comprises an RGB image-texture characteristic vector/high-level characteristic vector.
2. The tile hollowing identification method according to claim 1, wherein the wall climbing robot performs two-dimensional scanning along the wall surface at a set step pitch, sampling interval and frequency, triggers ultrasonic and acoustic detection, and image acquisition, and performs time alignment on acquired modal data.
3. The tile hollowing identification method according to claim 1, wherein in the multi-mode data preprocessing, preprocessing of ultrasonic mode data comprises sequentially adopting DC component removal, band-pass filtering, envelope detection and STFT conversion processing on the collected ultrasonic echo signals to generate a time-frequency diagram.
4. The tile blank recognition method according to claim 1, wherein in the multi-mode data preprocessing, the preprocessing of the acoustic ultrasonic mode data comprises the steps of obtaining a discrete time acoustic signal sequence through analog-to-digital conversion of an acquired acoustic signal, carrying out framing processing on the discrete time acoustic signal sequence, and calculating short-time energy, spectral amplitude, spectral entropy and mel-frequency cepstrum coefficient of sound waves by utilizing the framing signal.
5. The tile hollowing recognition method according to claim 4, wherein the short time energy of the sound wave Spectral amplitude Spectral entropy Mel-frequency cepstrum coefficient MFCC The method for recognizing the hollowness of the ceramic tile comprises the following steps of when And is also provided with When marking as high confidence empty candidate points And (2) and Higher order components of (2) being greater than a predetermined multiple of the sum of their historical mean remainders corresponding to standard deviations, or said spectral magnitudes The energy duty ratio in a preset medium-low frequency band exceeds a threshold value, the detection point is marked as a suspicious point, the suspicious point is triggered by the wall climbing robot to perform secondary sampling, the sound wave echo signals are recorded, the ultrasonic echo data are combined to perform empty drum judgment, 、 Short-time energy for all sampling points in the current detection area Arithmetic mean, standard deviation of (c).
6. The tile blank recognition method according to claim 1, wherein in the multi-mode data preprocessing, preprocessing of visual image mode data comprises gridding processing of an acquired image, performing adaptive histogram equalization enhancement based on the gridded image to enhance local contrast, performing local binary pattern texture description of the enhanced image, extracting texture feature vectors of each grid unit, extracting semantic features of the image through a convolutional neural network to obtain high-level feature vectors, judging whether texture continuity damage or semantic feature abnormality exists in a corresponding grid area or not based on the texture feature vectors and the high-level feature vectors, and marking the grid area as a visual suspicious blank area for subsequent multi-mode fusion judgment when the abnormality meets a preset condition.
7. The tile blank recognition method according to claim 1, wherein the ultrasonic branches adopt ResNet-18 networks, the hierarchical structure comprises 7×7 convolution layers (step size 2) +batch normalization+ReLU, 4 residual blocks, each residual block comprises two layers of 3×3 convolutions, and ultrasonic feature vectors are output after global average pooling ; The acoustic branch adopts a 1D-CNN+LSTM mixed structure, wherein the 1D-CNN extracts local sound spectrum characteristics, a pooling layer reduces time sequence redundancy, an LSTM unit captures the evolution trend of acoustic energy along with time, and a full-connection layer outputs acoustic characteristic vectors ; And the visual image branches adopt ResNet-18 backbone networks to extract surface texture and crack information. Outputting visual image feature vectors 。
8. The tile blank recognition method according to claim 7, wherein the transform fusion module outputs a fusion feature vector as: , Wherein, the 、、 Is a trainable weight parameter.
9. The tile hollowing identification method according to claim 1, wherein the multi-modal fusion model adopts a joint loss function as: , Wherein, the For classifying cross entropy loss, learning and judging empty drum/non-empty drum; the reconstruction loss of multi-mode consistency is restrained, and the reconstruction capability of self-coding or voiceprint characteristics of a time-frequency spectrogram is restrained; to account for physical consistency loss based on acoustic wave propagation equation constraints, 、 Is a super parameter.
10. The tile hollowing identification method according to claim 1 is characterized by further comprising a lightweight ultrasonic pre-screening model deployed on the edge side, wherein the lightweight ultrasonic pre-screening model comprises three layers of lightweight one-dimensional convolutional neural networks and is used for extracting time domain features from ultrasonic time domain echo signals, the extracted time domain features are subjected to dimension compression through global averaging pooling and then input to a Sigmoid output layer, and confidence scores corresponding to hollowing sampling points are generated, so that tile hollowing real-time pre-screening under the low-calculation-power edge calculation condition is achieved.

Description

Ceramic tile hollowing identification method based on multi-mode data acquired by wall climbing robot Technical Field The invention relates to the technical field of tile hollowing detection, in particular to a tile hollowing recognition method based on multi-mode data acquired by a wall climbing robot. Background The empty building exterior wall tile is a common building quality and potential safety hazard, and the existing detection method has the following defects of (1) strong subjectivity, low efficiency and safety cause anxiety, wherein the detection result is difficult to quantify, reproduce and trace due to unavoidable subjective randomness caused by judging by relying on experience and hearing of a detector. In addition, for high-rise buildings, even a scaffold is required to be built or a hanging basket is required to be used, the detection efficiency is extremely low, and the method cannot meet the severe requirements of modern building operation and maintenance on efficiency and safety. (2) The infrared thermal imaging method realizes non-contact large-area rapid scanning, but the physical mechanism determines the strong environment dependence. External factors such as sunlight, wind speed, rainfall, wall materials, humidity and the like can seriously interfere the heat conduction characteristics of the wall, and false alarm or missing alarm is extremely easy to generate. For example, shadows, wall stains, or material differences may exhibit thermal spots on thermal images similar to hollows, while hollows with deep, small areas, or insignificant temperature differences are difficult to capture effectively. Therefore, the method can be used as a preliminary screening means, and is difficult to independently provide reliable and accurate quantitative diagnosis conclusion. (3) The ultrasonic detection can identify internal defects by analyzing the propagation and reflection of stress waves in a medium, and has theoretical advantages for detecting empty drums in principle. However, conventional ultrasound detection relies on manual hand-held point-by-point measurements. In order to realize effective acoustic coupling, the probe is required to be perpendicular to the wall surface and apply constant pressure, and the operation mode is seriously dependent on manpower, is time-consuming and labor-consuming, and can not realize automatic data acquisition of a large-scale outer wall. More importantly, the single-point ultrasonic signal is extremely easy to be interfered by coupling of various factors such as sensor coupling state, wall roughness, internal steel bars and the like, so that the signal to noise ratio is low, interpretation is difficult, and robustness is insufficient. Disclosure of Invention The invention aims to provide the tile hollowing identification method based on the multi-mode data acquired by the wall climbing robot, which realizes complex multi-scene tile hollowing detection with high safety and detection precision and reproducibility. The tile hollowing identification method based on the multi-mode data acquired by the wall climbing robot comprises the steps of synchronously acquiring the multi-mode data including ultrasonic mode data, acoustic mode data and visual image mode data by the wall climbing robot, preprocessing the multi-mode data to construct a multi-mode data set and training a constructed multi-mode fusion model, wherein the multi-mode fusion model comprises an ultrasonic branch, an acoustic branch, a visual image branch and a transducer fusion module, and the transducer fusion module fuses the outputs of the ultrasonic branch, the acoustic branch and the visual image branch and then carries out hollowing/non-hollowing identification by using a Softmax classifier; The multi-mode data set ultrasonic mode sample structure comprises an ultrasonic echo signal-time spectrogram, the acoustic mode sample structure comprises a discrete time acoustic signal sequence-short time energy/frequency spectrum amplitude/spectral entropy/mel cepstrum coefficient characteristic, and the visual image mode sample structure comprises an RGB image-texture characteristic vector/high-level characteristic vector. Preferably, the wall climbing robot performs two-dimensional scanning along the wall surface at set steps, sampling intervals and frequencies, triggers ultrasonic and acoustic detection, acquires images, and performs time alignment on acquired modal data. Preferably, in the multi-mode data preprocessing, the preprocessing of the ultrasonic mode data comprises the steps of sequentially adopting DC component removal, band-pass filtering, envelope detection and STFT conversion processing on the acquired ultrasonic echo signals to generate a time-frequency diagram. Preferably, in the multi-mode data preprocessing, the preprocessing of the acoustic ultrasonic mode data comprises the steps of obtaining a discrete time acoustic signal sequence through analog-to-digital conversion of an acqu