Search

CN-120747621-B - Machine dog-assisted underground structure bolt missing detection method

CN120747621BCN 120747621 BCN120747621 BCN 120747621BCN-120747621-B

Abstract

The invention discloses a machine dog-assisted underground structure bolt missing detection method, which comprises the steps of firstly, ensuring the safety of shooting photos by a patrol machine dog by constructing an environment sensing algorithm based on fusion of laser radar point clouds and camera laser point clouds, then, moving shooting the bolt missing condition at the joint of an underground structure by using patrol mobile machine dog equipment, preprocessing the image, and finally, constructing an underground structure internal connecting bolt missing identification data feature, and identifying the bolt missing condition by using an improved Yolov8+ HorNet +DASI+ CBAM model so as to eliminate the potential safety hazard caused by underground structure bolt missing problem, thereby ensuring the operation safety of the underground structure. The invention solves the problem of difficult access of manual work by using the camera-carried inspection robot, has higher recognition precision compared with the prior method, and is more suitable for recognizing the missing bolts in severe and complex environments.

Inventors

  • LIU YANG
  • JIANG ZAIYANG
  • GAO QINGFEI
  • ZHOU ZHENG
  • YANG JIANXING
  • Shi Chenpeng

Assignees

  • 哈尔滨工业大学

Dates

Publication Date
20260508
Application Date
20250630

Claims (7)

  1. 1. The method for detecting the missing of the underground structure bolt assisted by the machine dog is characterized by comprising the following steps of: Constructing an environment sensing algorithm based on fusion of laser radar point clouds and camera laser point clouds, and realizing safe operation of the inspection robot dog in a complex environment based on fusion of point cloud data and a grid map; secondly, moving shooting the condition of bolt missing at the joint of the pipeline and the underground structure by using mobile machine dog equipment, and carrying out detail enhancement on the shot image in the dark environment of the underground structure based on a median filtering algorithm and gray threshold conversion; the specific steps of processing the blur noise by using the dynamic image based on the particle swarm are as follows: (1) Dividing an image into a plurality of small blocks with the size of (252) by a sliding window method, firstly, acquiring the height and the width of the image, creating an empty list [ ] for storing the blocks and the position information thereof, then, calculating the step length of each block, considering whether an overlapped area exists or not, if not, storing the position information into the empty list [ ], traversing the whole image by using a double circulation, extracting the current image block by using a slicing operation in each circulation, storing the current image block and the left upper corner coordinates of the block into the empty list, and finally, returning a list containing all the image blocks and the positions thereof by using a function; (2) And (3) carrying out brightness measurement on the overall brightness difference of the two images by adopting a particle swarm global optimization algorithm, namely, judging whether the average brightness of the images is similar or not, wherein the calculation formula is as follows: in the formula, And Is an image And Is used for the average value of (a), Is a constant; The contrast calculation formula is: in the formula, And Is an image And Is set in the standard deviation of (2), Is a constant; The structural term calculation formula is as follows: in the formula, Is an image And Covariance between; is a constant; after the brightness, contrast and structure are synthesized, a structural similarity index formula is obtained: in the formula, 、 、 Is a constant value, and is used for the treatment of the skin, Is a brightness item; Is a contrast term; is a structural item; each particle updates the velocity in each iteration by the following formula: in the formula, Is that In the first place The speed of the second iteration; for inertial weight, controlling the retention degree of the particles in the current direction; And Is a learning factor; And Is a random number; the best position found for particle i; The optimal position found in the particle swarm; the location update formula in each iteration is as follows: in the formula, Is a particle In the first place The current position of the next iteration; Is the updated speed; Calculating a gradient for each image block and calculating a standard deviation of the image block: in the formula, Is a local weighting value; is the gradient of the image block; is the standard deviation of the gradient; And measuring the structural similarity between the restored image and the original image through the global structural similarity index value: Based on the global structural similarity index, calculating the overall structural similarity between the restored image and the original image, and based on the local and global weighting, obtaining a final weighted fusion formula as follows: in the formula, The restored initial image; a weighted value calculated based on the local feature; a weighted value calculated based on the global structural similarity; The final image after the restoration; When the overlapping area is processed, a weighted average method is used for fusing recovery results, so that obvious transition problems at joints are avoided, and the weighted formula is as follows: in the formula, And A restored image which is a left-right overlapping portion; And Is a weighting coefficient; Splicing the processed image blocks according to the positions of the original positions of the processed image blocks in the image, reconstructing a complete image, and obtaining the complete image after all the image blocks are spliced; the method comprises the steps of collecting a bolt picture data set containing normal connecting bolts and bolt missing pictures, and expanding the picture data set by using an image expansion method; And fourthly, replacing a Conv convolution module with a HorNet module in a backup in a Yolov network, adding a DASI attention module and a CBAM attention module in a head end, constructing an improved Yolov 8+ HorNet +DASI+ CBAM model based on a Yolov8 basic model, and identifying the state of the underground structure bolt based on the improved Yolov model.
  2. 2. The method for detecting the missing of the bolt of the underground structure assisted by the machine dog according to claim 1, wherein the specific steps of the first step are as follows: carrying a laser radar on a four-foot inspection robot dog to realize 360 degrees x 90 degrees hemispherical ultra-wide angle sensing capability and acquire three-dimensional information of surrounding environment in real time; Step one, constructing a machine dog motion distortion removal method based on a leg-foot type odometer; And thirdly, performing distributed feature layer fusion by adopting point cloud information of laser radar point cloud data and camera laser point cloud data.
  3. 3. The method for detecting the missing of the bolt of the underground structure assisted by the machine dog according to claim 2, wherein the specific steps of the step two are as follows: Step two, taking laser point cloud of the inspection robot dog after 360 degrees of working scanning of the underground structure as a frame to output and record 、 The starting time and the ending time of one frame of laser radar point cloud data are respectively, the time interval of two frames of data is the time interval of the two frames of data, and the first milemeter is assumed Sum of all Data, start time And end time Corresponding inspection machine dog change bit pose 、 The method comprises the following steps of: step two and two, assume Is the odometer data of two adjacent moments and Interpolation processing is performed according to the following formula to obtain Machine dog pose corresponding to laser radar point cloud data at moment : Packaging the interpolation result into new laser radar point cloud data under the coordinates of an odometer, wherein the new laser radar point cloud data is one frame of laser radar point cloud data Pose corresponding to each laser point Are obtained by interpolation of the step two by two linear data, 、 Respectively the first First, second The pose corresponding to the laser points, And The method is characterized by respectively converting the coordinates before and after conversion, wherein the coordinate conversion calculation method is as follows: And step two and four, repackaging and outputting the laser radar point cloud data subjected to the movement distortion removal to finish the work of removing the distortion of the laser radar point cloud, thereby correctly reflecting the surrounding environment information of the inspection machine dog.
  4. 4. The method for detecting the missing of the bolt of the underground structure assisted by the machine dog according to claim 2, wherein the specific steps of the step one three are as follows: step one, three, converting the point cloud data of the two laser radars into PCL format under the world coordinate system The point cloud data are combined into the point cloud data consisting of three-dimensional coordinate points by using the PCL point cloud library And uses it in three-dimensional space vector matrix Is expressed in terms of (a); traversing vector matrix, plane projecting all vectors, calculating horizontal distance from projection point to machine dog-base coordinate system And with Included angle of axes ; Step one, three and three according to Calculating fused laser data of a current projection point in predefining Index sequence number in array If this is The distance value of the sequence number is selected according to the principle of the minimum distance value and assigned to In (a) and (b); step one, three and four, namely after traversing all column vectors, finishing assignment The array is returned to obtain 。
  5. 5. The method for detecting the missing of the underground structure bolt assisted by the machine dog according to claim 1, wherein the specific steps of the third step are as follows: manually marking the picture data by Labelimg marking software based on the collected normal connecting bolt picture and the bolt missing picture, and outputting a tag file in txt format; changing the illumination intensity of the collected picture data and the generated txt tag file, and changing the illumination intensity of the picture data according to different illumination intensities based on the illumination intensity of the collected picture data; Based on the position of the pixel of the acquired picture data in the picture, changing the pixel position of the picture data from different pixel ranges; Thirdly, changing the pixel position of the picture data from the horizontal mirror image, the vertical mirror image and the horizontal vertical mirror image based on the position of the pixel of the collected picture data in the picture; Based on the position of the pixel of the acquired picture data in the picture, rotating and changing the pixel position of the picture data in different angle rotation ranges by taking the center pixel of the picture as a circle center; step three, taking a bolt part in the image as an interested region ROI, and carrying out boundary extraction on the interested region by using an irregular polygon; and thirdly, extracting corner points and marking point positions from the image by adopting pgonCorners corner point detection algorithm.
  6. 6. The method for detecting the missing of the underground structure bolt assisted by the machine dog according to claim 5, wherein the third step is as follows: Let the input binarized image be Wherein And Respectively representing the height and width of the shot image, assuming that the number of vertexes of the convex polygon of the bolt is Finally, outputting the coordinates of k vertexes in the original image ; (1) Boundary extraction, namely extracting the boundary of the convex polygon through connected domain analysis according to the input binarized image B, wherein the length of the boundary is assumed to be Obtaining the coordinate sequence of the boundary Normalizing the boundary points; (2) Polar sampling for each point Calculate its polar angle to the center of the image And limit , Representing the center point coordinates of the image, namely: By at least one of Uniform sampling in range Obtaining polar angle sequence of a group of sampling points by polar angle values Polar angle for each sampling point Calculate its projection position on the boundary The method comprises the following steps: Wherein: Representing distance sampling points The nearest boundary point is at the polar angle of the clockwise direction on the boundary; (3) Grid statistics, namely, all projection points are counted Dividing the grids into K multiplied by K, counting the number of projection points contained in each grid, and finally selecting the first K grids with the largest number of projection points as candidate areas of the corner points; (4) Taking each candidate area as a vertex, selecting K points as the vertices of the bolt convex polygon, enabling the K points to be as close as possible to projection points on the boundary contour, and arranging and outputting the solved K vertices in ascending order of polar angle; and determining a searching radius according to the corner coordinates by using a color matching algorithm, traversing each corner, searching marked pixel points matching with color values in a specified radius range from the periphery of each corner, counting the number of pixels, storing corresponding corner index numbers, finding marked positions and drawing.
  7. 7. The method for detecting the missing of the underground structure bolt assisted by the machine dog according to claim 1, wherein the specific steps of the fourth step are as follows: Adding HorNet universal visual convolution module on Yolov backbone network to improve the space interaction of target detection task; Step four, adding a DASI attention mechanism to the head of the Yolov model to enhance jump connection in the Yolov model; Thirdly, adding CBAM attention mechanism to emphasize important features along two main dimensions of a channel and a space axis by utilizing a front feedback neural network CBAM attention mechanism module; Fourthly, in order to display the identification effect more intuitively and effectively, mAP@0.5 values, precision values, recall values and F1-Score values are selected to evaluate the detection performance of the model on the identification of the actual condition of the bolt.

Description

Machine dog-assisted underground structure bolt missing detection method Technical Field The invention belongs to the field of operation monitoring of underground structures such as tunnels, pipe galleries and pipelines, relates to a detection method for bolt loss during operation of an underground structure, and particularly relates to an underground structure bolt loss identification method based on a robot dog embedded improved Yolov model. Background The construction period of underground infrastructure is generally longer, and old ageing phenomenon not only appears in the underground infrastructure that falls into in early stage, and in the operation in-process of underground integrated structure, probably the pipeline can appear and piping lane structure connecting bolt is lost. Meanwhile, a large number of pipelines and pipelines are arranged in the underground comprehensive structure, the space is narrow, manual work is difficult to enter, small targets such as bolts are not easy to identify under dim light conditions, and therefore great potential safety hazards exist in structural operation such as tunnels, pipelines and pipe galleries. Disclosure of Invention Aiming at accurately identifying the problem of underground structure bolt missing in a complex environment, the invention provides a machine dog-assisted underground structure bolt missing detection method. According to the method, the underground structure environment which is difficult to enter manually is solved by using the camera-carried inspection robot, meanwhile, an improved Yolov model-based Yolov8 + HorNet +DASI+ CBAM image recognition model is provided, and the bolt missing condition of the complex environment such as an underground structure is accurately recognized, so that potential safety hazards caused by the problem of the bolt missing of the underground structure are eliminated, and the operation safety of the underground structure is guaranteed. The invention aims at realizing the following technical scheme: a method for detecting the missing of a bolt of an underground structure assisted by a machine dog comprises the following steps: Constructing an environment sensing algorithm based on fusion of laser radar point clouds and camera laser point clouds, and realizing safe operation of the inspection robot dog in a complex environment based on fusion of point cloud data and a grid map; secondly, moving shooting the condition of bolt missing at the joint of the pipeline and the underground structure by using mobile machine dog equipment, and carrying out detail enhancement on the shot image in the dark environment of the underground structure based on a median filtering algorithm and gray threshold conversion; the method comprises the steps of collecting a bolt picture data set containing normal connecting bolts and bolt missing pictures, and expanding the picture data set by using an image expansion method; And fourthly, replacing a Conv convolution module with a HorNet module in a backup in a Yolov network, adding a DASI attention module and a CBAM attention module in a head end, constructing an improved Yolov 8+ HorNet +DASI+ CBAM model based on a Yolov8 basic model, and identifying the state of the underground structure bolt based on the improved Yolov model. Compared with the prior art, the invention has the following advantages: The invention aims at the problem of visual recognition of the underground structure diseases detected by the machine dog, combines the characteristics of dynamic image data acquisition and underground structure internal bolt deletion of the machine dog detection based on the computer visual recognition technology, and is a method for recognizing the underground structure internal connecting bolt deletion which is respectively combined by a complex environment safety inspection, a dynamic shooting image enhancement preprocessing algorithm and a deep convolution and an attention mechanism and is more suitable for recognizing the bolt deletion in a severe complex environment. Drawings Fig. 1 is a flowchart of a method for identifying missing bolts in an underground structure of a robot dog carrying improvement Yolov. Fig. 2 is a diagram of a modified Yolov8 + HorNet +dasi+ CBAM model architecture. Fig. 3 is a photograph of a patrol robot dog. Fig. 4 is an 8K camera and mobile robot picture. Fig. 5 is a diagram showing a collected picture including a large number of cables and brackets, uneven light, severe shielding, and dim light. Fig. 6 is a photograph of the front and rear of the median filter algorithm (left before the treatment, right after the treatment). Fig. 7 is a photograph before and after gradation threshold conversion (left before processing, right after processing). Fig. 8 is a picture processed by the particle swarm global optimization algorithm (left before processing, right after processing). Fig. 9 is a graph of Yolov8 + HorNet +dasi+ CBAM model identification effects. FIG. 10