CN-121998930-A - Pipeline inner wall defect detection method, system and product based on point cloud and image depth fusion

CN121998930ACN 121998930 ACN121998930 ACN 121998930ACN-121998930-A

Abstract

The application discloses a pipeline inner wall defect detection method, system and product based on point cloud and image depth fusion, which relate to the field of drainage pipeline defect detection, and the method comprises the following steps that S100, a two-dimensional image sequence and a point cloud sequence of the pipeline inner wall are obtained; the method comprises the steps of S200, extracting two-dimensional image feature vectors of a two-dimensional image sequence, carrying out feature dimension increase to obtain a two-dimensional image voxel feature body, S300, mapping a point cloud sequence to a three-dimensional voxel grid space, carrying out feature extraction to obtain a point cloud voxel feature body, S400, carrying out feature fusion of the two-dimensional image voxel feature body and the point cloud voxel feature body to obtain a fused voxel feature body, S500, obtaining a BEV feature image and a perspective feature image based on the fused voxel feature body, identifying defects to obtain a three-dimensional voxel mask and corresponding defect point cloud sets, and S600, carrying out defect calculation based on the defect point cloud sets. According to the method, the three-dimensional precise reconstruction of the inner wall of the pipeline and the quantitative assessment of defects are realized by fusing the three-dimensional point cloud data and the two-dimensional image data.

Inventors

JIANG HONG
MOU JINMING
TANG MIAO
ZHANG QIANLEI

Assignees

上海锦兴市政设计咨询有限公司

Dates

Publication Date: 20260508
Application Date: 20260120

Claims (10)

1. A pipeline inner wall defect detection method based on point cloud and image depth fusion is characterized by comprising the following steps: s100, respectively acquiring a two-dimensional image sequence and a point cloud sequence of the inner wall of the pipeline; s200, extracting features of a two-dimensional image sequence to obtain a two-dimensional image feature vector, carrying out feature dimension lifting on the two-dimensional image feature vector, and back-projecting the two-dimensional image feature vector to a pre-constructed three-dimensional voxel grid space to obtain a two-dimensional image voxel feature body V img ; S300, mapping the point cloud sequence to the three-dimensional voxel grid space, carrying out feature extraction to obtain a point cloud feature vector, and further obtaining a point cloud voxel feature body V pts through the relation between the point cloud and the three-dimensional voxel; S400, carrying out feature fusion of a two-dimensional image voxel feature body V img and a point cloud voxel feature body V pts corresponding to three-dimensional voxels in the three-dimensional voxel grid space to obtain a fused voxel feature body V fused ; S500, based on the fused voxel feature body V fused , respectively obtaining a BEV feature image and a perspective feature image of the inner wall of the pipeline, obtaining a three-dimensional voxel mask for identifying three-dimensional defects of the inner wall of the pipeline based on the BEV feature image and the perspective feature image, and obtaining defect point clouds corresponding to the three-dimensional voxel mask; and S600, calculating the three-dimensional defects of the inner wall of the pipeline based on the defect point cloud set.
2. The method for detecting the defects of the inner wall of the pipeline based on the fusion of the point cloud and the image depth according to claim 1, wherein in the step S200, the specific method for obtaining the voxel characteristic body of the two-dimensional image is as follows: S201, for any pixel (u, v) on the two-dimensional image, calculating the back projection video cone rays of the pixel in the three-dimensional voxel grid space; Let the homogeneous coordinates of the pixels be The normalized coordinates thereof are that, Back projection cone-of-view rays are obtained by cone-of-view ray equations The method comprises the following steps: ; Wherein, the To acquire the position of the camera's optical center in the global coordinate system for a two-dimensional image, K is a camera internal parameter calibration matrix, T is an external parameter calibration matrix, and T E [0, ] is a ray parameter; S202, in the three-dimensional voxel grid space V, calculating back projection video cone rays by using a ray traversal algorithm All voxels passing through set ; S203, the feature vector corresponding to the pixel (u, v) Scattered into voxels through which the ray passes according to a weighting rule, wherein Representing a real number set, wherein C is the number of characteristic channels of the two-dimensional image; s204, for the same voxel V i projected by a plurality of rays, adopting weighted average pooling to obtain a two-dimensional image voxel characteristic body V img , wherein the method comprises the following steps: ; wherein S (v i ) is the set of all pixels projected to voxel v i , w k is the weight corresponding to the kth pixel, f k is the feature vector corresponding to the kth pixel, and the image voxel feature volume Where (L, W, H) is the voxel grid size and C is the number of two-dimensional image feature channels.
3. The method for detecting the defects of the inner wall of the pipeline based on the fusion of the point cloud and the image depth according to claim 1, wherein in the step S300, the specific method for obtaining the point cloud voxel characteristic body is as follows: s301, mapping point clouds in a global coordinate system to a voxel grid V; s302, for each voxel v i , calculating a plurality of statistical features of an internal point cloud subset of the voxel v i ; S303, combining a plurality of statistical features into a point cloud feature vector, performing MLP (multi-layer perceptron) coding on the point cloud feature vector by adopting a multi-layer perceptron to obtain a point cloud voxel feature body of the pixel body v i Where (L, W, H) is the voxel grid size, D pts is the number of point cloud feature channels, and V pts (v i ) =0 for voxel V i that does not contain a point cloud.
4. The method for detecting defects on the inner wall of a pipeline based on fusion of point cloud and image depth as claimed in claim 3, wherein in S301, the point cloud in the global coordinate system is Each point in the point cloud has an index i belonging to a voxel v i , and the point cloud subset in the voxel v i is P i ; In S302, the plurality of statistical features of each voxel v i including centroid, covariance matrix, feature value and point density, for each voxel v i , calculate its internal point cloud subset The specific method of the plurality of statistical characteristics of (a) is as follows: for centroid, there are: ; for ‌ covariance matrices, there are: ; For the eigenvalues, there are: ; For the dot density, there are: ; in the formula, Representing the real set, T is the extrinsic calibration matrix, and V cell is the volume of a single voxel.
5. The method for detecting the defects of the inner wall of the pipeline based on the fusion of the point cloud and the image depth according to claim 1, wherein in the step S400, the specific method for carrying out the feature fusion of the two-dimensional image voxel feature body and the point cloud voxel feature body is as follows: S401, for a voxel v i , splicing a two-dimensional image voxel feature body and a point cloud voxel feature body in a feature channel dimension to obtain a spliced feature body; S402, performing multi-layer convolution calculation on the spliced feature body by using a sparse three-dimensional convolution neural network; s403, generating multi-scale features by pooling operation at different depth layers of the network, and capturing defects of different scales; S404, outputting the fused voxel feature body Where (L, W, H) is the voxel grid size and D fused is the fused voxel feature channel number.
6. The method for detecting the defects of the inner wall of the pipeline based on the fusion of the point cloud and the image depth as set forth in claim 1, wherein the step S500 specifically includes: s501, carrying out maximum pooling on the fused voxel feature body to obtain a BEV feature map; S502, re-projecting the fused voxel characteristic to a camera image plane according to a projection matrix from a voxel to a camera pixel by utilizing the micro-projectable layer to obtain a projection function, and obtaining a perspective characteristic diagram by adopting average pooling or aggregation pooling; s503, constructing an alignment loss function, and performing feature alignment constraint on the BEV feature map and the perspective feature map; S504, respectively configuring two-dimensional convolution segmentation heads on the BEV feature map and the perspective feature map, respectively calculating segmentation probability maps, and then carrying out weighted summation to obtain a final segmentation probability map; s505, performing binarization thresholding on the final segmentation probability map to obtain a three-dimensional voxel mask with the final segmentation probability map calculation result exceeding a threshold value, and converting the three-dimensional voxel mask into a defect point cloud set through inverse mapping.
7. The method for detecting defects on the inner wall of a pipeline based on point cloud and image depth fusion according to claim 6, wherein in the step S600, the calculation of the three-dimensional defects on the inner wall of the pipeline comprises the steps of clustering defect point clouds, extracting three-dimensional frameworks of the clustered defect point clouds, performing spline fitting on the frameworks, and calculating the defect length, the defect width and the defect depth respectively.
8. The pipeline inner wall defect detection system based on the point cloud and image depth fusion is characterized by comprising a data acquisition module, a two-dimensional image feature dimension increasing module, a point cloud voxelized feature encoding module, a voxelized feature fusion module, a defect detection module and a defect calculation module, and is used for realizing the defect detection method according to any one of claims 1 to 7.
9. The system for detecting defects of an inner wall of a pipeline based on fusion of point cloud and image depth according to claim 8, wherein the data acquisition module comprises a coordinate system definition sub-module, and the coordinate system definition sub-module defines a coordinate system of global coordinates according to the shape of the inner wall of the pipeline.
10. A computer program product, characterized in that it comprises a computer program or instructions that enable the computer program or instructions to implement the steps in the method for detecting defects of inner walls of pipes based on point cloud and image depth fusion according to any one of claims 1 to 7.

Description

Pipeline inner wall defect detection method, system and product based on point cloud and image depth fusion Technical Field The application relates to the field of defect detection of drainage pipelines, in particular to a pipeline inner wall defect detection method, system and product based on point cloud and image depth fusion. Background The urban water supply and drainage pipeline is an important component of urban municipal infrastructure and bears the transportation task of sewage, rainwater or domestic water. With the aging of the pipe network, defects such as cracks, corrosion, disjointing, dislocation and the like frequently occur in the pipe system, and the defects seriously affect the structural integrity and functions of the pipe, such as improper treatment, leakage of the pipe can be caused, and serious consequences such as environmental pollution, ground subsidence and the like can be caused under the condition that some leakage is serious. The existing drainage pipeline detection method comprises the following steps: 1) Closed Circuit Television (CCTV) detection, which is to acquire two-dimensional video through a camera moving inside a pipeline, and to conduct subjective interpretation by means of experienced inspectors. However, the method has the problems that only two-dimensional image information is provided, depth and geometric dimension information is lacking, interpretation results are qualitative rather than quantitative, the influence of environmental factors such as illumination, water quality and the like is large, the working efficiency is relatively low, and the labor cost and subjective interpretation error are large. 2) And (3) laser scanning (LiDAR) detection, namely acquiring a three-dimensional point cloud of the section of the pipeline through a laser radar, and evaluating the geometric distortion of the section of the pipeline. However, the point cloud lacks texture and color information, and is difficult to accurately distinguish surface dirt, water stain and real structural defects, the resolution of fine defects (such as fine cracks) is limited, and quantitative indexes such as defect depth cannot be output. 3) And the two-dimensional projection fusion scheme is that part of researches project laser point clouds into pseudo-color depth maps, and feature fusion is carried out on the pseudo-color depth maps and gray or color images on a two-dimensional pixel plane so as to realize defect identification. However, since the projection process essentially compresses three-dimensional information to two dimensions, the rich information of normal distance (depth dimension) is severely compressed, fusion is still performed on a two-dimensional plane, geometric constraint of three-dimensional space cannot be fully utilized, and the final quantitative detection result (such as crack width) of the defect is still based on two-dimensional pixel estimation, so that accuracy is limited, and complex three-dimensional defect forms cannot be accurately processed. In summary, the main problem of the prior art is that the three-dimensional information of the detection method has low utilization rate, serious information loss, and abundant information along the normal direction (depth direction) is greatly lost. This results in difficulty in distinguishing surface stains from real defects in two-dimensional images under complex pipeline working conditions (such as in-pipe deposition, water accumulation and bubbles), insufficient recognition capability for complex pipeline deformation (such as ovalization and collapse) and deep defects (such as corrosion and stress corrosion on the inner side of a pipeline wall), insufficient accuracy of defect size estimation based on a two-dimensional pixel map, usually +/-15-30 mm in accuracy, lack of capability of accurate measurement and calculation of defect size, lack of capability of quantitative calculation for three-dimensional physical quantities such as crack depth and dent volume, and the like, and the indexes are critical for checking structural bearing capacity and sorting of repairing priorities based on limit states. Disclosure of Invention The application aims to overcome the defects of the prior art, and provides a pipeline inner wall defect detection method, system and product based on point cloud and image depth fusion, which are suitable for the fine inspection, structural condition assessment and maintenance decision of municipal infrastructure pipeline facilities such as drainage pipes, rain and sewage pipes, pressure pipes, box culverts and the like by fusing laser radar three-dimensional point cloud and high-resolution digital image information to realize the three-dimensional precise reconstruction of the pipeline inner wall and quantitative assessment of defects. In a first aspect, the application provides a method for detecting defects of an inner wall of a pipeline based on depth fusion of point cloud and image, which adopts t