CN-122020130-A - Multi-mode deep learning identification method, system and medium for rock mass structural surface of side slope under complex terrain

CN122020130ACN 122020130 ACN122020130 ACN 122020130ACN-122020130-A

Abstract

The invention discloses a multi-mode deep learning identification method, a system and a medium for a slope rock mass structural surface under complex topography, which relate to the technical field of geotechnical engineering monitoring and geological investigation, and are characterized in that fusion characteristics comprising geometric characteristics and spectral characteristics are constructed based on original point cloud data and hyperspectral image data, data missing in the original point cloud data is complemented through the fusion characteristics, semantic division is carried out to obtain rock face point cloud data, structural surface cluster identification is carried out based on the rock face point cloud data, non-rock face point cloud data is eliminated, the geometric characteristics are sensitive to space structures, the spectral characteristics can identify interferents such as vegetation, and the like, when the vegetation is covered by the vegetation, the rock mass and the vegetation are distinguished through the spectral characteristics, false closure is inhibited, and only the pure rock face point cloud is clustered, so that interference of the non-rock face points is avoided, and accuracy and stability of structural surface production identification are remarkably improved.

Inventors

FU HAIYING
TANG JIE
XU TONG
ZHAO YANYAN
ZHENG WEI
LI YUFAN

Assignees

四川省建筑机械化工程有限公司
西南交通大学

Dates

Publication Date: 20260512
Application Date: 20260410

Claims (10)

1. The multi-mode deep learning identification method for the rock mass structural surface of the side slope under the complex terrain is characterized by comprising the following steps of: Synchronously collecting multi-modal data of a target slope area, and preprocessing the multi-modal data, wherein the multi-modal data comprises original point cloud data and hyperspectral image data; Inputting the preprocessed multi-mode data into a multi-mode deep neural network to construct a rock face point cloud, and carrying out structural face clustering recognition on the rock face point cloud, wherein the multi-mode deep neural network comprises a feature extraction network and a multi-task decoding network, the multi-task decoding network comprises a first decoding branch, a second decoding branch and a third decoding branch, the feature extraction network extracts geometrical features and spectral features of the point cloud from the preprocessed multi-mode data and fuses the geometrical features and the spectral features to obtain fusion features, the first decoding branch is used for supplementing the original point cloud data based on the fusion features to obtain dense point cloud data, the second decoding branch is used for carrying out semantic segmentation on the dense point cloud data based on the fusion features to determine the rock face point cloud data and non-rock face point cloud data, and the third decoding branch is used for carrying out structural face clustering recognition on the rock face point cloud data based on the fusion features; And outputting a structural surface cluster recognition result.
2. The multi-modal deep learning identification method for the structural surface of the rock mass of the slope under the complex terrain according to claim 1, wherein the preprocessing method comprises the following steps: Firstly, performing outlier filtering and point cloud normalization processing on the original point cloud data, and performing radiation calibration, atmosphere correction and spectrum normalization processing on the hyperspectral image data; and back projecting each pixel of the hyperspectral image data into a three-dimensional space, performing point-by-point matching with the original point cloud data, so that each point cloud has corresponding spectrum information, and performing spectrum assignment on the point clouds without the corresponding spectrum information based on nearest neighbor interpolation or common view geometric relationship.
3. The multi-mode deep learning identification method for the structural surface of the rock mass of the side slope under the complex terrain according to claim 1 or 2 is characterized in that the characteristic extraction network comprises a geometric characteristic extraction branch, a spectral characteristic extraction branch and a characteristic fusion network; the geometrical feature extraction branch takes original point cloud data as input, and takes an improved PointNet ++ network as a backbone network to extract geometrical features of each point cloud; the spectrum characteristic extraction branch consists of a plurality of cascaded cavity convolution layers, the cavity rate of each cavity convolution layer is sequentially increased, and residual connection is introduced into each cavity convolution layer; The feature fusion network comprises a bimodal channel separation component, a dual-branch spatial attention component and a feature fusion component; The bimodal channel separation assembly performs channel dimension separation processing on the geometric characteristics and the spectral characteristics to obtain important channel characteristics and secondary important channel characteristics; The double-branch spatial attention component performs spatial attention enhancement on important channel characteristics and performs global calibration on channel dimensions on secondary important channel characteristics; And the feature fusion component performs weighted fusion on the features with enhanced spatial attention and the features with global calibration to obtain fusion features.
4. The multi-mode deep learning identification method for the structural surface of the rock mass of the side slope under the complex terrain according to claim 3 is characterized in that the improved PointNet ++ network comprises a set abstraction layer and a feature propagation layer, wherein the set abstraction layer is composed of multi-level and multi-scale set abstraction modules; the set abstraction module performs the following process: T1, selecting N center sampling points from original point cloud data, wherein the center sampling points uniformly cover the surface of a rock mass; Setting n search radii, namely searching neighborhood points in the search radius r b by taking each center sampling point as a sphere center to construct a field point set with different search radii, wherein b=1, 2, n; And T3, introducing a normal vector guided attention weight mechanism on the basis of the deep convolution operation to perform feature extraction on the point sets of each field to obtain local aggregation features of each center sampling point under different search radiuses, wherein the local aggregation features of the center sampling point i under any search radius are as follows: ; ; wherein N (i) represents a neighborhood point set of the central sampling point i; representing the relative coordinates between the center sampling point i and the neighborhood point j; Representing the normal vector of the neighborhood point j; Representing the original feature vector of the neighborhood point j; Representing the contribution weight of the neighborhood point j to the characteristic update of the center sampling point i; the normal vector consistency coefficient of the central sampling point i and the neighborhood point j is represented, when the difference between the normal vector of the central sampling point i and the neighborhood point j is larger, The closer to 1, otherwise, The closer to 0, the mlp () represents the multi-layer perceptron; representing a basis weight calculated based on the spatial distance and the relative position between the center sampling point i and the neighborhood point j; And after the feature propagation layer splices and fuses the local aggregation features under different search radii, upsampling is performed based on a distance weighted interpolation strategy to obtain the geometric features of each point cloud.
5. The multi-mode deep learning identification method for the structural surface of the rock mass of the slope under the complex terrain according to claim 1, wherein the acquisition method for the dense point cloud data comprises the following steps: Downsampling the original point cloud data P in and the fusion features by a sparse convolution layer to extract multi-scale semantic features, wherein the multi-scale semantic features are normalized and subjected to ReLU activation processing to obtain sparse point cloud data P en and a feature map F en ; Extracting global enhancement features F enc based on the feature map F en , extracting local neighborhood features F local based on the sparse point cloud data P en , and fusing the global enhancement features F enc and the local neighborhood features F local to obtain multi-scale expansion features F exp ; Up-sampling the sparse point cloud data P en and the multi-scale expansion feature F exp by a sparse transposition convolutional layer symmetrical to the sparse convolutional layer, simultaneously, jumping and connecting the features of the sparse convolutional layer to the features of the corresponding sparse transposition convolutional layer to predict the coordinate offset delta P of each up-sampling point, and superposing the coordinate offset delta P and the coordinates of each up-sampling point cloud to obtain generated point cloud data P new ; And merging the original point cloud data P in with the generated point cloud data P new to obtain dense point cloud data.
6. The multi-modal deep learning identification method for the structural surface of the rock mass of the slope under the complex terrain according to claim 1, wherein the determination method for the rock surface point cloud and the non-rock surface point cloud comprises the following steps: aligning the dense point cloud data with the fusion features, splicing the coordinates of each point cloud with the fusion features along the channel dimension, linearly mapping the spliced features to low dimensions based on a linear layer to obtain the original logic value of each point cloud, and applying a Softmax function to the original logic value to calculate the first probability p rock that each point cloud belongs to a rock surface and the second probability that each point cloud does not belong to the rock surface; And presetting a probability threshold p e , in dense point cloud data, if the first probability p rock of the current point cloud is larger than the probability threshold p e , judging that the current point cloud is a rock face point cloud, otherwise, judging that the current point cloud is a non-rock face point cloud.
7. The multi-modal deep learning identification method for the structural surface of the rock mass of the side slope under the complex terrain according to claim 6, wherein the method for the structural surface cluster identification comprises the following steps: Multiplying the fusion characteristic of each rock face point cloud with a first probability p rock to obtain a rock face point cloud weighting characteristic F rock ; Constructing a graph attention convolution network, and extracting graph convolution characteristics e of each rock face point cloud by combining the rock face point cloud weighting characteristics F rock ; and carrying out cluster recognition on the graph rolling feature e based on a differential mean shift clustering method to obtain a slope rock mass structural surface.
8. The multi-modal deep learning identification method for the structural surface of the rock mass of the side slope under the complex terrain according to claim 7, wherein the differential mean shift clustering method comprises the following steps: G31, taking the graph roll-up characteristic e of all the rock surface point clouds as a clustering candidate center C= { e 1 ,e 2 ,…,e j ,…,e N }, wherein N represents the total number of the rock surface point clouds, e j represents the graph roll-up characteristic of the jth rock surface point cloud, and e N represents the graph roll-up characteristic of the Nth rock surface point cloud; G32, calculating a drift vector m (C) for the current cluster candidate center C E C: ; wherein, K () represents a kernel function and is used for calculating the contribution weight of each point to the current clustering candidate center; g33, updating the current cluster candidate center c=c+m (c), and repeatedly executing steps G32 and G33 until convergence; g34, merging the cluster candidate centers with the distance smaller than the distance threshold tau to obtain a cluster center set; And G35, distributing each rock face point cloud to the nearest clustering center to obtain a slope rock mass structural face recognition result.
9. The multi-mode deep learning and identifying system for the structural surface of the rock mass of the side slope under the complex terrain is characterized by being used for realizing the multi-mode deep learning and identifying method for the structural surface of the rock mass of the side slope under the complex terrain, and comprises the following steps: The system comprises an acquisition module, a target slope area acquisition module and a display module, wherein the acquisition module is used for synchronously acquiring multi-mode data of the target slope area, wherein the multi-mode data comprises original point cloud data and hyperspectral image data; The preprocessing module is used for preprocessing the multi-mode data; The system comprises an identification module, a multi-modal depth neural network, a multi-modal decoding network and a third decoding branch, wherein the pre-processed multi-modal data is input into the multi-modal depth neural network to construct a rock face point cloud, the rock face point cloud is subjected to structural face clustering identification, the multi-modal depth neural network comprises a feature extraction network and the multi-modal decoding network, the multi-modal decoding network comprises a first decoding branch, a second decoding branch and the third decoding branch, the feature extraction network extracts geometric features and spectral features of the point cloud from the pre-processed multi-modal data, the geometric features and the spectral features are fused to obtain fusion features, the first decoding branch is used for supplementing the original point cloud data based on the fusion features to obtain dense point cloud data, the second decoding branch is used for carrying out semantic segmentation on the dense point cloud data based on the fusion features to determine the rock face point cloud data and non-rock face point cloud data, and the third decoding branch is used for carrying out structural face clustering identification on the rock face point cloud data based on the fusion features; and the output module is used for outputting the structure face cluster recognition result.
10. A computer readable medium having stored thereon a computer program, wherein execution of the computer program by a processor implements a multi-modal deep learning identification method of a rock mass structural plane of a side slope under a complex terrain as claimed in any one of claims 1 to 8.

Description

Multi-mode deep learning identification method, system and medium for rock mass structural surface of side slope under complex terrain Technical Field The invention relates to the technical field of geotechnical engineering monitoring and geological investigation, in particular to a multi-mode deep learning identification method, a multi-mode deep learning identification system and a multi-mode deep learning identification medium for a slope rock mass structural plane under complex terrain. Background In the geotechnical engineering construction of high and steep slope stability evaluation, geological disaster prevention and control, water electricity, traffic and the like, the accurate acquisition of geometric information such as the occurrence, spacing, extension length and the like of a rock mass structural surface (such as cracks, layers and joints) is a key precondition for carrying out rock mass quality classification and stability calculation. In recent years, with the development of point cloud acquisition technologies such as three-dimensional laser scanning and unmanned aerial vehicle oblique photography, an effective means is provided for acquiring surface geometric information of a slope rock mass in a non-contact, high-precision and high-density manner. However, in practical engineering applications, especially in areas with complex geological environments, the existing rock mass structural plane identification technology based on point cloud data faces a serious technical bottleneck, and is mainly expressed in the following two aspects: Firstly, the problem of point cloud data loss (cavity) caused by illumination condition change is that in the field data acquisition of a steep slope, especially for deep canyon zones with north-south trend or specific trend, the apparent movement of the sun causes the rock mass structural plane to undergo severe illumination change in one day. When sunlight is obliquely projected, the convex part of the high and steep terrain can cast large-area moving shadows on the adjacent rock mass structural faces. Due to insufficient light, characteristic points in the photogrammetry fail to match or laser radar echo signals are weak in the shadow areas, and finally a large-area hollow area without data is formed in the acquired point cloud data. Because the holes are exactly positioned on the critical rock mass structural plane, the subsequent structural plane identification algorithm based on the point cloud density cannot fit a correct plane in the area, and the critical geological information is permanently lost. Secondly, the vegetation coverage and noise interference cause the false closure problem that the slope is often covered by vegetation (such as herbs and shrubs) after the vegetation concrete greening engineering is implemented or in the areas where natural vegetation is clustered. In the point cloud acquisition process, laser points or image points often hit vegetation blades positioned in front of the rock surface preferentially, and a large number of noise points representing vegetation are generated. When vegetation is thick, the noise points can completely shade the real rock surface points at the back visually and spatially, and a continuous false surface formed by vegetation points is formed in the point cloud model. Although the existing denoising algorithm can remove partial outlier noise, it is difficult to effectively separate vegetation points growing close to a rock surface from real rock surface points. When the structural surface is identified, the algorithm often erroneously identifies the pseudo-closed curved surface formed by vegetation as a rock structural surface, so that incorrect occurrence information (such as misjudging a steep inclination structural surface as a gentle inclination) is calculated, or the covered real structural surface is completely ignored, so that misjudgment on the integrity and stability of the rock is caused. In summary, in the prior art, under the coupling effect of complex illumination change (resulting in structural surface cavity) and vegetation coverage (resulting in structural surface pseudo closure) facing high and steep slopes, it is difficult to stably and accurately extract real rock mass structural surface information, and effective application of technologies such as three-dimensional laser scanning in geotechnical engineering investigation in extreme environments is restricted. Therefore, a new method capable of overcoming the defects and accurately identifying the structural surface of the rock mass under the conditions of data loss and strong noise interference needs to be developed. Disclosure of Invention The invention aims to provide a multi-mode deep learning identification method, a system and a medium for a side slope rock structure surface under complex terrains, which are difficult to stably and accurately extract real rock structure surface information under the coupling action of complex