CN-122024160-A - River bank erosion three-dimensional deformation monitoring method based on multiple sensors

CN122024160ACN 122024160 ACN122024160 ACN 122024160ACN-122024160-A

Abstract

The invention belongs to the technical field of data processing and analysis, and particularly relates to a river bank erosion three-dimensional deformation monitoring method based on multiple sensors. The method comprises the steps of obtaining river bank images in a plurality of observation phases by utilizing unmanned aerial vehicle oblique photography, obtaining multi-phase unmanned aerial vehicle oblique photography point clouds through SfM three-dimensional reconstruction, forming purified river bank three-dimensional point clouds through statistical filtering and geometric filtering based on random sampling consistency RANSAC plane fitting, carrying out spatial clustering on the purified river bank three-dimensional point clouds in each observation phase to generate a plurality of super voxel nodes, constructing a river bank super voxel time space diagram structure based on spatial adjacent relations and time adjacent relations of the super voxel nodes, inputting multi-source time space comprehensive feature vectors into a time space diagram attention neural network, and mapping classification results back to the multi-source fusion river bank three-dimensional point clouds to output river bank erosion three-dimensional deformation distribution. The method can obtain the river bank erosion recognition result which is more stable and finer and has space continuity.

Inventors

ZHONG WEIJIE
SUI YANGANG
SONG WANGFANG
DOU FENGKE
XU PENG
LI HOUJI
SONG YIXIN
LIU BOHAN
ZHANG XIAOJIAO

Assignees

山东省煤田地质规划勘察研究院

Dates

Publication Date: 20260512
Application Date: 20260202

Claims (9)

1. The river bank erosion three-dimensional deformation monitoring method based on the multisensor is characterized by comprising the steps of obtaining river bank images in a plurality of observation time phases by utilizing unmanned aerial vehicle oblique photography, obtaining multi-time-phase unmanned aerial vehicle oblique photography point clouds by SfM three-dimensional reconstruction, obtaining corresponding ground laser radar bank slope point clouds by utilizing a ground laser radar, and unifying the multi-time-phase unmanned aerial vehicle oblique photography point clouds and the ground laser radar bank slope point clouds into the same coordinate system by a global satellite navigation system GNSS control point; the method comprises the steps of fusing multi-time-phase unmanned aerial vehicle oblique photographing point clouds and ground laser radar bank slope point clouds in each observation time phase by adopting initial registration based on feature matching and iterative closest point ICP precision registration to obtain multi-source fusion bank three-dimensional point clouds, forming purified bank three-dimensional point clouds through statistical filtering and geometric filtering based on random sampling consistency RANSAC plane fitting, carrying out spatial clustering on the purified bank three-dimensional point clouds in each observation time phase to generate a plurality of super-voxel nodes, constructing a river bank super-voxel time-space diagram structure based on spatial adjacency and time adjacency of the super-voxel nodes, extracting elevation change features, volume change features, normal change features, vegetation index change features, water level distance features and curvature change features for each super-voxel node, inputting multi-source time-space comprehensive feature vectors into a time-space diagram attention neural network, carrying out erosion area category super-voxel node, stacking area category super-voxel node category and stable area category super-voxel node category, and mapping the result back to the multi-source fusion bank three-dimensional point cloud to output river bank erosion three-dimensional distribution.
2. The method of claim 1, wherein the step of obtaining a multi-temporal unmanned aerial vehicle oblique photography point cloud comprises the steps of arranging a plurality of GNSS control points on a river reach to be monitored, carrying out multi-epoch static observation on each GNSS control point to obtain three-dimensional coordinates of each GNSS control point in a unified geodetic coordinate system, controlling an unmanned aerial vehicle oblique photography system to carry out multi-view aerial photography according to a preset course, altitude and front-back and side overlapping degree in each observation time phase, obtaining a multi-view image sequence covering a river bank slope and a near-shore area, inputting the image sequence into an SfM three-dimensional reconstruction process, sequentially carrying out image feature point extraction, inter-image feature point matching, joint beam adjustment and dense matching, generating a river bank three-dimensional point cloud under a camera coordinate system, and converting the river bank three-dimensional point cloud into the multi-temporal unmanned aerial vehicle oblique photography point cloud in the unified coordinate system according to the corresponding relation of the GNSS control points in the image.
3. The method of claim 2, wherein a plurality of ground laser radar scanning stations are arranged along the river bank, the river bank slope and the near-shore area are subjected to omnibearing three-dimensional scanning through the ground laser radar to obtain a ground laser radar bank slope point cloud in an instrument coordinate system, a reflection target corresponding to a GNSS control point is arranged in a ground laser radar scanning scene, a corresponding relation is established between a point corresponding to the reflection target in the ground laser radar bank slope point cloud and a GNSS control point three-dimensional coordinate, the ground laser radar bank slope point cloud is converted from the instrument coordinate system to a unified coordinate system based on three-dimensional rigid body transformation, and a multi-time-phase three-dimensional river bank point cloud data set is formed together with a multi-time-phase unmanned aerial vehicle oblique photography point cloud.
4. The method of claim 3, wherein the generation of the multi-source fusion river bank three-dimensional point cloud comprises the steps of performing voxel grid downsampling on multi-temporal unmanned aerial vehicle oblique photographing point clouds and ground laser radar bank slope point clouds of each observation time phase to control point cloud density, performing preliminary elimination on high-level extreme points to form a preprocessing point cloud, extracting corner points and edge points with obvious geometric features from the preprocessing point clouds, calculating three-dimensional feature descriptors, obtaining candidate matching point pair sets through feature descriptor similarity, applying RANSAC rigid body transformation estimation on the candidate matching point pair sets to obtain initial registration transformation of the multi-temporal unmanned aerial vehicle oblique photographing point clouds relative to the ground laser radar bank slope point clouds, performing ICP fine registration between the reference ground laser radar bank slope point clouds and the multi-temporal unmanned aerial vehicle oblique photographing point clouds to be registered, establishing a pairing point set through nearest neighbor search in each iteration, updating rigid body transformation parameters until the mean value change of the pairing point set is lower than a preset threshold, and obtaining the multi-source fusion river bank cloud under a unified coordinate system.
5. The method of claim 4, wherein the forming of the statistical filtering of the purified river bank three-dimensional point cloud and the geometric filtering based on the RANSAC plane fitting comprises the steps of establishing a neighborhood with a certain radius by taking each point as a center in the multi-source fusion river bank three-dimensional point cloud, calculating the distance statistic from the point to the mass center in the neighborhood, determining an abnormal distance range according to the mean value and the discrete degree of the distance statistic, eliminating the point with the distance statistic in the abnormal distance range as an isolated noise point, selecting a candidate water surface point set in a nearby water body area, applying the RANSAC plane model fitting to the candidate water surface point set, randomly extracting part of points in each round of RANSAC iteration, calculating the parameters of the plane model, counting the number of inner points meeting the plane model, calculating the distance from each point to the plane model after the plane model with the maximum number of inner points is determined, and integrally eliminating the point with the distance smaller than a preset water surface distance threshold as the water surface point, thereby obtaining the purified river bank three-dimensional point cloud expressing only the real form.
6. The method of claim 5, wherein constructing a river bank supervoxel time space graph structure comprises constructing a regular three-dimensional grid according to the spatial range of the purified river bank three-dimensional point cloud, taking points falling into the same grid unit as initial voxel units, calculating voxel centroid coordinates, local normal directions and local height differences for each initial voxel unit, merging adjacent voxel units with centroid distances lower than a neighborhood threshold value and with local normal directions being close and local height differences being limited step by using the initial voxel units as seed units in a region growing manner, determining centroid coordinates and basic attributes for each supervoxel node, establishing a spatial adjacent edge between the supervoxel nodes which are in a spatial adjacent range and have a shared boundary or surface adjacent relation in a three-dimensional space for the centroid distance in the same observation phase, and establishing a time adjacent edge between the adjacent observation phase for the supervoxel nodes with the spatial position of the centroid being close and the local normal direction difference being limited, thereby constructing the river bank supervoxel time space graph structure by using all the supervoxel nodes as graph nodes and the time adjacent edge as the time adjacent edges.
7. The method of claim 6, wherein in the construction of the multi-source space-time integrated feature vector, for each super-voxel node, an elevation average value of points in super-voxels is calculated in each observation time phase, and the elevation average value of adjacent observation time phases is differenced to form an elevation change feature sequence, for each super-voxel node, an envelope polyhedron is constructed and a volume is calculated for point clouds in super-voxels by utilizing a three-dimensional convex hull in each observation time phase, and the volume of adjacent observation time phases is differenced to form a volume change feature sequence, and for each super-voxel node, a local principal component analysis method is adopted, and a principal direction corresponding to the minimum feature value is used as a normal direction, and a normal change feature sequence is formed for an included angle in the normal direction of the adjacent observation time phases.
8. The method of claim 7, wherein, in constructing the multi-source space-time integrated feature vector, for each super-voxel node, the super-voxel interior points are projected onto the corresponding unmanned aerial vehicle oblique photographic image in each observation time phase, a vegetation index is calculated according to the image red-light wave band reflection value and the near-infrared wave band reflection value, and a vegetation index change feature sequence is formed by differentiating the vegetation index average value between adjacent observation time phases, for each super-voxel node, a water level distance feature sequence is formed according to the height difference between a water surface plane model and the super-voxel centroid in each observation time phase, for each super-voxel node, a quadric surface is fitted by taking the point cloud of the super-voxel node and the space adjacent super-voxel node as a local point set in each observation time phase, a curvature change feature sequence is formed by calculating a curvature value according to the main curvature of the quadric surface and differentiating between adjacent observation time phases, and the above feature sequences are cascaded into the multi-source space-time integrated feature vector with a fixed length in a predetermined order.
9. The method of claim 8, wherein the space-time diagram attention neural network comprises a plurality of cascade space-time diagram attention layers and a three-class output layer, each space-time diagram attention layer comprises a space attention subunit and a time attention subunit, and specifically, in the space attention subunit, any super voxel node in a river bank super voxel space-time diagram structure is taken as a target node, a multi-source space-time comprehensive feature vector of the target node and a multi-source space-time comprehensive feature vector of a space adjacent super voxel node connected with each space adjacent edge are respectively spliced and then input into a space relation mapping module, a one-dimensional space-correlation intermediate quantity is obtained through linear transformation and nonlinear activation calculation, each space-correlation intermediate quantity is converted into a space attention coefficient through a normalization function, the sum of all the space attention coefficients is one, and then the multi-source space-time comprehensive feature vector of each space adjacent super voxel node is summed according to the corresponding space attention coefficient, and the multi-source space-time comprehensive feature vector of the target node after the linear transformation is added to obtain a space update feature.

Description

River bank erosion three-dimensional deformation monitoring method based on multiple sensors Technical Field The invention belongs to the technical field of data processing and analysis, and particularly relates to a river bank erosion three-dimensional deformation monitoring method based on multiple sensors, belonging to an intelligent sensing system. Background River bank erosion is a common and long-term geomorphic evolution phenomenon, and is jointly influenced by various factors such as water flow scouring, slope instability induced by rainfall, ship wake disturbance, vegetation coverage change, river engineering activities and the like. The method has the advantages of high-precision and continuous three-dimensional deformation monitoring on river bank erosion, and has important significance for preventing disasters, evaluating river stability and guiding river bank remediation engineering. The traditional monitoring means mainly comprise modes of manual stepping investigation, displacement observation of a measuring pile, section level measurement and the like. Such a mode usually requires manual access to the site, is easily influenced by vegetation shielding, dangerous terrain and meteorological conditions, and has low monitoring efficiency and poor timeliness, so that dynamic changes of the river bank are difficult to continuously track on a large-scale and long-time scale. Meanwhile, the traditional mode is based on a discrete section or a small number of monitoring points, and a complete and continuous three-dimensional deformation process is difficult to reflect. With the development of unmanned aerial vehicle oblique photography technology, the construction of high-resolution three-dimensional point cloud by utilizing multi-view images through a motion restoration structure method has become a common means for river bank monitoring. The method can rapidly acquire data of a large range of river banks and generate a three-dimensional model with rich details. However, the accuracy of these three-dimensional models is highly dependent on image quality, lighting conditions and texture characteristics, and in areas where vegetation coverage is dense or slope texture is weak, the three-dimensional reconstruction accuracy is easily degraded. In addition, when multi-time phase change detection is performed only by relying on unmanned aerial vehicle oblique photographing point clouds, the overall accuracy is difficult to reach the centimeter level, and particularly when a fine erosion volume or potential landslide trend needs to be identified, the error is easily affected. The ground laser radar has the advantages of high precision, high density and strong permeability, and can effectively supplement the defects of unmanned aerial vehicle oblique photographing point clouds in complex terrains and low texture areas, in particular to areas below a steep bank and vegetation and close to a water body. However, the ground lidar has limitations in terms of spatial coverage, the number of viewing angles, and the measurement distance, it is generally difficult to cover the entire river bank area, and a complicated registration process is required between multi-station data. In practical application, if unmanned aerial vehicle oblique photography point cloud and ground laser radar point cloud are used in combination, the coverage range and the accuracy can be balanced. However, the existing multi-source point cloud fusion technology mostly adopts a simple registration method, is insufficient in noise identification, water surface point elimination and other treatments of the fused point cloud, and is easy to generate accumulated errors in final three-dimensional deformation analysis. Disclosure of Invention The invention mainly aims to provide a multi-sensor-based three-dimensional deformation monitoring method for river bank erosion, which can obtain a river bank erosion identification result which is more stable and finer and has space continuity. In order to solve the technical problems, the invention provides a multi-sensor-based river bank erosion three-dimensional deformation monitoring method, which comprises the steps of obtaining river bank images in a plurality of observation time phases by utilizing unmanned aerial vehicle oblique photography, obtaining multi-time-phase unmanned aerial vehicle oblique photography point clouds by SfM three-dimensional reconstruction, obtaining corresponding ground laser radar bank slope point clouds by utilizing a ground laser radar, and unifying the multi-time-phase unmanned aerial vehicle oblique photography point clouds and the ground laser radar bank slope point clouds into the same coordinate system by a global satellite navigation system GNSS control point; the method comprises the steps of fusing multi-time-phase unmanned aerial vehicle oblique photographing point clouds and ground laser radar bank slope point clouds in each observation time phase by adopting in