CN-121982226-A - Bridge digital twin disease mapping prediction method and system

CN121982226ACN 121982226 ACN121982226 ACN 121982226ACN-121982226-A

Abstract

The invention discloses a bridge digital twin disease mapping prediction method and system. The method comprises the steps of firstly obtaining multi-angle two-dimensional high-definition images and laser point clouds of a bridge, generating a watertight three-dimensional grid model through cascade registration fusion, secondly constructing a hierarchical bounding box acceleration structure based on a surface area heuristic strategy, accurately mapping two-dimensional disease pixels to the surface of the three-dimensional model by using a M-and-device-Trumbore ray tracing algorithm, finally converting the three-dimensional grid model after mapping diseases into a graph structure, constructing a space-time graph neural network formed by cascading a graph convolution layer and a bidirectional long-short-time memory network, extracting space association features and time sequence dependency features of the diseases, optimizing training through a weighted focus loss function, and outputting a risk thermodynamic diagram of the diseases. The invention solves the technical problems of two-dimensional and three-dimensional space disjoint, low massive data indexing efficiency and lack of dynamic prediction in the traditional detection, and realizes the refinement and intelligent management of the full life cycle of the bridge.

Inventors

SU MENG
LIU KE
ZHU YONGFA
ZHOU SHIJUN
XIONG WENHAO
LI NINGKANG
ZHU DEKAI
SUN TAO
LI CHUANG
WANG WEI
YUAN HAO
ZHAO JIONG
CAO WEN
ZHANG MINGTAO
TANG DENGWEI
DENG CHANGJUN

Assignees

中铁西南科学研究院有限公司

Dates

Publication Date: 20260505
Application Date: 20260409

Claims (10)

1. The bridge digital twin disease mapping prediction method is characterized by comprising the following steps of: s1, acquiring a multi-angle two-dimensional high-definition image and a laser point cloud of a bridge, aligning the laser point cloud and the point cloud generated based on the multi-angle two-dimensional high-definition image to the same coordinate system through cascade registration to obtain a fusion point cloud, sequentially performing downsampling and statistical filtering on the fusion point cloud to remove outlier noise points, estimating a normal line, setting a high-altitude virtual viewpoint for global orientation, and finally generating a watertight three-dimensional grid model through poisson surface reconstruction; s2, constructing a hierarchical bounding box space acceleration structure for the three-dimensional grid model based on a surface area heuristic strategy and combining a barrel-division scanning mechanism; S3, converting pixel coordinates of disease pixels in the multi-angle two-dimensional high-definition image into three-dimensional space rays by using an inner and outer parameter matrix of the camera, calculating intersection points of the three-dimensional space rays and triangular patches in the hierarchical bounding box accelerating structure by using a M-slope-edge-Trumbore algorithm, and mapping the disease pixels into UV texture coordinates of a three-dimensional grid model by using barycentric coordinate interpolation to obtain a three-dimensional grid model mapped with the disease; S4, converting the three-dimensional grid model mapped with the diseases into a graph structure, constructing a space-time graph neural network formed by cascading a graph convolution layer and a bidirectional long-short-time memory network, extracting space correlation features of the diseases by using the graph convolution layer, inputting the space correlation features into the bidirectional long-short-time memory network to extract time sequence dependency features, splicing forward and reverse hidden states to form space-time fusion features, performing model optimization training by using a weighted focus loss function, and finally outputting a risk thermodynamic diagram of the diseases.
2. The prediction method according to claim 1, wherein S1 specifically comprises: Aligning the laser point cloud and the point cloud generated based on the multi-angle two-dimensional high-definition image to the same coordinate system through cascade registration, extracting the fast point characteristic histogram characteristics of the two sets of point clouds, performing global coarse registration by using a random sampling consistency algorithm, and performing local fine registration by using a point-to-plane iterative closest point algorithm to obtain a fusion point cloud; Sequentially performing voxel downsampling and statistical filtering on the fusion point cloud, and calculating the average distance between each point in the fusion point cloud and k nearest neighbors Calculating a global average distance mean And standard deviation of When (when) Determining as outliers and eliminating, wherein A threshold coefficient preset according to the point cloud distribution characteristics; determining a local covariance matrix by adopting a mixed KD-Tree searching strategy, extracting a feature vector corresponding to a minimum feature value as a normal line through principal component analysis, and setting a high-altitude virtual viewpoint to force the normal line to be uniformly oriented; And reconstructing the point cloud with the normal line oriented through a poisson surface to generate a watertight three-dimensional grid model, and cutting out artifacts according to the sampling density mask.
3. The prediction method according to claim 1, wherein S2 specifically comprises: defining a surface area heuristic cost function : ; Wherein, the The overhead is traversed for the internal node, The cost of the intersection is calculated for the triangle, 、 The surface areas of the left sub-node bounding box and the right sub-node bounding box after being divided are respectively, 、 The number of triangles contained in the left child node and the right child node respectively, Surface area for bounding box of nodes before splitting; The nodes are tree structure units organized according to the space positions of bounding boxes when recursively dividing triangular patches of the three-dimensional grid model, and comprise leaf nodes for storing actual triangular patches and internal nodes containing child node bounding box information; uniformly dividing a triangular patch into a plurality of candidate barrels in the coordinate axis, only evaluating the cost function value of the barrel boundary position, selecting a segmentation scheme with the minimum cost function value to recursively generate child nodes, and controlling the time complexity to be equal to Where N is the total number of triangular patches in the three-dimensional mesh model.
4. The prediction method according to claim 1, wherein S3 specifically comprises: converting pixel coordinates into ray starting points under a world coordinate system through back projection transformation according to the camera internal reference matrix and the camera external reference matrix And direction vector ; For triangular patches in hierarchical bounding box acceleration structure According to the origin of the ray And direction vector Determining ray equations Simultaneous ray equation and triangle parameter equation Forming a linear system and solving the intersection distance by using the Cramer rule Barycentric coordinates of the intersection point Performing viewpoint back face elimination based on normal point multiplication in the solving process; According to triangular patches where intersection points are located Texture coordinates corresponding to three vertices of (a) Barycentric coordinates of the intersection point According to And calculating three-dimensional texture coordinates to obtain a three-dimensional grid model after mapping diseases.
5. The prediction method according to claim 1, wherein the step S4 of converting the three-dimensional grid model mapped with the diseases into a graph structure specifically comprises the steps of using vertexes of the three-dimensional grid model as graph nodes of the graph structure, constructing an adjacency matrix based on the space Euclidean distance between the nodes and a radial basis function, and when the distance between the nodes is smaller than a set threshold value, setting the adjacency matrix element as the radial basis function value, otherwise, setting the adjacency matrix element as 0.
6. The prediction method according to claim 1, wherein the step S4 of extracting spatial correlation features of the disease by using a graph convolution layer specifically comprises the steps of performing message transmission by using a spectrum domain graph convolution network, and aggregating second-order topological neighborhood features by using a cascade graph convolution layer to enable nodes to extract the spatial correlation features of the disease and perceive the propagation trend of a crack network.
7. The prediction method according to claim 1, wherein in S4, the spatial correlation features extracted by the graph convolution layer are input to a bidirectional long-short-time memory network unit, the cumulative effect of fatigue damage is captured in the forward direction, context constraints are provided in the reverse direction, and the space-time fusion features are output after the forward and reverse hidden states are spliced.
8. The prediction method according to claim 1, wherein the model optimization training by the weighted focus loss function in S4 comprises employing the weighted focus loss function As a training loss, where In order to be able to take the focus parameter as such, In order to dynamically adjust the class weights according to the positive and negative sample ratios, And carrying out parameter updating by adopting AdamW optimizers matched with cosine annealing learning rate for model optimization training.
9. The prediction method according to claim 1, wherein the step S4 of outputting the risk thermodynamic diagram of the disease specifically comprises the steps of generating the risk thermodynamic diagram of the surface of the three-dimensional grid model according to the space-time fusion characteristics output by the space-time diagram neural network, and dynamically displaying the space-time evolution trend of the disease.
10. A system for implementing the method of any one of claims 1 to 9, comprising: The data acquisition and preprocessing module is used for acquiring a multi-angle two-dimensional high-definition image and a laser point cloud of a bridge, aligning the laser point cloud and the point cloud generated based on the multi-angle two-dimensional high-definition image to the same coordinate system through cascade registration to obtain a fused point cloud, sequentially performing downsampling and statistical filtering on the fused point cloud to remove outlier noise points, estimating a normal line, setting a high-altitude virtual viewpoint for global orientation, and finally generating a watertight three-dimensional grid model through poisson surface reconstruction; the system comprises a space index and mapping module, a three-dimensional grid model, a three-dimensional space ray conversion module, a three-dimensional space ray generation module, a three-dimensional space image generation module and a three-dimensional space image generation module, wherein the space index and mapping module is used for constructing a hierarchical bounding box space acceleration structure for a three-dimensional grid model based on a surface area heuristic strategy and combining a barrel-division scanning mechanism; the intelligent evolution prediction module is used for converting the three-dimensional grid model mapped with the diseases into a graph structure, constructing a space-time graph neural network formed by cascading a graph convolution layer and a bidirectional long-short-time memory network, extracting space-related features of the diseases by using the graph convolution layer, inputting the space-related features into the bidirectional long-short-time memory network to extract time-sequence dependent features, splicing forward and reverse hidden states to form space-time fusion features, performing model optimization training by using a weighted focus loss function, and finally outputting a risk thermodynamic diagram of the diseases; the digital twin visualization module is used for rendering the three-dimensional grid model and the mapped disease texture and dynamically displaying the risk thermodynamic diagram.

Description

Bridge digital twin disease mapping prediction method and system Technical Field The invention relates to the technical field of civil engineering structure health monitoring, computer graphics and artificial intelligence crossing, in particular to a bridge digital twin disease mapping prediction method and system. Background The operation and maintenance safety of large bridges such as large-span suspension bridges and cable-stayed bridges is of great concern. The traditional manual inspection mode has the problems of large blind area, high risk coefficient, low efficiency and the like, and is difficult to meet the increasing operation and maintenance requirements of bridges. In recent years, unmanned aerial vehicles are increasingly popular by combining detection technology of computer vision, and a new technical means is provided for bridge disease detection. However, the existing bridge disease detection and management technology still faces the following technical problems in engineering application: Firstly, the two-dimensional image and the three-dimensional physical entity are spatially disjointed, so that disease positioning is fuzzy. Disease photographs taken by unmanned aerial vehicles are usually two-dimensional images, lacking global spatial coordinate information. The maintenance personnel are difficult to accurately correspond the local cracks in the mass photos to the actual bridge components such as sling positions and stiffening girder bottom plate areas, so that the diseases cannot be maintained at fixed points and tracked for a long time. This technical problem is essentially due to the dimensional gap between the two-dimensional visual data and the three-dimensional physical space, and the prior art lacks an efficient spatial mapping mechanism. Secondly, the mass point cloud and the grid data bring about rendering and calculation bottlenecks, and real-time interaction requirements are difficult to meet. Digital twin systems typically require loading of a hundred million level point cloud or a million level triangular patch, placing extremely high demands on computing resources. The traditional space index structure such as octree or midpoint segmentation hierarchical bounding box is easy to generate a large number of redundant blank nodes when processing unevenly distributed complex bridge components, so that the ray pickup and collision detection efficiency is extremely low. When engineers attempt to rotate, scale or pick disease areas in a digital twin model, the system response delay is severe, severely impacting the usability of the engineering application. Thirdly, the existing prediction model lacks of spatial topological association, and space-time evolution prediction cannot be realized. The existing disease prediction is often regarded as each measuring point in an isolated manner, and a single-point-based time sequence analysis such as ARIMA model is adopted. The mode ignores the spatial relevance and the spreading characteristics of the diseases of the bridge as a whole stressed structure. For example, a crack network caused by stress concentration at a certain place is often propagated along a stress path of a structure, and the spatial propagation rule cannot be captured simply by relying on single-point historical data, so that a significant deviation exists between a prediction result and actual disease evolution. In summary, how to realize accurate mapping from two-dimensional images to three-dimensional models under a digital twin framework, construct an efficient spatial index structure to support real-time interaction, and build a disease evolution prediction model with fusion spatial topological association becomes a technical problem to be solved in the art. Disclosure of Invention The invention aims to overcome the defects of space disjoint of a two-dimensional image and a three-dimensional model, low massive data indexing efficiency and lack of space-time evolution prediction in the prior art, and provides a bridge digital twin disease mapping prediction method and system. In order to achieve the above purpose, the technical scheme adopted by the invention is as follows: in one aspect, a method for mapping and predicting digital twin diseases of a bridge is provided, which comprises the following steps: s1, acquiring a multi-angle two-dimensional high-definition image and a laser point cloud of a bridge, aligning the laser point cloud and the point cloud generated based on the multi-angle two-dimensional high-definition image to the same coordinate system through cascade registration to obtain a fusion point cloud, sequentially performing downsampling and statistical filtering on the fusion point cloud to remove outlier noise points, estimating a normal line, setting a high-altitude virtual viewpoint for global orientation, and finally generating a watertight three-dimensional grid model through poisson surface reconstruction; s2, constructing a hierarchical bound