CN-121999181-A - Disaster site-oriented multi-unmanned aerial vehicle rapid three-dimensional reconstruction and decision-making auxiliary method and related equipment

CN121999181ACN 121999181 ACN121999181 ACN 121999181ACN-121999181-A

Abstract

The application provides a disaster site-oriented multi-unmanned-plane rapid three-dimensional reconstruction and decision-making auxiliary method and related equipment, and belongs to the technical field of emergency mapping and three-dimensional reconstruction. The method comprises the steps of cooperatively collecting image data of a disaster scene by using multiple unmanned aerial vehicles, constructing an uncertainty index for representing the data deletion degree, taking the uncertainty index as input, selecting an optimal viewpoint to generate a complementary acquisition task and execute the complementary acquisition task based on the contribution degree of candidate viewpoints to reducing uncertainty and the comprehensive cost of executing the viewpoints, obtaining an image data set with coverage integrity, screening a key frame set for three-dimensional reconstruction from the image data set, carrying out motion recovery structure and space partitioning based on the key frame set, fusing depth priori and semantic information to obtain an initialization point cloud, extracting structural elements based on a three-dimensional Gaussian sputtering model, constructing a risk cost map, and obtaining a rescue route and a disaster decision report. The application can support emergency command and rescue actions.

Inventors

LIU PENG
ZHANG JIAN
ZHAO LU
ZONG JIZHOU
GUO LEI
TIAN YUCHUN
LUO WENQIANG
HUANG YULIN
DENG CHAOYI
OUYANG YANG

Assignees

中国能源建设集团广东省电力设计研究院有限公司
广东科诺勘测工程有限公司

Dates

Publication Date: 20260508
Application Date: 20260407

Claims (10)

1. The rapid three-dimensional reconstruction and decision-making auxiliary method of the multi-unmanned aerial vehicle facing the disaster site is characterized by comprising the following steps of: The method comprises the steps of cooperatively collecting image data of a disaster site by using a plurality of unmanned aerial vehicles, discretizing a target area into grid cells, and constructing an uncertainty index for representing the degree of data deletion based on the observation coverage condition, imaging quality and semantic importance of each grid cell; The uncertainty index is used as input, a high uncertainty area, in which the uncertainty index exceeds a preset threshold, is identified, and for the high uncertainty area, an optimal viewpoint is selected to generate a supplementary acquisition task and is executed based on the contribution degree of candidate viewpoints to uncertainty reduction and the comprehensive cost of executing viewpoints, so that an evaluation-driven acquisition closed loop is formed, and an image data set covering completeness is obtained; Screening a key frame set for three-dimensional reconstruction from the image data set by integrating image quality and coverage contribution; Performing motion recovery structure and space blocking based on the key frame set, and fusing depth priori and semantic information to obtain an initialization point cloud; constructing a semantic three-dimensional Gaussian sputtering model based on the initialization point cloud; And based on the three-dimensional Gaussian sputtering model, extracting structural elements, and constructing a risk cost map to obtain a rescue route and a disaster decision report.
2. The method of claim 1, wherein the cooperatively acquiring image data of the disaster site using the multiple unmanned aerial vehicles, discretizing the target area into grid cells, and constructing an uncertainty index for representing the degree of data missing based on the observed coverage, the imaging quality and the semantic importance of each grid cell, comprises: the method comprises the steps of cooperatively collecting image data of a disaster site by using a plurality of unmanned aerial vehicles, discretizing a target area into a plurality of grid cells, and constructing a normalized uncertainty index for each grid cell based on semantic importance, effective observation times, view angle baseline sufficiency and comprehensive imaging quality of the grid cell, wherein the uncertainty index is used for quantifying the data deletion degree and the complement priority of the grid cell; multiplying the weight of the semantic importance by a result obtained by weighted summation of an observation times attenuation term, a view angle baseline insufficient term and an imaging quality insufficient term; The weight of the semantic importance is used for representing the priority of areas of roads, bridges, dykes and dams, personnel-intensive areas and disaster boundaries meeting preset conditions; The observation times attenuation term is calculated in an exponential decay form based on the effective observation times, is used for representing the speed of uncertainty reduction along with the increase of the observation times, and is the number of observation sets available for the grid cells under the mass and geometric constraints; the visual angle baseline sufficiency index is used for measuring whether the included angle between different observation visual lines and baseline distribution meet the geometric conditions of triangulation and surface recovery or not; The imaging quality deficiency term is obtained by subtracting a comprehensive imaging quality index, wherein the comprehensive imaging quality index is used for representing the influence of definition, exposure, smoke, rain and fog shielding and motion blur factors on the imaging quality; the observation times attenuation item is configured with a first weight coefficient, wherein the first weight coefficient is used for adjusting the duty ratio of the observation times attenuation item in an uncertainty index according to the field task type; the visual angle baseline insufficient item is configured with a second weight coefficient, and the second weight coefficient is used for adjusting the duty ratio of the visual angle baseline insufficient item in an uncertainty index according to the field task type; the imaging quality insufficient item is configured with a third weight coefficient, and the third weight coefficient is used for adjusting the duty ratio of the imaging quality insufficient item in the uncertainty index according to the field task type.
3. The method of claim 1, wherein the taking the uncertainty index as an input, identifying a high uncertainty region for which the uncertainty index exceeds a preset threshold, selecting an optimal viewpoint to generate a complement task and performing based on a contribution of candidate viewpoints to reducing uncertainty and a comprehensive cost of performing viewpoints for the high uncertainty region, forming an evaluation-driven acquisition closed loop, and obtaining an image dataset of coverage integrity, comprises: The uncertainty index is used as input to identify a high uncertainty area with the uncertainty index exceeding a preset threshold value, wherein the high uncertainty area is an area with data missing or insufficient coverage; Selecting an optimal view point to generate a complementary acquisition task and executing the complementary acquisition task according to the contribution degree of the candidate view point to the uncertainty reduction and the comprehensive cost of executing the candidate view point aiming at the high uncertainty region; The information gain function is calculated based on the sum of products of the uncertainty indexes and the visibility functions of all grid cells which can be observed by the candidate view point; the information gain function is used for quantifying the total amount of uncertainty eliminated by the complementary extraction from the candidate viewpoint; The comprehensive cost is formed by weighted summation of time cost, energy consumption cost and risk cost; the time cost is used for representing the transition time, the climbing time, the descending time and the time consumption corresponding to the route length required by executing the supplementary mining task, the energy consumption cost is used for representing the battery consumption required by executing the supplementary mining task, and the risk cost is used for representing the safety risks of a no-fly zone, an obstacle dense zone, a strong wind zone, a low-altitude cable and a smoke dust high risk zone facing the execution of the supplementary mining task; Selecting the candidate viewpoint with the maximum difference of the information gain calculated by the information gain function minus the comprehensive cost from a feasible viewpoint set meeting preset unmanned aerial vehicle range, communication, flight prohibition and task constraints as the optimal viewpoint; and after the complementary acquisition task is executed, fusing the newly acquired image data with the original image data to obtain an image data set with coverage integrity.
4. The method of claim 1, wherein said screening out a set of keyframes for three-dimensional reconstruction from said image dataset, integrating image quality, overlay contribution, comprises: comprehensively scoring each frame of image from the image data set, and screening a key frame set for three-dimensional reconstruction according to the comprehensive score; the comprehensive score is obtained by weighting and summing image quality scores, coverage contribution scores, baseline contribution scores, important area weighting scores and pose credibility scores; The image quality score is used for comprehensively measuring the definition, exposure condition, smoke, rain and fog shielding degree and motion blur degree of the image; The coverage contribution component is used for measuring the newly increased coverage of the frame image to the high uncertainty area or the uncovered area; the baseline contribution score is used for representing the visual angle difference and baseline length gain between the frame image and the selected key frame set; The important area weighting is used for improving the sampling density of roads, bridges, ponding boundaries, landslide bodies, collapse buildings and people or vehicle gathering areas to a preset range; the pose reliability is used for reducing the interference of images with poor pose quality to a preset range, wherein the poor pose quality comprises the conditions of dynamic differential positioning lock losing or visual positioning drifting; The image quality score, the coverage contribution score, the baseline contribution score, the important area weighting score and the pose credibility score are respectively provided with an adjustable weight coefficient for adjusting the duty ratio of each factor in the comprehensive score according to the field task type and the calculation power configuration; and reserving the images with the comprehensive scores higher than a preset threshold as the key frame set through the comprehensive scores.
5. The method of claim 1, wherein the performing motion restoration structure and spatial partitioning based on the keyframe set, fusing depth priors and semantic information, and obtaining an initialization point cloud, comprises: performing motion restoration structure processing based on the keyframe set to obtain sparse point cloud and camera pose; calculating a scene bounding box based on the sparse point cloud, and recursively dividing a three-dimensional space by utilizing an octree to obtain a plurality of sub-blocks; for each sub-block, constructing a camera view cone of each key frame based on the camera pose, and screening key frames with intersections of the camera view cone and the space of the sub-block from the key frame set to be used as a visible key frame set of the sub-block; Selecting a key frame subset with a base line larger than a preset value from the key frame set, generating a pixel-level depth map by using a depth prediction model for each selected frame, and back-projecting pixel coordinates to a camera coordinate system based on camera internal parameters to obtain three-dimensional points under the camera coordinate system; Based on a rotation matrix and a translation vector contained in the camera pose, transforming the three-dimensional points under the camera coordinate system to a world coordinate system to obtain the three-dimensional points under the world coordinate system, and performing fusion, denoising and downsampling on point clouds obtained by back projection of a plurality of key frames in the key frame subset to obtain an initialized point cloud.
6. The method of claim 5, wherein the constructing a semantically three-dimensional gaussian sputtering model based on the initialization point cloud comprises: Converting the initialization point cloud into an initial Gaussian set of three-dimensional Gaussian sputtering, wherein each Gaussian point takes the three-dimensional position of the Gaussian point as a mean value, and endowing initial scale parameters and color parameters; Clipping Gaussian subsets in the space range of each sub-block from the initial Gaussian set to serve as starting points of sub-block training; Generating a semantic graph for the key frames in the key frame set by using a visual language model, wherein the semantic graph comprises semantic category information at a pixel level; For each sub-block, based on the visible key frame set of the sub-block, carrying out three-dimensional Gaussian sputtering training by taking the Gaussian subset as a starting point, and constructing a joint loss function in the training process to optimize, wherein the joint loss function comprises luminosity loss, semantic loss and shape regular terms, the luminosity loss is used for restraining consistency of a color image obtained by rendering and an original image, the semantic loss is used for restraining consistency of semantic probability distribution obtained by rendering and the semantic image generated by a visual language model, and the shape regular terms are used for restraining the anisotropism degree of Gaussian points; and optimizing the joint loss function, obtaining the optimized Gaussian subset by each sub-block, and fusing the Gaussian subsets of each sub-block and performing boundary consistency processing to obtain a semantic three-dimensional Gaussian sputtering model containing geometric information and semantic information.
7. The method of claim 1, wherein the extracting structural elements based on the three-dimensional gaussian sputtering model, constructing a risk cost map, and obtaining rescue routes and disaster decision reports, comprises: Based on the three-dimensional Gaussian sputtering model, extracting structural elements in space through a three-dimensional semantic field, and converting the structural elements into a vectorized or gridded expression form, wherein the structural elements comprise a road communication area, a water body or water accumulation area, a fire point and smoke influence area, a collapse building contour, a landslide body boundary and personnel or vehicle point positions; The method comprises the steps of constructing a risk cost map based on the structural elements and the geometric features of the terrain, dividing a scene into spatial positions of an unviewable region, a restricted passage region and a safe passage region by the risk cost map, and defining comprehensive cost for each spatial position, wherein the comprehensive cost consists of a distance or time cost item, a risk cost item and a passage obstacle cost item, the risk cost item is comprehensively obtained by semantic elements of landslide body proximity, ponding depth, smoke coverage intensity and a high-risk region of a secondary disaster, and the passage obstacle cost item is used for expressing an unviewable region with broken circuit, collapse accumulation or excessively narrow road; Searching a minimum total cost path from a starting point to a target point based on the risk cost map, outputting a main route and an alternative route, and giving out detour reasons and risk explanation of each route to generate a disaster situation decision report, wherein the disaster situation decision report comprises an affected area range, key risk point distribution, personnel and vehicle distribution statistics, road accessibility conclusion, risk level distribution thermodynamic diagram, recommended action route, predicted time consumption and key material putting point suggestion.
8. A disaster site oriented multi-unmanned aerial vehicle rapid three-dimensional reconstruction and decision assistance system for implementing the method of any one of claims 1 to 7, the system comprising: The system comprises a first module, a second module and a third module, wherein the first module is used for cooperatively acquiring image data of a disaster scene by using a plurality of unmanned aerial vehicles, discretizing a target area into grid cells, and constructing an uncertainty index for representing the degree of data deletion based on the observation coverage condition, imaging quality and semantic importance of each grid cell; The second module is used for taking the uncertainty index as input, identifying a high uncertainty area of which the uncertainty index exceeds a preset threshold value, selecting an optimal viewpoint to generate a complementary acquisition task and executing the complementary acquisition task based on the contribution degree of the candidate viewpoint to the reduction of uncertainty and the comprehensive cost of the execution viewpoint aiming at the high uncertainty area, forming an acquisition closed loop of evaluation driving, and obtaining an image data set of coverage completeness; the third module is used for integrating image quality and coverage contribution from the image data set and screening out a key frame set for three-dimensional reconstruction; A fourth module, configured to perform motion recovery structure and space partitioning based on the keyframe set, and fuse depth priori and semantic information to obtain an initialized point cloud; A fifth module for constructing a semantically three-dimensional Gaussian sputtering model based on the initialization point cloud; And a sixth module, configured to extract structural elements based on the three-dimensional gaussian sputtering model, construct a risk cost map, and obtain a rescue route and a disaster decision report.
9. An electronic device comprising a memory storing a computer program and a processor implementing the method of any of claims 1 to 7 when the computer program is executed by the processor.
10. A computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the method of any one of claims 1 to 7.

Description

Disaster site-oriented multi-unmanned aerial vehicle rapid three-dimensional reconstruction and decision-making auxiliary method and related equipment Technical Field The application relates to the technical field of emergency mapping and three-dimensional reconstruction, in particular to a disaster site-oriented multi-unmanned-plane rapid three-dimensional reconstruction and decision-making auxiliary method and related equipment. Background In the related art, a fixed route is adopted for blind flight, and a data blind area cannot be perceived in the acquisition process. Once the model cavity is caused by shielding, the flight is required to be manually found and reorganized in the later period, and the rescue progress is seriously dragged. In summary, the technical problems in the related art are to be improved. Disclosure of Invention The embodiment of the application mainly aims to provide a disaster site-oriented multi-unmanned-plane rapid three-dimensional reconstruction and decision-making auxiliary method and related equipment, which can realize three-dimensional modeling of disaster sites, automatically generate a decision-making scheme and support emergency command and rescue actions. In order to achieve the above purpose, an aspect of the embodiments of the present application provides a disaster site-oriented multi-unmanned aerial vehicle rapid three-dimensional reconstruction and decision-making assistance method, which includes the following steps: The method comprises the steps of cooperatively collecting image data of a disaster site by using a plurality of unmanned aerial vehicles, discretizing a target area into grid cells, and constructing an uncertainty index for representing the degree of data deletion based on the observation coverage condition, imaging quality and semantic importance of each grid cell; The uncertainty index is used as input, a high uncertainty area, in which the uncertainty index exceeds a preset threshold, is identified, and for the high uncertainty area, an optimal viewpoint is selected to generate a supplementary acquisition task and is executed based on the contribution degree of candidate viewpoints to uncertainty reduction and the comprehensive cost of executing viewpoints, so that an evaluation-driven acquisition closed loop is formed, and an image data set covering completeness is obtained; Screening a key frame set for three-dimensional reconstruction from the image data set by integrating image quality and coverage contribution; Performing motion recovery structure and space blocking based on the key frame set, and fusing depth priori and semantic information to obtain an initialization point cloud; constructing a semantic three-dimensional Gaussian sputtering model based on the initialization point cloud; And based on the three-dimensional Gaussian sputtering model, extracting structural elements, and constructing a risk cost map to obtain a rescue route and a disaster decision report. In some embodiments, the cooperatively acquiring image data of a disaster site by using multiple unmanned aerial vehicles, discretizing a target area into grid cells, and constructing an uncertainty index for representing the degree of data missing based on the observed coverage condition, the imaging quality and the semantic importance of each grid cell, including: the method comprises the steps of cooperatively collecting image data of a disaster site by using a plurality of unmanned aerial vehicles, discretizing a target area into a plurality of grid cells, and constructing a normalized uncertainty index for each grid cell based on semantic importance, effective observation times, view angle baseline sufficiency and comprehensive imaging quality of the grid cell, wherein the uncertainty index is used for quantifying the data deletion degree and the complement priority of the grid cell; multiplying the weight of the semantic importance by a result obtained by weighted summation of an observation times attenuation term, a view angle baseline insufficient term and an imaging quality insufficient term; The weight of the semantic importance is used for representing the priority of areas of roads, bridges, dykes and dams, personnel-intensive areas and disaster boundaries meeting preset conditions; The observation times attenuation term is calculated in an exponential decay form based on the effective observation times, is used for representing the speed of uncertainty reduction along with the increase of the observation times, and is the number of observation sets available for the grid cells under the mass and geometric constraints; the visual angle baseline sufficiency index is used for measuring whether the included angle between different observation visual lines and baseline distribution meet the geometric conditions of triangulation and surface recovery or not; The imaging quality deficiency term is obtained by subtracting a comprehensive imaging quality index, wherein the