CN-122023703-A - Real-time three-dimensional semantic mapping and danger labeling method based on nerve radiation field

CN122023703ACN 122023703 ACN122023703 ACN 122023703ACN-122023703-A

Abstract

The application relates to the technical field of image processing and discloses a real-time three-dimensional semantic mapping and danger labeling method based on a nerve radiation field, which comprises the steps of obtaining RGB images and camera pose parameters; training a nerve radiation field model based on the method, extracting a three-dimensional density field of a current field of view and generating a local three-dimensional grid, carrying out semantic segmentation on an RGB image to obtain a two-dimensional semantic label, carrying out back projection on the two-dimensional semantic label to the local three-dimensional grid, determining the semantics of each point through multi-view fusion, constructing a local three-dimensional semantic map, carrying out risk detection and marking types and grades based on geometric and semantic risk rules, and fusing the marked local map to a global map and updating. The application constructs a high-quality three-dimensional map by utilizing multi-source fusion positioning and nerve radiation fields, and constructs a three-dimensional semantic map with dangerous area labeling by a dangerous labeling method.

Inventors

ZHANG XIAOBO
Sheng Yikai
ZHENG YUNPU
JIN ZEYU
WANG FUSONG
SUN BO

Assignees

西南交通大学

Dates

Publication Date: 20260512
Application Date: 20260413

Claims (7)

1. The real-time three-dimensional semantic mapping and danger labeling method based on the nerve radiation field is characterized by comprising the following steps of: acquiring an RGB image of an environment and camera pose parameters corresponding to each frame of RGB image; training a nerve radiation field model based on RGB images and camera pose parameters, extracting a three-dimensional density field in a current visual field range from the nerve radiation field model in the training process, and generating a local three-dimensional grid according to the three-dimensional density field; carrying out semantic segmentation on the RGB image to obtain a two-dimensional semantic tag of each frame of image; The two-dimensional semantic tags are back projected to the local three-dimensional grids according to camera pose parameters, semantic information of each point in the local three-dimensional grids is determined through multi-view fusion, and a local three-dimensional semantic map is constructed; Detecting a dangerous area of the local three-dimensional semantic map based on a preset geometric dangerous rule and a semantic dangerous rule, marking the dangerous type and the dangerous grade for the detected dangerous area, and obtaining a marked local three-dimensional semantic map; And fusing the marked local three-dimensional semantic map to a global map, and updating the global map.
2. The method for real-time three-dimensional semantic mapping and danger marking based on nerve radiation fields according to claim 1, wherein the camera pose parameters are obtained through a multi-source fusion positioning system, and the multi-source fusion positioning system comprises an inertial measurement unit, a visual odometer and a UWB base station and is used for realizing positioning in a GPS-free environment.
3. The method for real-time three-dimensional semantic mapping and risk labeling based on a neural radiation field according to claim 1, wherein the neural radiation field model is trained in an incremental manner, and the method comprises the following steps: Acquiring data through a sliding window, and taking an RGB image meeting the conditions and corresponding camera pose parameters as input to obtain the color and depth of each pixel point of a predicted image; Constructing a loss function according to the color and depth of each pixel point of the predicted image, and respectively optimizing pose and scene parameters based on the loss function by using an alternating optimization method; updating the image frames according to the optimized pose and scene parameters, and screening to obtain key frames based on preset conditions; based on the key frames, incremental training of the model is realized through sliding window optimization and progressive grid expansion.
4. The method for real-time three-dimensional semantic mapping and risk labeling based on a nerve radiation field according to claim 1, wherein the generating a local three-dimensional grid according to the three-dimensional density field comprises extracting an isosurface of a preset density threshold value from the three-dimensional density field by adopting an isosurface extraction algorithm, and discretizing the isosurface into a triangular grid formed by vertexes and patches, so as to obtain the local three-dimensional grid.
5. The method for real-time three-dimensional semantic mapping and danger labeling based on nerve radiation fields according to claim 1, wherein the determining semantic information of each point in the local three-dimensional grid through multi-view fusion comprises back projecting each three-dimensional point in the local three-dimensional grid into a multi-frame RGB image according to camera pose parameters, aggregating two-dimensional semantic information observed by images of different view angles, and determining final semantic information category of the three-dimensional point through a weighted voting method.
6. The method for real-time three-dimensional semantic mapping and risk labeling based on a nerve radiation field according to claim 1, wherein the geometric risk rule comprises risk judgment based on at least one of a gradient threshold, a height difference threshold and a surface roughness threshold; and the dangerous area detection judges whether the local three-dimensional semantic map triggers the geometric dangerous rule or the semantic dangerous rule by calculating and matching the geometric attribute and the semantic information of the local three-dimensional semantic map, and comprehensively evaluates the dangerous grade of the triggered area according to the rule type and the triggering degree.
7. The method for real-time three-dimensional semantic mapping and danger labeling based on the nerve radiation field according to claim 1 is characterized in that progressive updating is adopted for the global map, the method comprises the steps of calculating updating priorities of all areas in the global map according to time freshness, observation sufficiency and spatial position importance of data, and triggering a re-optimization flow for areas with the priorities meeting preset conditions, wherein the re-optimization flow comprises the steps of re-extracting local three-dimensional grids, re-carrying out semantic information fusion and re-carrying out danger area detection and labeling.

Description

Real-time three-dimensional semantic mapping and danger labeling method based on nerve radiation field Technical Field The application relates to the technical field of image processing, in particular to a real-time three-dimensional semantic mapping and danger labeling method based on a nerve radiation field. Background The mine environment is complex and dangerous, and the traditional inspection mode is difficult to adapt. The bionic hexapod robot imitates an insect movement mechanism, combines intelligent sensing and decision making technologies, can effectively cope with underground rugged ground, narrow road and burst risks, and improves safety and efficiency. The method comprises the steps of constructing a three-dimensional environment map containing semantic understanding and safety information in real time, wherein the three-dimensional environment map is a core foundation for supporting a robot to conduct autonomous decision making and path planning. In recent years, neural radiation field (NeRF) technology has received attention for its ability to reconstruct high-fidelity, continuous scene geometry and appearance from multi-view images. However, most of the conventional NeRF methods are used for reconstructing a scene in an off-line and non-real-time manner, have large calculation cost and long training time consumption, and are difficult to be directly applied to autonomous navigation systems with extremely high real-time requirements. In addition, the existing method mostly depends on preset two-dimensional image recognition rules or simple geometric height map analysis, and lacks comprehensive analysis capability on complex geometric features (such as curved surface gradient and continuous height change) and semantic contexts (such as water area ground attributes) of a three-dimensional space. This results in high false alarm rate and poor adaptability of the hazard detection, and it is difficult to implement real-time and accurate labeling in a dynamic unknown environment. Based on the problems and the shortcomings, the real-time three-dimensional semantic map construction and dangerous area marking method based on the nerve radiation field is provided, the bionic hexapod robot can run in real time in a GPS-free environment, the high-precision three-dimensional geometric reconstruction and semantic understanding depth are fused, and the integrated technical scheme of dangerous area analysis and marking can be automatically completed, so that the perception capability and navigation safety of an autonomous system in a complex unknown environment are improved. Disclosure of Invention In order to solve the problems, the application provides a real-time three-dimensional semantic mapping and danger marking method based on a nerve radiation field, which aims to solve the problems that the conventional nerve radiation field technology is difficult to be directly applied to an autonomous navigation system with extremely high real-time requirements, and the comprehensive analysis capability of complex geometric features and semantic contexts of a three-dimensional space is lacking in the aspect of dangerous region marking, so that the false alarm rate of danger detection is high, the adaptability is poor, and the real-time and accurate marking is difficult to be realized in a dynamic unknown environment. The first aspect of the embodiment of the invention provides a real-time three-dimensional semantic mapping and danger labeling method based on a nerve radiation field, which comprises the following steps: acquiring an RGB image of an environment and camera pose parameters corresponding to each frame of RGB image; training a nerve radiation field model based on RGB images and camera pose parameters, extracting a three-dimensional density field in a current visual field range from the nerve radiation field model in the training process, and generating a local three-dimensional grid according to the three-dimensional density field; carrying out semantic segmentation on the RGB image to obtain a two-dimensional semantic tag of each frame of image; The two-dimensional semantic tags are back projected to the local three-dimensional grids according to camera pose parameters, semantic information of each point in the local three-dimensional grids is determined through multi-view fusion, and a local three-dimensional semantic map is constructed; Detecting a dangerous area of the local three-dimensional semantic map based on a preset geometric dangerous rule and a semantic dangerous rule, marking the dangerous type and the dangerous grade for the detected dangerous area, and obtaining a marked local three-dimensional semantic map; And fusing the marked local three-dimensional semantic map to a global map, and updating the global map. In an alternative embodiment, the camera pose parameters are obtained by a multi-source fusion positioning system comprising an inertial measurement unit, a visual odometer and