Search

CN-121977538-A - ROS-based cross-platform collaborative hierarchical hybrid map SLAM system and method

CN121977538ACN 121977538 ACN121977538 ACN 121977538ACN-121977538-A

Abstract

The application discloses a cross-platform collaborative hierarchical hybrid map SLAM system and method based on ROS, and belongs to the field of collaboration of a plurality of unmanned platforms. The application provides a targeted solution around a multi-source data calibration, closed-loop optimization and scene characterization technology, and improves calibration precision, global pose consistency and scene reconstruction effect by means of innovation of 3DGS-NeRF mixed characterization conflict resolution of fusion feature-weight dual-drive calibration, observation association enhancement type cross-platform closed-loop optimization and residual error reconciliation, so that high-efficiency and high-precision multi-source data fusion processing is realized. The system and the method are based on a full-link cooperative mechanism of data calibration-pose optimization-representation fusion, adopt a full-link data flow of data input-calibration pretreatment-pose optimization-representation fusion-result output, and realize high-efficiency interaction and self-adaptive adjustment among the three modules through four control mechanisms of data flow, dynamic triggering, resource scheduling and error feedback.

Inventors

  • LIU DAN
  • GUO HUAZE
  • WANG FULIN
  • ZHANG YU
  • ZHANG YINGYING
  • ZHANG JIAN
  • WANG ZHANGU

Assignees

  • 山东科技大学

Dates

Publication Date
20260505
Application Date
20260409

Claims (10)

  1. 1. A cross-platform collaborative hierarchical hybrid map SLAM system based on ROS is characterized in that based on a full-link collaborative mechanism of data calibration-pose optimization-characterization fusion, the fusion and processing of multi-mode ROS Topic data are realized through linkage and collaboration among a multi-mode feature-weight driving calibration module, an observation association enhanced cross-platform closed-loop optimization module and a residual reconciliation 3DGS-NeRF hierarchical hybrid characterization conflict resolution module; The multi-modal feature-weight driving calibration module fuses cross-platform hash feature association and a two-dimensional dynamic weight distribution mechanism to realize space-time reference unification of multi-source data, and provides basic data input for subsequent pose optimization and characterization fusion; the observation association enhancement type cross-platform closed-loop optimization module introduces an observation association enhancement mechanism, and realizes accurate correction of global pose through a graph optimization framework by means of motion consistency and feature association double-loss constraint; The residual-blended 3DGS-NeRF hierarchical hybrid representation conflict resolution module realizes dimension unification and semantic compatibility of various representations by constructing an MLP explicit-implicit representation conversion bridge, relieves representation differences by NeRF residual feature blending strategies, combines geometric consistency and residual regularization double-loss constraint, resolves fusion conflicts to consider real-time rendering advantages of 3D GS and NeRF, and meets the requirements of collaborative map building and scene visualization; and a cooperative working mechanism is adopted for the three modules, a full-link data flow including data input, calibration preprocessing, pose optimization, characterization fusion and result output is adopted, and high-efficiency interaction and self-adaptive adjustment among the three modules are realized through four control mechanisms including data flow, dynamic triggering, resource scheduling and error feedback.
  2. 2. A ROS-based cross-platform collaborative hierarchical hybrid map SLAM method employing the ROS-based cross-platform collaborative hierarchical hybrid map SLAM system of claim 1, comprising, Step (1), multi-source sensor data calibration processing; based on a multi-mode characteristic-weight driving calibration module, performing multi-source sensor data calibration processing by adopting a characteristic driving cross-platform association and weight driving dynamic allocation combined mode; step (2), global pose correction processing; Based on an observation association enhanced cross-platform closed-loop optimization module, performing global pose correction processing by combining closed-loop candidate detection, observation association enhanced matching and double-loss constraint global optimization; Step (3), characterization fusion and conflict resolution processing; 3DGS-NeRF hierarchical hybrid representation conflict resolution module based on residual reconciliation is used for carrying out representation fusion and conflict resolution processing by adopting hierarchical representation extraction, MLP explicit-implicit representation conversion bridge construction and NeRF residual feature reconciliation double-loss constraint conflict resolution in combination; step (4), the modules work cooperatively; The method is characterized in that the data interaction and the self-adaptive adjustment of a multi-mode characteristic-weight dual-drive calibration module, an observation association enhanced cross-platform closed-loop optimization module and a residual reconciliation 3DGS-NeRF hierarchical hybrid representation conflict resolution module are realized based on data flow, dynamic triggering, resource scheduling and an error feedback mechanism.
  3. 3. The cross-platform collaborative hierarchical hybrid map SLAM method based on ROS of claim 2, wherein said step (4) comprises, Step (4.1), data circulation; The multi-mode feature-weight dual-drive calibration module, the observation association enhanced cross-platform closed-loop optimization module and the residual-blended 3DGS-NeRF hierarchical hybrid representation conflict resolution module realize the full-link data flow of data input-calibration-optimization-fusion-output through the ROS Topic; Step (4.2), dynamically triggering; According to the related flow of dynamic triggering of the system state and the data quality, the triggering condition is judged to be the core by using a threshold value, and the formula is as follows: (63) Wherein, the The trigger flow is indicated as being a function of the trigger flow, Indicating that the trigger is not activated; the matching score is represented as such, The value of the error is indicated and, Representing the computational load of the hardware, 、 、 Representing a corresponding decision threshold; Step (4.3), resource scheduling; Based on the distributed architecture and the hardware computing power state, the efficient utilization of computing/storage resources is realized; step (4.4), error feedback; And establishing a full-link error monitoring and feedback mechanism, and continuously optimizing system parameters and algorithm logic through real-time acquisition, analysis and feedback of the multidimensional error index.
  4. 4. The cross-platform collaborative hierarchical hybrid map SLAM method based on ROS of claim 3, wherein the data flow logic relationship in step (4.1) comprises, The original data of the multi-source sensor is published to the original ROS Topic, and the calibration module subscribes to be used as input; the calibration module outputs standardized data and issues the standardized data to/calibrated/data as basic input of the optimization module and the fusion module; the optimization module outputs a global optimization pose and issues the global optimization pose to the optimized/pose for correcting the coordinate reference of the fusion module; the fusion module outputs a mixed characterization result and issues the mixed characterization result to the fusion module for collaborative positioning map building and scene rendering; the fusion module calculates the geometric consistency error, and issues the geometric consistency error to the fusion/error, and the calibration module subscribes and serves as a secondary calibration triggering basis.
  5. 5. The cross-platform collaborative hierarchical hybrid map SLAM method based on ROS of claim 3, wherein the triggering rules in step (4.2) include, Closed loop optimization triggering: Or (b) In the time-course of which the first and second contact surfaces, Triggering global pose optimization and synchronously updating the coordinate reference of the fusion module; and (3) triggering secondary calibration: Or (b) Or (b) In the time-course of which the first and second contact surfaces, Triggering a calibration module to calibrate secondarily; And (3) resource adjustment triggering: Or (b) In the time-course of which the first and second contact surfaces, Triggering a hot resource scheduling mechanism.
  6. 6. The cross-platform collaborative hierarchical hybrid map SLAM method based on ROS of claim 3, wherein the core schedule in step (4.3) includes, A. cross-platform resource allocation, namely, an unmanned vehicle main node bears a core computing task and performs calculation amount allocation according to the following formula: (64) Wherein, the 、 Respectively represents the calculated amount of the unmanned aerial vehicle and the unmanned aerial vehicle, Representing the core calculation amount distribution coefficient, Representing the total calculated amount of the system; b. Dynamic resource adjustment, dynamically adjusting algorithm parameters according to hardware load and 3DGS Gaussian distribution quantity Number of NeRF network neurons The adjustment formula is as follows: (65) Wherein, the 、 Representing the parameter values after/before adjustment, Represents the adjustment coefficient of the device, , Representing a load threshold; c. The storage resources are managed as follows: (66) Wherein, the Indicating the current storage usage rate and, Representing the memory size occupied by the key frame database, Representing the memory size occupied by the feature library, Representing the total storage capacity when When the data cleaning is performed, the following is performed: (67) Wherein, the Representing the total amount of stored data, Representing the earliest invalid data item in question, Representing data remaining after cleaning.
  7. 7. The cross-platform collaborative hierarchical hybrid map SLAM method based on ROS of claim 3, wherein said step (4.4) comprises, Step (4.4.1), calculating a multi-dimensional error monitoring index; three core indexes of calibration errors, pose optimization errors and fusion errors are defined, and the formula is as follows: (68) Wherein, the 、 Representing the spatial error and the temporal error of the calibration module; 、 Representing the rotation error and the translation error of the optimization module; 、 representing the geometric consistency error and residual error of the fusion module; step (4.4.2), error feedback and parameter adjustment; When (when) 、 、 、 Or (b) And triggering parameter adjustment or flow re-execution of the corresponding module, wherein an adjustment formula is as follows: (69) Wherein, the 、 Indicating that the module parameters before/after adjustment, Represents the adjustment coefficient of the device, , The current error value is indicated and, Representing an error threshold; The specific feedback logic is that if Or (b) Triggering a calibration module to calibrate for the second time, increasing the feature extraction quantity and expanding the size of a sliding window; If it is Or (b) Adding a closed loop side weight coefficient, re-executing global pose optimization, and triggering re-closed loop detection if the iteration is still not satisfied for 3 times; If it is Or (b) Adjusting MLP conversion bridge network parameters and self-adaptive residual error weight initial values, and re-executing the characterization fusion process; step (4.4.3), a long-term optimization mechanism; and (3) periodically counting error data of each module, and averagely mining an error change trend through a sliding window, wherein the formula is as follows: (70) Wherein, the The average value of the errors in the window is indicated, Indicating the length of the sliding window, Represent the first An error value of the time; Based on The change trend optimization principle of (1) is that if Continuously rise Analyzing error source, optimizing algorithm core logic, if Stabilize around a threshold value Parameters are finely tuned to promote system robustness.
  8. 8. The cross-platform collaborative hierarchical hybrid map SLAM method based on ROS of claim 2, wherein said step (1) comprises, Step (1.1), ROS Topic data pretreatment; converting the multisource original data of the unmanned vehicles and the unmanned vehicles to standardized data which represents a unified format, is aligned in time and space and has low noise; step (1.2), cross-platform association of feature drive; By extracting multi-modal features, constructing a cross-platform hash feature library and calculating feature matching similarity, accurate matching of unmanned vehicles and unmanned vehicle homologous data is realized, and the space-time calibration is represented to provide association constraint; Step (1.3), dynamic allocation of weight driving; dynamically distributing weight coefficients according to the reliability of sensor data and the motion state of a platform, wherein the weight coefficients comprise three stages of sensor confidence weight calculation, motion mode weight distribution and total dynamic weight fusion; Step (1.4), driving, calibrating and fusing; The feature association result and the dynamic weight are fused, and the cross-platform space-time transformation matrix is solved through least square optimization, so that space-time reference unification of multi-source data is realized, and the method comprises three stages of optimizing objective function construction, optimizing solving process, calibrating result verification and feedback.
  9. 9. The ROS-based cross-platform collaborative hierarchical hybrid map SLAM method of claim 2, wherein: The step (2) comprises the steps of, Step (2.1), closed loop candidate detection; The method comprises the steps of quickly searching potential closed-loop candidate frames overlapped with a current frame in a scene from historical key frames, wherein the potential closed-loop candidate frames comprise three stages of global feature extraction, word bag model construction and search and candidate frame geometric verification; Step (2.2), observing closed loop matching with enhanced association; strengthening cross-modal feature coordination and cross-platform feature mapping, and designing a differential matching strategy aiming at intra-platform/cross-platform closed loops, wherein the differential matching strategy comprises three stages of intra-platform closed loop matching and cross-platform closed loop matching, and matching result optimization; step (2.3), global pose optimization with double loss constraint; Based on closed-loop constraint and odometer constraint, motion consistency loss and feature relevance loss are minimized through a graph optimization framework, and global pose accurate correction is achieved, wherein the method comprises four stages of optimizing graph construction, double-loss constraint design, optimizing solution and result output, optimizing result verification and iteration.
  10. 10. The cross-platform collaborative hierarchical hybrid map SLAM method based on ROS of claim 2, wherein said step (3) comprises, Step (3.1), extracting layering characterization; the method comprises the steps of respectively extracting 3DGS explicit characterization and NeRF implicit characterization to provide a basis for fusion, wherein the two steps of 3DGS explicit characterization extraction and NeRF implicit characterization extraction are included; Step (3.2), MLP explicit-implicit characterization conversion bridge construction; construction of three-layer MLP conversion bridge Converting the 3DGS explicit characterization parameters into feature vectors consistent with NeRF implicit characterization dimensions, so as to realize dimension unification and semantic compatibility; step (3.3), neRF residual feature reconciliation; calculating residual vectors of the converted 3DGS features and NeRF implicit features, and generating fusion features through self-adaptive residual weight and semantic difference, wherein the three stages comprise residual vector calculation, self-adaptive residual weight calculation and fusion feature generation; step (3.4), conflict resolution of double loss constraint; based on geometric consistency loss and residual regularization loss, geometric conflict and feature redundancy in the fusion process are eliminated through back propagation optimization, and the method comprises three stages of loss function construction, parameter optimization process and fusion result output.

Description

ROS-based cross-platform collaborative hierarchical hybrid map SLAM system and method Technical Field The application relates to the field of collaboration of a plurality of unmanned platforms, and particularly provides an adaptive ROS (Robot Operating System) system, a layered hybrid map SLAM system and a layered hybrid map SLAM method for co-locating and multi-source data fusion of a plurality of unmanned platforms (such as unmanned vehicles and unmanned planes). Background The simultaneous localization and mapping SLAM method is a technique that uses multi-sensor (e.g., camera, lidar, IMU, etc.) data to estimate sensor pose and simultaneously reconstruct a map of the surrounding environment. The SLAM technology can solve the problems of autonomous positioning and navigation of a robot or equipment in an unknown environment, and is one of key technologies for realizing a fully autonomous mobile robot. The existing ROS system is a core development and operation framework of intelligent mobile platforms such as unmanned vehicles, unmanned planes and the like, realizes the distribution and interaction of multi-sensor data through a Topic mechanism, and is the basis of multi-platform co-location and mapping. In the unmanned vehicle-unmanned plane cooperative scene, the multi-mode sensor data such as a laser radar, a camera, an IMU and the like are required to be fused, and the sensor installation positions and the working characteristics of different platforms are large in difference, so that the problems of time-space asynchronism, inconsistent precision and the like of ROS (reactive oxygen species) Topic data are caused, and the calibration precision directly influences the subsequent data fusion effect. The multi-mode data calibration is a key preprocessing link of a collaborative system, in the prior art, a single-feature driving or fixed weight driving calibration mode is adopted, wherein the feature driving method depends on single-mode feature matching and is easily influenced by environmental illumination and texture change, and the weight driving method adopts static weight distribution and cannot adapt to platform motion mode switching and sensor confidence dynamic change, so that the calibration precision is insufficient. Meanwhile, the characteristic relevance of the cross-platform data is not fully utilized, and the fusion reliability of the multi-source ROS Topic data is further reduced. Closed loop optimization is a core technology for improving co-location global consistency and is divided into intra-platform (intra) closed loop and inter-platform (inter) closed loop. The existing intra-closed loop optimization only depends on geometric feature matching, cross-modal feature association is not considered, matching robustness is poor, single loss constraint (such as geometric error loss) is adopted in inter-closed loop optimization, cooperative constraint of motion consistency and feature association is ignored, global pose optimization precision is insufficient, and high-precision positioning requirements of unmanned vehicle-unmanned plane cooperation are difficult to meet. Hybrid characterization techniques of 3D gaussian splats (3 DGS) and neural radiation fields (NeRF), combining the real-time rendering advantages of 3DGS with the implicit scene modeling capabilities of NeRF, have been applied to collaborative mapping scenes. However, the existing hybrid characterization method directly superimposes two characterization results, lacks an effective conversion mechanism of explicit-implicit characterization, is easy to generate geometric conflict and feature redundancy, and does not design a targeted reconciliation strategy aiming at the conflict, so that the integrity and consistency of scene reconstruction are reduced. The scheme is characterized in that image characteristics are enhanced through an improved Real-ESRGAN super-resolution algorithm, ORB characteristic points are extracted, a local map is constructed, a main star adopts a hierarchical visual word bag to detect a map overlapping region, and a global map is generated through optimization fusion by a beam adjustment method. The super-resolution network calculation cost of the method is high, the inter-satellite map matching depends on a fixed visual word bag model, the adaptability to space dynamic environments is insufficient, the capability of autonomously adjusting feature extraction and map fusion strategies according to environment feedback is lacking, and the uncertainty of complex illumination and shielding scenes is difficult to deal with. In view of this, the present application has been made. Disclosure of Invention The cross-platform collaborative hierarchical hybrid map SLAM system and method based on the ROS, disclosed by the application, aim to overcome the defects of cross-platform collaborative multi-mode data processing in the prior art, propose a targeted solution around multi-source data calibration, closed-loop