CN-121982240-A - Positioning and mapping method, system and storage medium in bank dynamic illumination environment

CN121982240ACN 121982240 ACN121982240 ACN 121982240ACN-121982240-A

Abstract

The invention discloses a positioning and mapping method, a positioning and mapping system and a storage medium in a bank dynamic illumination environment, and relates to the technical field of robot mapping, wherein the method comprises the steps of acquiring RGB images, depth images and inertial measurement data in the bank environment; performing illumination equalization processing on an RGB image, separating an illumination component and a reflection component through a multi-scale algorithm to obtain a stable reflection image, performing defect restoration on a depth image, extracting feature points based on the reflection image and the restored depth image, acquiring three-dimensional coordinates of the feature points, performing pose estimation by combining inertial measurement data, performing loop detection by combining geometric matching with auxiliary constraint, screening effective loops, fusing a pose estimation result and the effective loops to perform pose image optimization to obtain a corrected pose, and constructing a three-dimensional environment map based on the corrected pose and the restored depth image. The method is suitable for a special bank environment, effectively improves positioning robustness and mapping accuracy, and can provide reliable support for subsequent robot inspection, space planning and other applications.

Inventors

HUANG WANZHONG
YANG ZONGFENG
WANG XIANSHI
HUANG CHAOWEI
LUO MINGYANG
ZHOU ZIHONG

Assignees

昆仑之数(成都)科技有限公司

Dates

Publication Date: 20260505
Application Date: 20260128

Claims (10)

1. The positioning and mapping method in the bank dynamic illumination environment is characterized by comprising the following steps: Step 1, acquiring image data and inertial measurement data in a banking environment, wherein the image data comprises RGB images and depth images; Step 2, performing illumination equalization processing on the RGB image, and separating an illumination component and a reflection component of the image through a multi-scale algorithm to obtain a reflection image after illumination equalization; extracting feature points based on the reflection image and the repaired depth image, acquiring three-dimensional coordinates of the feature points through the repaired depth image, and performing pose estimation by combining inertial measurement data to obtain a pose estimation result; step 4, loop detection is carried out through geometric matching and auxiliary constraint, and effective loop is obtained through screening; Step5, performing pose graph optimization by combining the pose estimation result and the effective loop to obtain a corrected pose; And 6, constructing and obtaining a three-dimensional environment map based on the corrected pose and the corrected depth image.
2. The method for locating and mapping in a dynamic illumination environment of a bank according to claim 1, wherein the performing defect repair processing on the depth image to obtain a repaired depth image comprises the following steps: Performing semantic segmentation on the RGB image to obtain a first semantic segmentation result, and marking a high-reflection area based on the first semantic segmentation result; acquiring a depth value in the region in the current depth image, and judging the region as an invalid region of the current depth image if the depth value is zero or exceeds the effective measurement range of the camera; acquiring the current depth image, continuously acquiring the depth images of a plurality of frames of bank environments, screening depth information corresponding to areas with depth values which are neither zero nor exceed the effective measurement range of the camera in the depth images of the plurality of frames of bank environments, and forming a plurality of frames of historical effective depth information; Projecting multi-frame historical effective depth information to a coordinate system of a current depth image, and filling an ineffective area of the current depth image in a weighted average mode to obtain a repaired depth image.
3. The method for locating and mapping under the dynamic illumination environment of the bank according to claim 2, wherein the method is characterized in that the multi-frame historical effective depth information is projected to the coordinate system of the current depth image, and the invalid area of the current depth image is filled in a weighted average mode, and specifically comprises the following steps: the pose similarity of each historical frame and the current frame is calculated by weighting and calculating the rotation angle deviation and the translation distance deviation of the current frame and the historical frame; If the pose similarity of the historical frame and the current frame is higher than a preset similarity threshold, judging that the historical frame is the same-view frame, and distributing a first fusion weight to the corresponding historical effective depth information; If the pose similarity of the historical frame and the current frame is lower than a preset similarity threshold, judging that the historical frame is an out-of-view frame, and distributing second fusion weight to the corresponding historical effective depth information; And carrying out weighted average calculation on the multi-frame historical effective depth information based on the distributed first fusion weight and second fusion weight, and completing filling of the current ineffective area.
4. The method for locating and mapping under dynamic illumination environment of bank according to claim 1, wherein the steps of extracting feature points based on the reflected image and the repaired depth image, obtaining three-dimensional coordinates of the feature points through the repaired depth image, and performing pose estimation in combination with inertial measurement data to obtain a pose estimation result comprise the following steps: And extracting feature points from the reflected image by adopting a feature extraction algorithm, distributing three-dimensional coordinates for the feature points by using the repaired depth image, distributing semantic trust weights for the feature points based on semantic segmentation results, taking the semantic trust weights as weighting factors of reprojection errors, and executing weighted nonlinear optimization by combining inertial measurement data and the three-dimensional coordinates to obtain a relative pose of the camera as a pose estimation result.
5. The method for locating and mapping in a dynamic illumination environment of a bank according to claim 4, wherein the assigning semantic trust weights to feature points based on semantic segmentation results comprises the following steps: determining semantic region types of the feature points according to semantic segmentation results, wherein the semantic region types comprise a high reflection region, a weak texture region, a semantic significant object region and a strong texture region; If the feature points are located in the high reflection area or the weak texture area, semantic trust weights of a first preset weight range are distributed to the feature points; If the feature points are located in the semantically significant object area or the strong texture area, semantic trust weights of a second preset weight range are distributed to the feature points, and the values of the first preset weight range are smaller than those of the second preset weight range.
6. The method for locating and mapping in a dynamic illumination environment of a bank according to claim 1, wherein the loop detection is performed by combining geometric matching with auxiliary constraint, and the effective loop is obtained by screening, which comprises the following steps: Determining a current frame corresponding to current image data, and matching the current frame with a historical frame set to obtain a plurality of potential loop candidate frames and geometric matching scores corresponding to the potential loop candidate frames; Executing semantic segmentation on the RGB image of the current frame to obtain a second semantic segmentation result of the current frame; respectively executing semantic segmentation on the RGB images of each potential loop candidate frame to obtain a third semantic segmentation result of each potential loop candidate frame; Calculating the semantic tag matching degree of each potential loop candidate frame based on the second semantic segmentation result and the third semantic segmentation result; Weighting and fusing the geometric matching score and the semantic tag matching degree according to preset balance weights to obtain loop scores corresponding to all potential loop candidate frames; And comparing the loop scores with a preset threshold, and if the loop scores exceed the preset threshold, judging the corresponding potential loop candidate frame as a valid loop.
7. The method for locating and mapping in a dynamic illumination environment of a bank according to claim 1, wherein the step of combining the pose estimation result with the effective loop to perform pose map optimization, and the step of obtaining the corrected pose comprises the following steps: combining the odometer constraint corresponding to the pose estimation result, the loop constraint corresponding to the effective loop and the IMU constraint corresponding to the inertial measurement data to construct a total error function; And solving to obtain a globally consistent camera pose sequence by minimizing the total error function, and taking the camera pose sequence as a corrected pose.
8. The method for locating and mapping under the dynamic illumination environment of the bank according to claim 1, wherein the construction of the three-dimensional environment map based on the corrected pose and the corrected depth image comprises the following steps: And (3) taking the corrected pose as a space reference, combining the repaired depth image, and carrying out fusion processing by adopting a preset three-dimensional map building algorithm to construct and obtain a three-dimensional environment map.
9. The positioning and mapping system in a dynamic illumination environment of a bank is characterized in that the positioning and mapping system in the dynamic illumination environment of the bank is applied to the positioning and mapping method in the dynamic illumination environment of the bank as claimed in any one of claims 1 to 8, and comprises the following steps: The system comprises a multi-source data acquisition module, a data acquisition module and a data processing module, wherein the multi-source data acquisition module is used for acquiring image data and inertial measurement data in a banking environment, wherein the image data comprises RGB images and depth images; The illumination self-adaptive preprocessing module is used for performing illumination equalization processing on the RGB image, and separating an illumination component and a reflection component of the image through a multi-scale algorithm to obtain a reflection image after illumination equalization; The semantic depth pose module is used for performing defect repair processing on the depth image to obtain a repaired depth image, extracting characteristic points based on the reflection image and the repaired depth image, acquiring three-dimensional coordinates of the characteristic points through the repaired depth image, and performing pose estimation by combining inertial measurement data to obtain a pose estimation result; The constraint weighted loop detection module is used for performing loop detection by combining geometric matching with auxiliary constraint, and screening to obtain an effective loop; The pose image optimization module is used for executing pose image optimization by combining a pose estimation result and an effective loop to obtain a corrected pose; And the map construction module is used for constructing and obtaining a three-dimensional environment map based on the corrected pose and the corrected depth image.
10. A readable storage medium having stored thereon computer instructions which, when executed by a processor, implement a method of location mapping in a bank dynamic lighting environment as claimed in any one of claims 1 to 8.

Description

Positioning and mapping method, system and storage medium in bank dynamic illumination environment Technical Field The invention relates to the technical field of robot mapping, in particular to a positioning mapping method, a positioning mapping system and a storage medium in a bank dynamic illumination environment. Background Along with the expansion of the application of intelligent technology in the financial field, the demands of the bank hall on autonomous positioning and three-dimensional mapping technology are increasingly urgent, and the technology can provide basic support for subsequent robot inspection, space planning and other applications. The existing RGB-DSLAM (synchronous positioning and mapping) algorithm has certain positioning and mapping capability in a static and illumination-stable general indoor environment. However, as a typical commercial indoor scene, a bank hall has the common environmental characteristics that dynamic illumination and a high-reflection surface coexist, on one hand, facilities such as a large-area landing window, a high-brightness spotlight and the like can cause frequent change of the intensity of the ambient illumination, local overexposure, underexposure or shadow mutation occur to influence the characteristic stability of an RGB image, and on the other hand, high-reflection materials such as a glass counter, polished marble ground and the like can cause deviation of depth signals received by an RGB-D camera to cause holes or noise of the depth image. The environmental factors directly cause the quality degradation of the image data acquired by the existing RGB-DSLAM algorithm, so that the problem that the three-dimensional map is distorted or the details are missing is caused, and the high-precision requirement of a bank hall scene on positioning and mapping is difficult to meet. Disclosure of Invention The invention provides a positioning and mapping method, a positioning and mapping system and a storage medium under a dynamic illumination environment of a bank, aiming at solving the technical problem that the accuracy of a constructed three-dimensional map is reduced due to the dynamic illumination environment of the bank in the prior art. The technical scheme adopted by the invention is as follows: The first aspect of the application provides a positioning and mapping method in a bank dynamic illumination environment, which comprises the following steps: Step 1, acquiring image data and inertial measurement data in a banking environment, wherein the image data comprises RGB images and depth images; Step 2, performing illumination equalization processing on the RGB image, and separating an illumination component and a reflection component of the image through a multi-scale algorithm to obtain a reflection image after illumination equalization; extracting feature points based on the reflection image and the repaired depth image, acquiring three-dimensional coordinates of the feature points through the repaired depth image, and performing pose estimation by combining inertial measurement data to obtain a pose estimation result; step 4, loop detection is carried out through geometric matching and auxiliary constraint, and effective loop is obtained through screening; Step5, performing pose graph optimization by combining the pose estimation result and the effective loop to obtain a corrected pose; And 6, constructing and obtaining a three-dimensional environment map based on the corrected pose and the corrected depth image. Preferably, the performing defect repair processing on the depth image, and obtaining the repaired depth image includes the following contents: Performing semantic segmentation on the RGB image to obtain a first semantic segmentation result, and marking a high-reflection area based on the first semantic segmentation result; acquiring a depth value in the region in the current depth image, and judging the region as an invalid region of the current depth image if the depth value is zero or exceeds the effective measurement range of the camera; acquiring the current depth image, continuously acquiring the depth images of a plurality of frames of bank environments, screening depth information corresponding to areas with depth values which are neither zero nor exceed the effective measurement range of the camera in the depth images of the plurality of frames of bank environments, and forming a plurality of frames of historical effective depth information; Projecting multi-frame historical effective depth information to a coordinate system of a current depth image, and filling an ineffective area of the current depth image in a weighted average mode to obtain a repaired depth image. Preferably, the feature points are extracted based on the reflected image and the repaired depth image, and three-dimensional coordinates of the feature points are obtained through the repaired depth image, and pose estimation is performed in combination with inertial