US-12625268-B2 - Robot relocalization method and apparatus, and storage medium and electronic device

Abstract

A robot relocalization method and apparatus, and a storage medium and an electronic device. The method comprises: performing feature coding on local visual features of image data to obtain image vector features, and searching, in a pre-set key frame database, for the image vector features subjected to dimension reduction processing, so as to obtain a plurality of search results (S 120 ); solving the pose of a robot according to each search result, so as to obtain a first pose initial value, and performing matching on laser point cloud data in a pre-set two-dimensional grid map according to speed data and displacement data, so as to obtain a first matching result (S 130 ); and generating a second matching result according to the first pose initial value and the first matching result, and performing robot relocalization according to a second pose initial value, the first matching result and the second matching result.

Inventors

• Xiujun Yao
• Fuqiang Ma
• Chao Wang
• Chenguang Gui
• Lihua Cui

Assignees

• Jingdong Technology Information Technology Co., Ltd.

Dates

Publication Date

20260512

Application Date

20210608

Priority Date

20200715

Claims (20)

1. 1 . A method for repositioning a robot, comprising: acquiring image data of the robot collected by a camera apparatus, speed data and displacement data of the robot collected by an inertial measuring apparatus and a displacement measuring apparatus, and laser point cloud data of the robot collected by a laser radar; performing a feature encoding on a local visual feature of the image data to obtain an image vector feature, and retrieving the image vector feature after a dimension reduction processing in a preset key frame database to obtain a plurality of retrieval results; solving a post of the robot according to the retrieval results to obtain a first post initial value, and matching the laser point cloud data in a preset two-dimensional grid map according to the speed data and displacement data to obtain a first matching result; and in response to determination that the first post initial value and the first matching result are inconsistent, generating a second matching result according to the first post initial value and the first matching result, and repositioning the robot according to a second pose initial value, the first matching result, and the second matching result.
2. 2 . The method for repositioning a robot according to claim 1 , wherein the performing the feature encoding on the local visual feature of the image data to obtain the image vector feature comprises: extracting the local visual feature of the image data, and searching for a visual word adjacent to the local visual feature in a preset visual dictionary; calculating a residual vector between the local visual feature and the visual word, and performing a cumulative sum operation on the residual vectors belonging to a same visual word to obtain a residual sum vector; and connecting the residual sum vectors according to an index order of the visual words in the preset visual dictionary to obtain the image vector feature.
3. 3 . The method for repositioning a robot according to claim 1 , wherein the retrieving the image vector feature after the dimension reduction processing in the preset key frame database to obtain the plurality of retrieval results, comprises: performing the dimension reduction processing on the image vector feature, and spatially dividing the image vector feature after the dimension reduction processing to obtain a plurality of sub vectors; acquiring a cluster center corresponding to each sub vector and an index of each cluster center in the preset key frame database, and calculating a distance between each sub vector and each cluster center; in response to determination that the distance is less than a preset threshold, acquiring an image vector corresponding to the cluster center according to the index of the cluster center, and obtaining a plurality of the retrieval results according to a plurality of the image vectors.
4. 4 . The method for repositioning a robot according to claim 1 , wherein the solving the post of the robot according to the retrieval results to obtain the first post initial value, comprises: performing a feature matching on each retrieval result to obtain matching features between individual retrieval results and a matching relationship between the matching features; solving the pose of the robot according to the matching features and the matching relationship between the matching features, to obtain the first pose initial value.
5. 5 . The method for repositioning a robot according to claim 1 , wherein the matching the laser point cloud data in the preset two-dimensional grid map according to the speed data and displacement data to obtain the first matching result, comprises: fusing the speed data and the displacement data to obtain a second pose initial value; matching the laser point cloud data in the preset two-dimensional grid map based on the second pose initial value to obtain the first matching result.
6. 6 . The method for repositioning a robot according to claim 1 , wherein the repositioning the robot according to the second pose initial value, the first matching result and the second matching result, comprises: configuring a first weight value, a second weight value and a third weight value respectively for the second pose initial value, the first matching result and the second matching result; performing a weighted summation on the second pose initial value, the first matching result, the second matching result, and the first weight value, the second weight value, and the third weight value respectively; repositioning the robot according to a result of the weighted summation.
7. 7 . The method for repositioning a robot according to claim 1 , wherein the method for repositioning a robot further comprises: determining whether the image data meets a preset condition; if yes, storing the image data in the key frame database as a key frame; wherein the preset condition comprises at least one of the following: a difference between the speed data and speed data at a previous moment corresponding to the speed data is greater than a preset speed threshold; a difference between the displacement data and displacement data at a previous moment corresponding to the displacement data is greater than a preset displacement threshold; an overlapping part between the image data and image data of a previous frame corresponding to the image data is less than a preset overlapping threshold.
8. 8 . An apparatus for repositioning a robot, comprises: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to: acquire image data of the robot collected by a camera apparatus, speed data and displacement data of the robot collected by an inertial measuring apparatus and a displacement measuring apparatus, and laser point cloud data of the robot collected by a laser radar; perform a feature encoding on a local visual feature of the image data to obtain an image vector feature, and retrieve the image vector feature after a dimension reduction processing in a preset key frame database to obtain a plurality of retrieval results; solve a pose of the robot according to the retrieval results to obtain a first pose initial value, and match the laser point cloud data in a preset two-dimensional grid map according to the speed data and displacement data to obtain a first matching result; and in response to determination that the first post initial value and the first matching result are inconsistent, generate a second matching result according to the first pose initial value and the first matching result, and reposition the robot according to a second pose initial value, the first matching result and the second matching result.
9. 9 . A non-transitory computer-readable storage medium on which a computer program is stored, wherein when the computer program is executed by a processor, the processor is configured to: acquire image data of the robot collected by a camera apparatus, speed data and displacement data of the robot collected by an inertial measuring apparatus and a displacement measuring apparatus, and laser point cloud data of the robot collected by a laser radar; perform a feature encoding on a local visual feature of the image data to obtain an image vector feature, and retrieve the image vector feature after a dimension reduction processing in a preset key frame database to obtain a plurality of retrieval results; solve a pose of the robot according to the retrieval results to obtain a first pose initial value, and match the laser point cloud data in a preset two-dimensional grid map according to the speed data and displacement data to obtain a first matching result; and in response to determination that the first post initial value and the first matching result are inconsistent, generate a second matching result according to the first pose initial value and the first matching result, and reposition the robot according to a second pose initial value, the first matching result and the second matching result.
10. 10 . The apparatus for repositioning a robot according to claim 8 , wherein the processor is further configured to: extract the local visual feature of the image data, and searching for a visual word adjacent to the local visual feature in a preset visual dictionary; calculate a residual vector between the local visual feature and the visual word, and performing a cumulative sum operation on the residual vectors belonging to a same visual word to obtain a residual sum vector; and connect the residual sum vectors according to an index order of the visual words in the preset visual dictionary to obtain the image vector feature.
11. 11 . The apparatus for repositioning a robot according to claim 8 , wherein the processor is further configured to: perform the dimension reduction processing on the image vector feature, and spatially dividing the image vector feature after the dimension reduction processing to obtain a plurality of sub vectors; acquire a cluster center corresponding to each sub vector and an index of each cluster center in the preset key frame database, and calculating a distance between each sub vector and each cluster center; in response to determination that the distance is less than a preset threshold, acquire an image vector corresponding to the cluster center according to the index of the cluster center, and obtain a plurality of the retrieval results according to a plurality of the image vectors.
12. 12 . The apparatus for repositioning a robot according to claim 8 , wherein the processor is further configured to: perform a feature matching on each retrieval result to obtain matching features between individual retrieval results and a matching relationship between the matching features; solve the pose of the robot according to the matching features and the matching relationship between the matching features, to obtain the first pose initial value.
13. 13 . The apparatus for repositioning a robot according to claim 8 , wherein the processor is further configured to: fuse the speed data and the displacement data to obtain a second pose initial value; match the laser point cloud data in the preset two-dimensional grid map based on the second pose initial value to obtain the first matching result.
14. 14 . The apparatus for repositioning a robot according to claim 8 , wherein the processor is further configured to: configure a first weight value, a second weight value and a third weight value respectively for the second pose initial value, the first matching result and the second matching result; perform a weighted summation on the second pose initial value, the first matching result, the second matching result, and the first weight value, the second weight value, and the third weight value respectively; reposition the robot according to a result of the weighted summation.
15. 15 . The apparatus for repositioning a robot according to claim 8 , wherein the processor is further configured to: determine whether the image data meets a preset condition; if yes, storing the image data in the key frame database as a key frame; wherein the preset condition comprises at least one of the following: a difference between the speed data and speed data at a previous moment corresponding to the speed data is greater than a preset speed threshold; a difference between the displacement data and displacement data at a previous moment corresponding to the displacement data is greater than a preset displacement threshold; an overlapping part between the image data and image data of a previous frame corresponding to the image data is less than a preset overlapping threshold.
16. 16 . The non-transitory computer-readable storage medium according to claim 9 , wherein the processor is further configured to: extract the local visual feature of the image data, and searching for a visual word adjacent to the local visual feature in a preset visual dictionary; calculate a residual vector between the local visual feature and the visual word, and performing a cumulative sum operation on the residual vectors belonging to a same visual word to obtain a residual sum vector; and connect the residual sum vectors according to an index order of the visual words in the preset visual dictionary to obtain the image vector feature.
17. 17 . The non-transitory computer-readable storage medium according to claim 9 , wherein the processor is further configured to: perform the dimension reduction processing on the image vector feature, and spatially dividing the image vector feature after the dimension reduction processing to obtain a plurality of sub vectors; acquire a cluster center corresponding to each sub vector and an index of each cluster center in the preset key frame database, and calculating a distance between each sub vector and each cluster center; in response to determination that the distance is less than a preset threshold, acquire an image vector corresponding to the cluster center according to the index of the cluster center, and obtain a plurality of the retrieval results according to a plurality of the image vectors.
18. 18 . The non-transitory computer-readable storage medium according to claim 9 , wherein the processor is further configured to: perform a feature matching on each retrieval result to obtain matching features between individual retrieval results and a matching relationship between the matching features; solve the pose of the robot according to the matching features and the matching relationship between the matching features, to obtain the first pose initial value.
19. 19 . The non-transitory computer-readable storage medium according to claim 9 , wherein the processor is further configured to: fuse the speed data and the displacement data to obtain a second pose initial value; match the laser point cloud data in the preset two-dimensional grid map based on the second pose initial value to obtain the first matching result.
20. 20 . The non-transitory computer-readable storage medium according to claim 9 , wherein the processor is further configured to: configure a first weight value, a second weight value and a third weight value respectively for the second pose initial value, the first matching result and the second matching result; perform a weighted summation on the second pose initial value, the first matching result, the second matching result, and the first weight value, the second weight value, and the third weight value respectively; reposition the robot according to a result of the weighted summation.

Description

CROSS REFERENCE The present application is an US national phase of International Application No. PCT/CN2021/098981, filed Jun. 8, 2021, which claims priority to Chinese Patent Application No. 202010680255.2, filed Jul. 15, 2020, titled “ROBOT RELOCALIZATION METHOD AND APPARATUS, AND STORAGE MEDIUM AND ELECTRONIC DEVICE”. TECHNICAL FIELD The embodiments of the present disclosure relate to the field of robot technology, in particular, to a method for repositioning a robot, an apparatus for repositioning a robot, a computer-readable storage medium, and an electronic device. BACKGROUND With the continuous development of artificial intelligence, the application scenarios of robots have been greatly expanded, bringing great convenience to people's production and life. At the same time, the requirements for the efficiency and intelligence of robots are also increasingly high. For example, for autonomous walking robots, it is required that robots have more efficient walking path planning for walking routes. At present, the positioning method based on vision and laser simultaneous localization and mapping (SLAM) can use environmental information for active positioning, without the need to modify the environment. SUMMARY According to an aspect of the present disclosure, a method for repositioning a robot is provided, including: acquiring image data of the robot collected by a camera apparatus, speed data and displacement data of the robot collected by an inertial measuring apparatus and a displacement measuring apparatus, and laser point cloud data of the robot collected by a laser radar;performing a feature encoding on a local visual feature of the image data to obtain an image vector feature, and retrieving the image vector feature after a dimension reduction processing in a preset key frame database to obtain a plurality of retrieval results;solving a post of the robot according to the retrieval results to obtain a first post initial value, and matching the laser point cloud data in a preset two-dimensional grid map according to the speed data and displacement data to obtain a first matching result; andin response to determination that the first post initial value and the first matching result are inconsistent, generating a second matching result according to the first post initial value and the first matching result, and repositioning the robot according to a second pose initial value, the first matching result, and the second matching result. In an exemplary embodiment of the present disclosure, the performing the feature encoding on the local visual feature of the image data to obtain the image vector feature includes: extracting the local visual feature of the image data, and searching for a visual word adjacent to the local visual feature in a preset visual dictionary;calculating a residual vector between the local visual feature and the visual word, and performing a cumulative sum operation on the residual vectors belonging to a same visual word to obtain a residual sum vector; andconnecting the residual sum vectors according to an index order of the visual words in the preset visual dictionary to obtain the image vector feature. In an exemplary embodiment of the present disclosure, the retrieving the image vector feature after the dimension reduction processing in the preset key frame database to obtain the plurality of retrieval results, includes: performing the dimension reduction processing on the image vector feature, and spatially dividing the image vector feature after the dimension reduction processing to obtain a plurality of sub vectors;acquiring a cluster center corresponding to each sub vector and an index of each cluster center in the preset key frame database, and calculating a distance between each sub vector and each cluster center;in response to determination that the distance is less than a preset threshold, acquiring an image vector corresponding to the cluster center according to the index of the cluster center, and obtaining a plurality of the retrieval results according to a plurality of the image vectors. In an exemplary embodiment of the present disclosure, the solving the post of the robot according to the retrieval results to obtain the first post initial value, includes: performing a feature matching on each retrieval result to obtain matching features between individual retrieval results and a matching relationship between the matching features;solving the pose of the robot according to the matching features and the matching relationship between the matching features, to obtain the first pose initial value. In an exemplary embodiment of the present disclosure, the matching the laser point cloud data in the preset two-dimensional grid map according to the speed data and displacement data to obtain the first matching result, includes: fusing the speed data and the displacement data to obtain a second pose initial value;matching the laser point cloud data in the preset two-dimensiona