Search

CN-121999033-A - Robot multi-camera positioning key frame selection method, device and storage medium

CN121999033ACN 121999033 ACN121999033 ACN 121999033ACN-121999033-A

Abstract

The application provides a method, a device and a storage medium for selecting a positioning key frame of a robot multi-camera, which are applied to the field of automatic control and comprise the steps of selecting a real-time frame of a first camera, obtaining a plurality of first key frames and generating a first key frame sequence; the method comprises the steps of carrying out real-time frame selection on a second camera to obtain a plurality of second key frames, generating a second key frame sequence, carrying out optimization pose calculation on the basis of the first key frame sequence and the second key frame sequence to obtain a plurality of first poses corresponding to the first camera and a plurality of second poses corresponding to the second camera.

Inventors

  • CUI ZHIPENG

Assignees

  • 浙江白马科技有限公司

Dates

Publication Date
20260508
Application Date
20241103

Claims (13)

  1. 1. A method for selecting a multi-camera positioning key frame of a robot, the robot comprising a first camera and at least one second camera, the method comprising: performing key frame selection on the real-time frames of the first camera to obtain a plurality of first key frames, and generating a first key frame sequence; Performing key frame selection on the real-time frames of the second camera to obtain a plurality of second key frames, and generating a second key frame sequence; and performing optimization pose calculation based on the first key frame sequence and the second key frame sequence to obtain a plurality of first poses corresponding to the first camera and a plurality of second poses corresponding to the second camera, wherein the plurality of first poses and the plurality of second poses are used for optimizing the robot pose of the robot in a specific time period.
  2. 2. The method of claim 1, wherein the step of determining the position of the substrate comprises, The second key frame sequence includes a second camera real-time frame at the same time instant as the plurality of first key frames.
  3. 3. The method of claim 1, wherein the step of determining the position of the substrate comprises, The key frame selection is performed on the real-time frames of the first camera to obtain a plurality of first key frames, and the generation of a first key frame sequence comprises the following steps: Performing feature matching screening on the real-time frames of the first camera to obtain a plurality of first key frames, and generating a first key frame sequence; The key frame selection is performed on the real-time frames of the second camera to obtain a plurality of second key frames, and the generation of a second key frame sequence comprises the following steps: and selecting the real-time frame of the second camera at the same moment as the first key frames as the second key frames.
  4. 4. The method of claim 1, wherein the step of determining the position of the substrate comprises, The key frame selection is performed on the real-time frames of the first camera to obtain a plurality of first key frames, and the generation of a first key frame sequence comprises the following steps: Performing feature matching screening on the real-time frames of the first camera to obtain a plurality of first key frames, and generating a first key frame sequence; The key frame selection is performed on the real-time frames of the second camera to obtain a plurality of second key frames, and the generation of a second key frame sequence comprises the following steps: And performing feature matching screening on the real-time frames of the second camera to obtain the plurality of second key frames, and generating the second key frame sequence.
  5. 5. The method of claim 4, wherein the step of determining the position of the first electrode is performed, The key frame selection is performed on the real-time frames of the second camera to obtain a plurality of second key frames, and the generation of a second key frame sequence comprises the following steps: And selecting the real-time frame of the second camera with the same time as the first key frames as the second key frame and adding the second key frame sequence.
  6. 6. The method of claim 1, wherein the performing key frame selection on the real-time frames of the first camera to obtain a plurality of first key frames, and the performing key frame selection on the real-time frames of the second camera to obtain a plurality of second key frames, comprises: performing feature matching screening on the real-time frames of the first camera to obtain a plurality of first candidate key frames; Performing feature matching screening on the real-time frames of the second camera to obtain a plurality of second candidate key frames; and screening the first candidate key frame and the second candidate key frame at the same moment to serve as the first key frame and the second key frame respectively.
  7. 7. The method according to claim 3-6, wherein, Before performing key frame selection on the real-time frames of the first camera to obtain a plurality of first key frames and generating a first key frame sequence, the method further comprises: selecting the real-time frame of the first camera as a first candidate real-time frame according to a specific inter-frame interval; the key frame selection is performed on the real-time frames of the first camera to obtain a plurality of first key frames, and the generation of a first key frame sequence comprises the following steps: and performing key frame selection on the real-time frames of the first camera from the first candidate real-time frames to obtain a plurality of first key frames.
  8. 8. The method of claims 3-6, wherein prior to keyframe selection of the real-time frames of the second camera to obtain a plurality of second keyframes, generating a second sequence of keyframes, the method further comprises: selecting the real-time frame of the second camera as a second candidate real-time frame according to the specific inter-frame interval; The key frame selection is performed on the real-time frames of the second camera to obtain a plurality of second key frames, and the generation of the first key frame sequence comprises the following steps: And selecting the key frames of the real-time frames of the second camera from the second candidate real-time frames to obtain a plurality of second key frames.
  9. 9. The method of any of claims 3-6, wherein the feature matching screening comprises: performing image feature matching on the real-time frame of one first camera and at least one other first key frame to obtain the number of matching feature points, or performing feature matching on the real-time frame of one second camera and at least one other second key frame to obtain the number of matching feature points; and under the condition that the number of the matching characteristic points is larger than or equal to a matching threshold value, taking the corresponding real-time frame of the first camera as the first key frame or taking the corresponding real-time frame of the second camera as the second key frame.
  10. 10. The method of claim 1, wherein the optimizing pose calculation based on the first key frame sequence and the second key frame sequence comprises: and performing beam adjustment optimization based on the first key frame and the second key frame in the first key frame sequence and the second key frame sequence to obtain a first pose corresponding to the first camera and a second pose corresponding to the second camera.
  11. 11. A multi-camera positioning key frame selection device for a robot, wherein the control device performs the multi-camera positioning key frame selection method for a robot according to any one of claims 1 to 10, the device comprising: the first selecting module is used for selecting key frames of the real-time frames of the first camera to obtain a plurality of first key frames and generating a first key frame sequence; the second selecting module is used for selecting the key frames of the real-time frames of the second camera to obtain a plurality of second key frames and generating a second key frame sequence; the pose calculation module is used for carrying out optimization pose calculation based on the first key frame sequence and the second key frame sequence to obtain a plurality of first poses corresponding to the first camera and a plurality of second poses corresponding to the second camera, wherein the plurality of first poses and the plurality of second poses are used for optimizing the robot pose of the robot in a specific time period.
  12. 12. A computer readable storage medium having stored therein at least one instruction that is loaded and executed by a processor to implement the robotic multi-camera positioning keyframe selection method of any one of claims 1-10.
  13. 13. A robotic multi-camera positioning keyframe selection device, comprising: The readable storage medium has stored therein at least one instruction that is loaded and executed by a processor to implement the robotic multi-camera positioning keyframe selection method of any one of claims 1-10.

Description

Robot multi-camera positioning key frame selection method, device and storage medium Technical Field The present application relates to the field of robot positioning, and in particular, to a method and apparatus for selecting a multi-camera positioning key frame of a robot, and a storage medium. Background With the development of science and technology, robots are increasingly applied to human production and life, are often used for carrying, inspecting, cleaning and the like of articles instead of manpower, and are generally provided with sensors, a navigation and positioning system and a control algorithm, so that the robots can sense the surrounding environment and perform corresponding actions. At present, various robots exist in the market to assist people to finish work, such as a sweeper, a mower, a dust collector and the like, and convenience is provided for the production and the living of people. One of the keys for realizing the intellectualization and automation of the robot is a robot positioning technology, and the vision positioning system is widely used in robot positioning due to the factors of wide application scene, low cost and the like. The prior art visual positioning system is mainly composed of a single set of visual sensors, such as a single monocular camera, a single binocular camera or a single depth camera. However, since the vision positioning system mainly relies on the vision sensor to perform environmental feature recognition so as to realize pose calculation of the vision positioning system, when a single set of vision sensor of the robot is shielded by an obstacle or no obvious feature exists in the range of the field of view, the pose of the robot can not be obtained accurately, so that the vision positioning system of the robot can not work effectively, the whole operation of the robot is affected, and under the condition that the robot is provided with a plurality of sets of vision sensors, how to reasonably select real-time frame images obtained by the vision sensors as key frames so as to utilize the key frames to perform optimal calculation on the pose of the robot is a problem to be solved in the present stage. Disclosure of Invention In view of the above, the embodiments of the present application provide a method, an apparatus, and a storage medium for selecting a multi-camera positioning key frame of a robot, which can reasonably select key frames acquired by multiple cameras when the robot performs multi-camera positioning, so as to perform optimization calculation on the pose of the robot by using the key frames, and improve the accuracy of robot positioning. The technical scheme is as follows: In a first aspect, an embodiment of the present application provides a method for selecting a multi-camera positioning key frame of a robot, where the method includes performing key frame selection on a real-time frame of a first camera to obtain a plurality of first key frames, and generating a first key frame sequence; Performing key frame selection on the real-time frames of the second camera to obtain a plurality of second key frames, and generating a second key frame sequence; and performing optimization pose calculation based on the first key frame sequence and the second key frame sequence to obtain a plurality of first poses corresponding to the first camera and a plurality of second poses corresponding to the second camera, wherein the plurality of first poses and the plurality of second poses are used for optimizing the robot pose of the robot in a specific time period. Further, the second key frame sequence includes a second camera real-time frame at the same time instant as the plurality of first key frames. Further, the performing key frame selection on the real-time frames of the first camera to obtain a plurality of first key frames, and generating a first key frame sequence includes: Performing feature matching screening on the real-time frames of the first camera to obtain a plurality of first key frames, and generating a first key frame sequence; The key frame selection is performed on the real-time frames of the second camera to obtain a plurality of second key frames, and the generation of a second key frame sequence comprises the following steps: and selecting the real-time frame of the second camera at the same moment as the first key frames as the second key frames. Further, the performing key frame selection on the real-time frames of the first camera to obtain a plurality of first key frames, and generating a first key frame sequence includes: Performing feature matching screening on the real-time frames of the first camera to obtain a plurality of first key frames, and generating a first key frame sequence; The key frame selection is performed on the real-time frames of the second camera to obtain a plurality of second key frames, and the generation of a second key frame sequence comprises the following steps: And performing feature matching scr