KR-102965313-B1 - METHOD AND APPARATUS FOR ESTIMATING POSE, COMPUTER-READABLE STORAGE MEDIUM AND COMPUTER PROGRAM FOR CONTROLLING THE HOLDER DEVICE

KR102965313B1KR 102965313 B1KR102965313 B1KR 102965313B1KR-102965313-B1

Abstract

A pose estimation method according to an embodiment may include the steps of: collecting a first image from a camera; collecting a plurality of second images from a DB using location information of the terminal; estimating a plurality of first poses by comparing the first image with the plurality of second images; estimating a plurality of second poses by continuously tracking the pose of the camera based on the plurality of first poses; and estimating a third pose based on the remaining second poses after removing the second poses that do not satisfy a preset standard among the plurality of second poses. The embodiment has the effect of estimating a more accurate camera pose by estimating the final pose using a plurality of first poses.

Inventors

임승욱
유연걸
윤찬민
정준영

Assignees

에스케이텔레콤 주식회사

Dates

Publication Date: 20260513
Application Date: 20191029

Claims (10)

A method performed in a pose estimation device for estimating the pose of a camera equipped in a terminal, A step of collecting a first image and location information of the terminal from the camera; A step of collecting a plurality of second images from a DB using the location information of the above terminal; A step of estimating a plurality of first poses by comparing the first image and the plurality of second images; A step of continuously tracking the pose of the camera based on the plurality of first poses to estimate a plurality of second poses for each time interval; and The method includes the step of removing a second pose that does not satisfy a preset standard among the plurality of second poses and estimating a third pose based on the remaining second poses; The above steps are performed sequentially at a first time point and a second time point later than the first time point, respectively. A pose estimation method characterized by the fact that the distance between the second pose estimated from the first pose estimated at the first time point and the second pose estimated from the first pose estimated at the second time point, which is estimated as the position of the camera at the same time point, is less than or equal to a predetermined distance.
In paragraph 1, In the step of estimating the third pose above, A pose estimation method that estimates the second pose as the third pose if the second pose exists within a certain range.
In paragraph 2, In the step of estimating the third pose above, A pose estimation method that removes the second pose when the second pose is outside a certain range.
In paragraph 1, The step of estimating the third pose above is, A step of continuously tracking the second pose based on the first point in time; A step of continuously tracking the second pose based on the second viewpoint; and A step of removing the second pose using the distance between the second pose measured based on the first time point at the same time point and the second pose measured based on the second time point. A pose estimation method including
In paragraph 4, In the step of estimating the third pose above, A pose estimation method for removing the second pose using the Mahalanobis Distance for the distance between the second pose measured based on the first time point and the second pose measured based on the second time point.
In paragraph 5, In the step of estimating the third pose above, A pose estimation method for removing the second poses in which the above Mahalanobis distance is greater than or equal to a first threshold.
In paragraph 1, In the step of collecting the plurality of second images above, A pose estimation method in which the location information of the terminal is measured from the GPS information of the terminal or the information of the first image.
A camera image collection unit that collects a first image and location information of the terminal from the camera of the terminal; A DB image collection unit that collects a plurality of second images from a DB using GPS information of the above terminal; A first pose estimation unit that estimates a plurality of first poses by comparing the first image and the plurality of second images; A second pose estimation unit that continuously tracks the pose of the camera based on the plurality of first poses and estimates a plurality of second poses for each time interval; and A third pose estimation unit that removes a second pose among the plurality of second poses that does not satisfy a preset standard and estimates a third pose based on the remaining second poses; wherein Each of the above-mentioned camera image acquisition unit, DB image acquisition unit, first pose estimation unit, second pose estimation unit, and third pose estimation unit operates sequentially at a first time point and at a second time point later than the first time point, respectively. A pose estimation device characterized by the fact that the distance between the second pose estimated from the first pose estimated at the first time point and the second pose estimated from the first pose estimated at the second time point, which is estimated as the position of the camera at the same time point, is less than or equal to a predetermined distance.
As a computer-readable recording medium storing a computer program, The above computer program is, A computer-readable recording medium comprising instructions for a processor to perform a method according to any one of claims 1 through 7.
As a computer program stored on a computer-readable recording medium, The above computer program is, A computer program comprising instructions for a processor to perform a method according to any one of paragraphs 1 through 7.

Description

Method and apparatus for estimating pose, computer-readable storage medium, and computer program The embodiment relates to a pose estimation method for effectively estimating the pose of a camera equipped in a terminal. Augmented Reality (AR) is a technology that uses location and orientation information to determine an approximate location, identifies the services desired by the user through a comparison between facility information—such as surrounding buildings—and real-world video information input based on camera movement, and provides relevant information. More specifically, augmented reality is a field of virtual reality (VR) that uses computer graphics techniques to superimpose virtual objects onto a real environment, making them appear as if they exist within the original environment. Unlike conventional virtual reality, which focuses solely on virtual spaces and objects, augmented reality is a technology that superimposes virtual objects onto a foundation of the real world to provide supplementary information that is difficult to obtain from the real world alone. With the commercialization of 5G communication, such augmented reality technology is gaining prominence in the field of mobile AR technology used in communication terminals, and marker-based mobile AR technology or sensor-based mobile AR technology is generally used in current mobile AR technology applications. Marker-based mobile AR technology is a technology that recognizes a real object by recognizing a marker corresponding to the real object when photographing a real object to be augmented using a virtual object, and sensor-based mobile AR technology is a technology that infers the current location and the direction the terminal is looking by using GPS and a digital compass installed in the terminal, and overlays POI (Point of Interests) information corresponding to the image in the inferred direction. However, marker-based mobile AR technology has the problem that it cannot augment virtual objects without markers, and sensor-based mobile AR technology has the problem of being unable to accurately augment virtual objects onto specific objects due to errors in the current location and orientation of the detected device. FIG. 1 is a block diagram showing an augmented reality system according to an embodiment. FIG. 2 is a block diagram showing a pose estimation device according to an embodiment. FIG. 3 is a block diagram illustrating a pose estimation method according to an embodiment. Figure 4 is a diagram illustrating the process of collecting DB images according to an embodiment. FIG. 5 is a diagram illustrating the step of estimating a third pose according to an embodiment. FIG. 6 is a block diagram illustrating a method for estimating a third pose according to an embodiment. Figures 7 and 8 are drawings for explaining a method for estimating the third pose of Figure 6. Hereinafter, embodiments will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing an augmented reality system according to an embodiment, and FIG. 2 is a block diagram showing a pose estimation device according to an embodiment. Referring to FIG. 1, an augmented reality system (1000) according to an embodiment may include a pose estimation device (100). The pose estimation device (100) can perform the task of estimating the pose of the camera (240). The pose estimation device (100) includes a processor, and the processor can control the overall operation of the pose estimation device (100). The pose estimation device (100) may include a transceiver (not shown). The pose estimation device (100) may receive an image from a map generation server (300) using the transceiver and transmit pose information to a terminal (200). The pose may include information about the position of the camera (240) and information about the direction. The pose estimation device (100) can receive an initialization value, map information, GPS information of the terminal (200), a 2D image captured by the terminal (200), and/or information about the camera (240) included in the terminal (200) from the terminal (200) using a transceiver. The pose estimation device (100) may include a memory (not shown). The memory may store information required for the execution of a program and received images. The augmented reality system according to the embodiment may include a terminal (200). The terminal (200) may include a processor (210), a transceiver (220), a memory (230), a camera (240) and a sensor unit (250), and an AR program (260). The processor (210) can control the overall operation of the terminal (200). The processor (210) can collect images captured using the camera (240) and save them as 2D images. Additionally, the processor (210) can further transmit initialization values, map information of the area to be augmented with virtual objects, GPS information and/or information about the camera (240) to the pose estimation device (100) using the transceiver (220