Search

KR-102965319-B1 - METHOD FOR ESTIMATING POSE, COMPUTER-READABLE STORAGE MEDIUM AND COMPUTER PROGRAM FOR CONTROLLING THE HOLDER DEVICE

KR102965319B1KR 102965319 B1KR102965319 B1KR 102965319B1KR-102965319-B1

Abstract

The pose estimation method of the embodiment may include the steps of: generating a plurality of images including a first image of a first reference size and a second image of a second reference size larger than the size of the first image; extracting feature points of the first image; estimating the pose of the first image by matching feature points of the first image with feature points of a plurality of DB images; setting a region of interest (ROI) using the pose of the first image; extracting feature points of the second image within the region of interest; estimating the pose of the second image by matching feature points of the second image with feature points of the plurality of DB images; and determining one pose among the estimated plurality of poses according to a preset criterion. In this implementation, by extracting second feature points only within the ROI area, unnecessary feature points are not extracted, which has the effect of improving the speed of the operation.

Inventors

  • 임승욱
  • 유연걸
  • 윤찬민
  • 정준영

Assignees

  • 에스케이텔레콤 주식회사

Dates

Publication Date
20260513
Application Date
20191029

Claims (10)

  1. A method performed in a pose estimation device for estimating the pose of a camera equipped in a terminal, A step of generating a plurality of images including a first image of a first reference size and a second image of a second reference size larger than the size of the first image, based on an input image collected from the camera; A step of extracting feature points of the first image; A step of estimating the pose of the first image by matching feature points of the first image with feature points of a plurality of DB images; A step of setting a region of interest (ROI) in the second image using the pose of the first image; A step of extracting feature points of the second image within the region of interest; A step of estimating the pose of the second image by matching the feature points of the second image with the feature points of the plurality of DB images; and A step of determining one pose as the camera pose among the estimated plurality of poses of the second images according to a preset criterion. A pose estimation method including
  2. In paragraph 1, The step of setting the above region of interest is, A pose estimation method for setting the region of interest (ROI) by projecting the 3D coordinate values of the DB image onto the second image.
  3. In paragraph 2, A pose estimation method for determining the size of the region of interest by calculating the mean and covariance matrix using the 3D coordinate values and using the covariance matrix.
  4. In paragraph 3, The above covariance matrix is a pose estimation method calculated using 3D coordinates, roll, pitch, and yaw values.
  5. In paragraph 1, The step of generating the first image and the second image is A pose estimation method in which, if the size of the input image is larger than the second reference size, the size of the input image is changed to a size between the first image and the second image, and if the size of the input image is smaller than the second reference size, the input image is used as the second image.
  6. In paragraph 1, The step of estimating the pose of the first image above is, A step of matching feature points of the first image with feature points of the plurality of DB images; A step of removing unnecessary feature points among the above-mentioned matched plurality of feature points; and Step of estimating the pose of the first image above A pose estimation method including
  7. In paragraph 1, The step of determining a pose according to the above-mentioned preset criteria is a pose estimation method that determines the pose by considering the number of matched feature points between the second image and the plurality of DB images or the error rate of the average distance between the matched feature points.
  8. In paragraph 1, The step of generating the first image and the second image is A pose estimation method for generating the first image and the second image by changing the size of the input image along the major axis.
  9. As a computer-readable recording medium storing a computer program, The above computer program is, A computer-readable recording medium comprising instructions for a processor to perform a method according to any one of claims 1 through 8.
  10. As a computer program stored on a computer-readable recording medium, The above computer program is, A computer program comprising instructions for a processor to perform a method according to any one of paragraphs 1 through 8.

Description

Method for Estimating Pose, Computer-Readable Storage Medium and Computer Program for Controlling the Holder Device The embodiment relates to a pose estimation method for effectively estimating the pose of a camera equipped in a terminal. Augmented Reality (AR) is a technology that uses location and orientation information to determine an approximate location, identifies the services desired by the user through a comparison between facility information—such as surrounding buildings—and real-world video information input based on camera movement, and provides relevant information. More specifically, augmented reality is a field of virtual reality (VR) that uses computer graphics techniques to superimpose virtual objects onto a real environment, making them appear as if they exist within the original environment. Unlike conventional virtual reality, which focuses solely on virtual spaces and objects, augmented reality is a technology that superimposes virtual objects onto a foundation of the real world to provide supplementary information that is difficult to obtain from the real world alone. With the commercialization of 5G communication, such augmented reality technology is gaining prominence in the field of mobile AR technology used in communication terminals, and marker-based mobile AR technology or sensor-based mobile AR technology is generally used in current mobile AR technology applications. Marker-based mobile AR technology is a technology that recognizes a real object by recognizing a marker corresponding to the real object when photographing a real object to be augmented using a virtual object, and sensor-based mobile AR technology is a technology that infers the current location and the direction the terminal is looking by using GPS and a digital compass installed in the terminal, and overlays POI (Point of Interests) information corresponding to the image in the inferred direction. However, marker-based mobile AR technology has the problem that it cannot augment virtual objects without markers, and sensor-based mobile AR technology has the problem of being unable to accurately augment virtual objects onto specific objects due to errors in the current location and orientation of the detected device. FIG. 1 is a block diagram schematically showing an apparatus for performing a pose estimation method according to an embodiment. FIG. 2 is a flowchart illustrating a pose estimation method according to an embodiment. FIG. 3 is a diagram illustrating the step of generating an image according to an embodiment. FIG. 4 is a diagram showing the feature points of the first image according to an embodiment. FIG. 5 is a flowchart illustrating a method for estimating the pose of a first image according to an embodiment. FIGS. 6 and FIGS. 7 are drawings for explaining a method for estimating the pose of a first image according to an embodiment. FIGS. 8 to 10 are drawings for explaining a method for estimating the pose of a second image according to an embodiment. Hereinafter, embodiments will be described in detail with reference to the drawings. FIG. 1 is a block diagram schematically showing an apparatus for performing a pose estimation method according to an embodiment. Referring to FIG. 1, the pose estimation method according to the embodiment can be performed in a pose estimation device (100). The pose estimation device (100) may receive an input image from a terminal (200). The terminal (200) may include various terminals such as mobile phones and computers. The terminal (200) may be equipped with a camera or a sensor. The pose estimation device (100) may obtain an input image from the camera or sensor of the terminal (200). The pose estimation device (100) can receive DB images from DB (300). DB (300) may be an area where accurate image information is stored. DB (300) may be created internally by collecting actual images and may be an area of a portal site such as Naver, Daum, or Google. The pose estimation device (100) can estimate the position of the terminal (200) using the input image and DB image, and can establish a foundation for providing AR services using this. The pose estimation device (100) can generate input images into multiple images. The pose estimation device (100) can generate an input image having a first image having a first reference size and a second image having a second reference size larger than the first reference size. Alternatively, the pose estimation device (100) may generate three or more images. The sizes of the first and second images may be changed to be smaller than the size of the input image. The second image may be the same size as the input image. The pose estimation device (100) can extract feature points for the first image. Since the size of the first image is smaller than the size of the second image, the number of feature points in the first image may be less than the number of feature points in the second image. The pose estimation device (100) can estimate a p