US-12627893-B2 - Device and method for producing panoramic image

US12627893B2US 12627893 B2US12627893 B2US 12627893B2US-12627893-B2

Abstract

A panoramic image generation device includes a photographing unit for receiving an input of multiple photographic images; a photographing information generation unit for generating photographing information of the photographic image on the basis of photographing posture information including information on a camera posture at which the photographic image is obtained; an image conversion unit for generating an output image obtained by converting the photographic image into a low-resolution image; an output unit for disposing the output image; an image stitching unit for sequentially performing stitching for the multiple photographic images corresponding to the output image disposed in the three-dimensional spatial area; a first memory unit provided to load the output image output from the output unit and the photographic image; and a second memory unit for storing the photographic image and the output image obtained through the conversion of the photographic image.

Inventors

Geun Woong Ryu
Yong Won CHO

Assignees

VISIT INC.

Dates

Publication Date: 20260512
Application Date: 20230203
Priority Date: 20220225

Claims (16)

1 . A panoramic image generation apparatus for generating a panoramic image based on a plurality of images, the panoramic image generation apparatus comprising: a photographing unit configured to receive a plurality of photographing images; a photographing information generation unit configured to generate photographing information of the photographing images based on photographing posture information comprising photographed camera posture information of the photographing images; an image conversion unit configured to generate output images that are obtained by converting the photographing images into low resolution images; an output unit configured to output the output images converted from the photographing images by disposing the output images in a three-dimensional space region based on the photographing information of the photographing images; an image stitching unit configured to generate a panoramic image by sequentially performing stitching on the plurality of photographing images corresponding to the output images disposed in the three-dimensional space region; a first memory unit provided so that the output images output to the output unit and the photographing images for the execution of the image stitching by the image stitching unit are loaded thereon; and a second memory unit configured to store the photographing images and the output images converted from the photographing images, wherein the image stitching unit performs the stitching by blending overlap regions that are regions that overlap between the photographing images disposed in the three-dimensional space region, and performs the blending between the photographing images by measuring a plane distance between the photographing images of the overlap region, which is measured by performing raycasting from an origin point of the three-dimensional space region, and adjusting a gradient of the overlap region of the photographing images based on the measured plane distance.
2 . The panoramic image generation apparatus of claim 1 , wherein: the output unit outputs the output image disposed in the three-dimensional space region by loading the output image stored in the second memory unit onto the first memory unit, and the image stitching unit generates the panoramic image by removing the output images loaded onto the first memory unit from the first memory unit, loading the photographing images corresponding to the output images onto the first memory unit, and performing the stitching between the photographing images loaded onto the first memory unit.
3 . The panoramic image generation apparatus of claim 2 , wherein the image stitching unit performs initial stitching that generates a first stitching image by performing stitching between arbitrary photographing images that neighbor each other, performs N-th stitching that generates an N-th stitching image by performing stitching between an (N−1)-th (N is an integer equal to or greater than 2) stitching image and a photographing image that neighbors the (N−1)-th stitching image, and generates the panoramic image based on the N-th stitching image by repeating the N-th stitching.
4 . The panoramic image generation apparatus of claim 3 , wherein the image stitching unit generates the N-th stitching image by loading the generated (N−1)-th stitching image and the photographing image that neighbors the (N−1)-th stitching image from the second memory unit to the first memory unit and performing the stitching between the (N−1)-th stitching image and the photographing image that neighbors the (N−1)-th stitching image through a single process.
5 . The panoramic image generation apparatus of claim 4 , wherein when performing the N-th stitching, the image stitching unit removes the generated (N−1)-th stitching image and the photographing image that neighbors the (N−1)-th stitching image on which the stitching has been completed, from the first memory unit.
6 . The panoramic image generation apparatus of claim 3 , wherein: the photographing information generation unit generates the photographing information based on the photographing posture information comprising photographed orientation information of the photographing image, and the image stitching unit performs the stitching between the photographing images so that a photographing image having the orientation information adjacent to preset orientation information that is a reference by which the photographing images included in the panoramic image are preferentially output is disposed at a center of the panoramic image.
7 . The panoramic image generation apparatus of claim 1 , further comprising a path information generation unit configured to generate stitching path information that is arrangement information of the photographing images corresponding to the output images disposed in the three-dimensional space region, wherein the photographing information generation unit generates the photographing information in a form of a file name of the photographing image, the path information generation unit generates the stitching path information based on the file name of the photographing image, and the image stitching unit performs the stitching between the photographing images by disposing the photographing images in the three-dimensional space region based on the stitching path information.
8 . The panoramic image generation apparatus of claim 1 , wherein the photographing unit receives the plurality of photographing images photographed based on nodes that are disposed at preset intervals in the three-dimensional space region.
9 . A panoramic image generation method using a panoramic image generation apparatus, comprising: a photographing image input step of receiving a plurality of photographing images; a photographing information generation step of generating photographing information of the photographing images based on photographing posture information comprising photographed camera posture information of the photographing images; an image conversion step of generating output images that are obtained by converting the photographing images into low resolution images; an image storage step of storing the photographing images and the output images converted from the photographing images in a second memory unit; a photographing image output step of outputting the output images converted from the photographing images by disposing the output images in a three-dimensional space region based on the photographing information of the photographing images; and an image stitching execution step of generating a panoramic image by sequentially performing stitching on the plurality of photographing images corresponding to the output images disposed in the three-dimensional space region, wherein the photographing image output step comprises outputting the output images disposed in the three-dimensional space region by loading the output images stored in the second memory unit onto a first memory unit, and the image stitching execution step comprises loading the photographing images onto the first memory unit for the image stitching and performing the stitching between the photographing images loaded onto the first memory unit, and wherein in the image stitching execution step, the stitching is performed by blending overlap regions that are regions that overlap between the photographing images disposed in the three-dimensional space region, and the blending between the photographing images is performed by measuring a plane distance between the photographing images of the overlap region, which is measured by performing raycasting from an origin point of the three-dimensional space region, and adjusting a gradient of the overlap region of the photographing images based on the measured plane distance.
10 . The panoramic image generation method of claim 9 , wherein in the image stitching execution step, the panoramic image is generated by removing the output images loaded onto the first memory unit from the first memory unit, loading the photographing images corresponding to the output images onto the first memory unit, and performing the stitching between the photographing images loaded onto the first memory unit.
11 . The panoramic image generation method of claim 10 , wherein the image stitching execution step comprises: an initial stitching execution step of generating a first stitching image by performing stitching between arbitrary photographing images that neighbor each other; and an N-th stitching execution step of generating an N-th stitching image by performing stitching between an (N−1)-th (N is an integer equal to or greater than 2) stitching image and a photographing image that neighbors the (N−1)-th stitching image, wherein the panoramic image is generated based on the N-th stitching image by repeatedly performing the N-th stitching execution step.
12 . The panoramic image generation method of claim 11 , wherein in the N-th stitching execution step, the N-th stitching image is generated by loading the generated (N−1)-th stitching image and the photographing image that neighbors the (N−1)-th stitching image from the second memory unit to the first memory unit and performing the stitching between the (N−1)-th stitching image and the photographing image that neighbors the (N−1)-th stitching image through a single process.
13 . The panoramic image generation method of claim 12 , wherein in the N-th stitching execution step, the generated (N−1)-th stitching image and the photographing image that neighbors the (N−1)-th stitching image on which the stitching has been completed are removed from the first memory unit.
14 . The panoramic image generation method of claim 11 , wherein: in the photographing information generation step, the photographing information is generated based on the photographing posture information comprising photographed orientation information of the photographing image, and in the image stitching execution step, the stitching between the photographing images is performed so that a photographing image having the orientation information adjacent to preset orientation information that is a reference by which the photographing images included in the panoramic image are preferentially output is disposed at a center of the panoramic image.
15 . The panoramic image generation method of claim 9 , further comprising a path information generation step of generating stitching path information that is arrangement information of the photographing images corresponding to the output images disposed in the three-dimensional space region, wherein in the photographing information generation step, the photographing information is generated in a form of a file name of the photographing image, in the path information generation step, the stitching path information is generated based on the file name of the photographing image, and in the image stitching execution step, the stitching between the photographing images is performed by disposing the photographing images in the three-dimensional space region based on the stitching path information.
16 . The panoramic image generation method of claim 9 , wherein in the photographing image input step, the plurality of photographing images photographed based on nodes that are disposed at preset intervals in the three-dimensional space region is received.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application is a national phase of International Application No. PCT/KR2023/001551 filed on Feb. 3, 2023, which claims priority to Korean Patent Application No. 10-2022-0025212 filed on Feb. 25, 2022, the entire contents of which are herein incorporated by reference. TECHNICAL FIELD The present disclosure relates to a panoramic image generation apparatus and method, and to a panoramic image generation apparatus and method which generate a panoramic image based on a plurality of photographing images. BACKGROUND ART In order to implement a 360-degree image, a panoramic image may be generated based on an image stitched by using a large number of photographing images. A 360-degree image stitched by mapping the panoramic image to a surface of a sphere may be output to a user. In this case, in order to smoothly provide the user with the panoramic image constructed by stitching the plurality of photographing images, sufficient memory availability of a device from which the panoramic image is output needs to be secured. In a conventional technology, in order to consider the solving means, memory management is dynamically performed by loading only an image that is output to a user, unloading an image that is not output, and leaving only a memory reference. DISCLOSURE Technical Problem Embodiments of the present disclosure are intended to provide a panoramic image generation apparatus and method which provide a user with a panoramic image that is generated based on a plurality of photographing images for which memory has been optimized. Technical Solution A panoramic image generation apparatus for generating a panoramic image based on a plurality of images according to an aspect of an embodiment of the present disclosure includes a photographing unit configured to receive a plurality of photographing image that has been photographed, a photographing information generation unit configured to generate photographing information of the photographing images based on photographing posture information including photographed camera posture information of the photographing images, an image conversion unit configured to generate output images that are obtained by converting the photographing images into low resolution images, an output unit configured to output the output images converted from the photographing images by disposing the output images in a three-dimensional space region based on the photographing information of the photographing images, an image stitching unit configured to generate a panoramic image by sequentially performing stitching on the plurality of photographing images corresponding to the output images disposed in the three-dimensional space region, a first memory unit provided so that the output images output to the output unit and the photographing images for the execution of the image stitching by the image stitching unit are loaded thereon, and a second memory unit configured to store the photographing images and the output images converted from the photographing images. Furthermore, the output unit may output the output image disposed in the three-dimensional space region by loading the output image stored in the second memory unit onto the first memory unit. The image stitching unit may generate the panoramic image by removing the output images loaded onto the first memory unit from the first memory unit, loading the photographing images corresponding to the output images onto the first memory unit, and performing the stitching between the photographing images loaded onto the first memory unit. Furthermore, the image stitching unit may perform initial stitching that generates a first stitching image by performing stitching between arbitrary photographing images that neighbor each other, may perform N-th stitching that generates an N-th stitching image by performing stitching between an (N−1)-th (N is an integer equal to or greater than 2) stitching image and a photographing image that neighbors the (N−1)-th stitching image, and may generate the panoramic image based on the N-th stitching image by repeating the N-th stitching. Furthermore, the image stitching unit may generate the N-th stitching image by loading the generated (N−1)-th stitching image and the photographing image that neighbors the (N−1)-th stitching image from the second memory unit to the first memory unit and performing the stitching between the (N−1)-th stitching image and the photographing image that neighbors the (N−1)-th stitching image through a single process. Furthermore, when performing the N-th stitching, the image stitching unit may remove the generated (N−1)-th stitching image and the photographing image that neighbors the (N−1)-th stitching image on which the stitching has been completed, from the first memory unit. Furthermore, the photographing information generation unit may generate the photographing information based on the photographing posture information including pho