Search

EP-3520072-B1 - ELECTRONIC APPARATUS AND METHOD FOR PROCESSING IMAGE THEREOF

EP3520072B1EP 3520072 B1EP3520072 B1EP 3520072B1EP-3520072-B1

Inventors

  • KIM, SUNG-HYUN
  • KIM, BO-EUN
  • LEE, JONG-IN
  • KIM, YONG-DEOK

Dates

Publication Date
20260506
Application Date
20171110

Claims (12)

  1. An electronic apparatus (100) comprising: a storage (110) ; and a processor (120) configured to : convert an original image to a spherical image, obtain a plurality of two-dimensional, 2D, images corresponding to a respective plurality of points on the spherical image, wherein the plurality of 2D images are obtained by projecting the spherical image onto a plane in a plurality of different directions at the plurality of points, wherein each of the plurality of 2D images corresponds to the spherical image so that a given area of the spherical image is present in each of the plurality of 2D images and is distorted to different extents in each of the plurality of 2D images, and store the obtained 2D images in the storage, wherein in response to image analysis for a selected region of the original image being started, the processor is configured to select a 2D image of the stored 2D images in the storage, where the selected region is located in a predetermined region of each 2D image from among the plurality of 2D images and the selected region corresponds to the least-distorted region that is a region within a predetermined distance from a center horizontal line of the 2D image, and to perform the image analysis for the selected region in the selected 2D image.
  2. The apparatus as claimed in claim 1, wherein the processor is configured to divide the spherical image into a plurality of regions, to identify the plurality of points for the respective plurality of regions, and to obtain the plurality of 2D images by projecting a corresponding region of the spherical image onto the plane for each of the identified points.
  3. The apparatus as claimed in claim 2, wherein the processor is configured to extend the respective plurality of regions to overlap each other and to acquire the plurality of 2D images by projecting the respective extended regions of the spherical image onto the plane for each of the identified points.
  4. The apparatus as claimed in claim 3, wherein in response to image processing for an overlapped region in the plurality of 2D images being performed and the plurality of 2D images being reconstructed to a new original image, the processor is configured to reconstruct the new original image using at least one of: an average value, a median value, a maximum value, and a minimum value of pixels corresponding to each other in the overlapped region.
  5. The apparatus as claimed in claim 2, wherein the processor is configured to identify a plurality of additional regions including boundaries of the plurality of regions, to identify additional points for the respective plurality of additional regions, and to acquire a plurality of additional 2D images by projecting a corresponding additional region of the spherical image onto the plane with reference to the identified additional points.
  6. The apparatus as claimed in claim 2, wherein the processor is configured to divide the spherical image into a plurality of new regions based on a new criterion different from a criterion for dividing the plurality of regions, to identify the points for the respective plurality of new regions, to obtain a plurality of new 2D images by projecting a corresponding region of the spherical image onto the plane for each of the identified points, and to perform the image analysis using the plurality of 2D image and the plurality of new 2D images.
  7. The apparatus as claimed in claim 1, wherein the processor is configured to add a region in a first side of a 2D image from among the plurality of 2D images to a side opposite to the first side of the 2D image and to change the 2D image.
  8. The apparatus as claimed in claim 1, wherein the storage stores an original video including a plurality of successive original images, wherein the processor configured to convert the plurality of original images to spherical images respectively, and in response to a random object being detected from a first spherical image which is one of the converted spherical images, to obtain a first two-dimensional, 2D, image by projecting the first spherical image onto the plane with reference to a first point where the object was detected, wherein the processor is configured to detect the object from a second spherical image subsequent to the first spherical image, in response to a distance between the first point and a second point where the object was detected being at least longer than a predetermined distance, to obtain a second two-dimensional, 2D, image by projecting the second spherical image onto the plane with reference to the second point, and in response to the distance between the first point and the second point being shorter than the predetermined distance, to obtain the second 2D image by projecting the second spherical image onto the plane with reference to the first point.
  9. The apparatus as claimed in claim 1, further comprising: a display, wherein the processor is configured to control the display to display a predetermined first region of one of the plurality of 2D images, to detect a 2D image where a predetermined object is located in a predetermined second region of each 2D image from among the plurality of 2D images, and to store information on the detected 2D image in the storage.
  10. A computer-implemented method for processing an image of an electronic apparatus, the method comprising: converting an original image to a spherical image; obtaining a plurality of two-dimensional, 2D, images corresponding to a respective plurality of points on the spherical image, wherein the plurality of 2D images are obtained by projecting the spherical image onto a plane in a plurality of different directions at the plurality of points; and selecting, in response to image analysis for a selected region of the original image being started, a 2D image where the selected region is located in a predetermined region of each 2D image from among the plurality of 2D images and the selected region corresponds to the least-distorted region that is a region within a predetermined distance from a center horizontal line of the 2D image, and performing the image analysis for the selected region in the selected 2D image; wherein each of the plurality of 2D images corresponds to the spherical image so that a given area of the spherical image is present in each of the plurality of 2D images and is distorted to different extents in each of the plurality of 2D images.
  11. The method as claimed in claim 10, wherein the obtaining comprises dividing the spherical image to a plurality of regions, identifying the plurality of points for the respective plurality of regions, and acquiring the plurality of 2D images by projecting a corresponding region of the spherical image onto the plane for each of the identified points.
  12. The method as claimed in claim 11, wherein the obtaining comprises extending the respective plurality of regions to overlap each other and acquiring the plurality of 2D images by projecting the respective extended regions of the spherical image onto the plane for each of the identified points.

Description

[Technical Field] The present disclosure relates generally to an electronic apparatus and a method for processing an image thereof, and for example, to an electronic apparatus that processes a spherical image and a method for processing an image thereof. [Background Art] With the increasing interest in Virtual Realty (VR), various types of 360-degree photographing apparatuses for personal use have been developed and came on the market. Further, it is expected that the 360-degree photographing apparatus will be applied to an omnidirectional Closed Circuit Television (CCTV), a black box, and so on. ZELNIK-MANOR LET AL: "SQUARING THE CIRCLES IN PANORAMAS", COMPUTER VISON, 2005. ICCV 2005. THENTH IEEE INTERNATIONAL CONFERENCE ON BEIJING, CHINA 17-20 OCT. 2005, PISCATAWY, JN,USA, IEEE, LOS ALAMITOS, CA, USA discloses a projection method which incorporate multiple local projections with multiple viewpoints into the same panorama. US 2009/123088 A1 relates to a method of using tiled projection to process round earth data. Johannes Kopf ET AL: "Locally adapted Projections to Reduce Panorama Distortions", Computer graphics forum: journal of the European Association for Computer Graphics, vol. 28, no. 4, 1 June 2009 (2009-06-01), pages 1083-1089 relates to a method of constructing a projection surface comprising both straight and planar sections. As 360-degree images are produced in diverse fields, there is a growing necessity for analyzing the images. As an example, in a CCTV or a black box, it is necessary to detect an event or detect and track an object. As another example, various kinds of apparatuses, such as, a Head Mounted Device (HMD), a personal computer (PC), or a television (TV), provide a function for a user to enjoy 360-degree contents. In this case, image analysis is required to provide auto-movement of viewpoint. There are diverse methods for detecting a feature, extracting a feature point, applying an image filter, detecting an object, or tracking an object, but such methods have been developed only for the conventional image and are still inadequate to process a planar image generated by projecting a 360-degree image. By way of example, an equirectangular image is generated by projecting a surface of a sphere in an equirectangular manner. The equirectangular image is characterized in that its shape is more distorted towards both poles, and the poles are extended in a widthwise direction of the image. Accordingly, an image analyzing method for a general image is not suitable for the equirectangular image, and the performance may be degraded due to distortion of the equirectangular image. Further, in the 360-degree image, edge parts of the image are cut off unavoidably as a sphere is spread out to a plane. That is, in the equirectangular image, an object on a left edge part and an object on a right edge part are connected originally, and it is difficult to detect a feature or an object from the disconnected part. Accordingly, there are efforts to seek a method for improving performance of analyzing and processing a 360-degree image. [Disclosure] [Technical Problem] The present disclosure addresses the aforementioned and other problems and disadvantages occurring in the related art, and an example aspect of the present disclosure provides an electronic apparatus for processing a spherical image without (and/or reducing) distortion and a method for processing an image thereof. [Technical Solution] According to an example embodiment of the present disclosure, an electronic apparatus is provided in accordance with claim 1. A method for processing an image of an electronic apparatus is provided in accordance with claim 10. [Advantageous Effects] According to the above-described various example embodiments of the present disclosure, the electronic apparatus may acquire a plurality of 2D images from a spherical image and analyze the plurality of 2D images thereby minimizing and/or reducing distortion that may occur while analyzing the spherical image. [Description of Drawings] The above and/or other aspects, features and attendant advantages of the present disclosure will be more apparent and readily appreciated from the following detailed description, taken in conjunction with the accompanying drawings, in which like reference numerals refer to like elements, and wherein: FIG. 1 is a block diagram illustrating an example structure of an electronic apparatus according to an example embodiment of the present disclosure;FIGS. 2A, 2B and 2C are diagrams illustrating an example spherical image for better understanding of the present disclosure;FIGS. 3A, 3B, 3C and 3D are diagrams illustrating an example of converting an original image itself according to an example embodiment of the present disclosure;FIGS. 4A, 4B, 4C and 4D are diagrams illustrating an example of converting an original image itself according to another example embodiment of the present disclosure;FIG. 5 is a diagram illustrating an example metho