Search

US-12621438-B2 - Method and apparatus for reconstructing 360-degree image according to projection format

US12621438B2US 12621438 B2US12621438 B2US 12621438B2US-12621438-B2

Abstract

Disclosed are methods and apparatuses for image data encoding/decoding. A method for decoding a 360-degree image includes the steps of: receiving a bitstream obtained by encoding a 360-degree image; generating a prediction image by making reference to syntax information obtained from the received bitstream; adding the generated prediction image to a residual image obtained by dequantizing and inverse-transforming the bitstream, so as to obtain a decoded image; and reconstructing the decoded image into a 360-degree image according to a projection format. Therefore, the performance of image data compression can be improved.

Inventors

  • Ki Baek Kim

Assignees

  • B1 INSTITUTE OF IMAGE TECHNOLOGY, INC.

Dates

Publication Date
20260505
Application Date
20250730
Priority Date
20161004

Claims (4)

  1. 1 . A method of decoding an image with a decoding apparatus, comprising: receiving encoding data for a current block included in the image; determining, based on the encoding data, whether to perform partitioning for the current block, the current block being partitioned into a plurality of sub-partitions when the partitioning being performed for the current block; parsing syntax elements for a residual block of the current block from the encoding data; obtaining residual coefficients of the residual block based on the syntax elements for the residual block; obtaining the residual block based on the residual coefficients of the residual block, the residual block being obtained for each of the plurality of sub-partitions when the partitioning being performed for the current block; generating a prediction block for the current block, the prediction block being generated for each of the plurality of sub-partitions when the partitioning being performed for the current block; reconstructing the current block based on the residual block and the prediction block to reconstruct the image; and performing a post processing on the reconstructed image based on post processing information included in the encoding data, wherein the residual coefficients are obtained by performing inverse-transformation, wherein the post processing comprises padding at least one region to the reconstructed image, wherein the padding is performed by determining at least one equation defined as a function of a width of the image or a height of the image, determining a sample inside the reconstructed image based on the equation, and using the sample inside the reconstructed image for padding the region, and wherein the padding is performed by partitioning the reconstructed image into a plurality of partitioning units; and performing the padding for the each partitioning unit.
  2. 2 . The method of claim 1 , wherein the post processing information is a SEI message.
  3. 3 . A method of encoding an image with an encoding apparatus, comprising: determining whether to perform partitioning for a current block included in the image, the current block being partitioned into a plurality of sub-partitions when the partitioning being performed for the current block, the determination being encoded into encoding data for the current block; generating a prediction block of the current block included in the image, the prediction block being generated for each of the plurality of sub-partitions when the partitioning being performed for the current block; obtaining a residual block based on the prediction block, the residual block being obtained for each of the plurality of sub-partitions when the partitioning being performed for the current block; encoding residual coefficients of the residual block; including syntax elements for the residual coefficients into encoding data for the current block; and encoding post processing information into the encoding data, wherein the residual coefficients are encoded by performing transformation, wherein the post processing information is used for performing a post processing on a reconstructed image, the reconstructed image being an image obtained based on the residual block of the current block, and wherein the post processing comprises padding at least one region to the reconstructed image, wherein the padding is performed by determining at least one equation defined as a function of a width of the image or a height of the image, determining a sample inside the reconstructed image based on the equation, and using the sample inside the reconstructed image for padding the region, and wherein the padding is performed by partitioning the reconstructed image into a plurality of partitioning units; and performing the padding for the each partitioning unit.
  4. 4 . A method of transmitting a bitstream, the method comprises: determining whether to perform partitioning for a current block included in the image, the current block being partitioned into a plurality of sub-partitions when the partitioning being performed for the current block, the determination being encoded into encoding data for the current block; generating a prediction block of the current block included in the image, the prediction block being generated for each of the plurality of sub-partitions when the partitioning being performed for the current block; obtaining a residual block based on the prediction block, the residual block being obtained for each of the plurality of sub-partitions when the partitioning being performed for the current block; encoding residual coefficients of the residual block; including syntax elements for the residual coefficients into encoding data for the current block; encoding post processing information into the encoding data; and transmitting the bitstream including the encoding data, wherein the residual coefficients are encoded by performing transformation, wherein the post processing information is used for performing a post processing on a reconstructed image, the reconstructed image being an image obtained based on the residual block of the current block, and wherein the post processing comprises padding at least one region to the reconstructed image, wherein the padding is performed by determining at least one equation defined as a function of a width of the image or a height of the image, determining a sample inside the reconstructed image based on the equation, and using the sample inside the reconstructed image for padding the region, and wherein the padding is performed by partitioning the reconstructed image into a plurality of partitioning units; and performing the padding for the each partitioning unit.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation of U.S. patent application Ser. No. 18/971,895 filed on Dec. 6, 2024, which is a continuation of U.S. patent application Ser. No. 18/672,265 filed on May 23, 2024, which is now U.S. Pat. No. 12,244,930, which is a continuation of U.S. patent application Ser. No. 18/466,442 filed on Sep. 13, 2023, which is now U.S. Pat. No. 12,167,138, which is a continuation of U.S. patent application Ser. No. 17/487,277 filed on Sep. 28, 2021, which is now U.S. Pat. No. 11,831,818, which is a continuation of U.S. patent application Ser. No. 16/372,237 filed on Apr. 1, 2019, now U.S. Pat. No. 11,165,958, which is a continuation of International Patent Application Number PCT/KR2017/011143, filed Oct. 10, 2017, which claims priority to Korean Patent Application Number 10-2016-0127878, filed Oct. 4, 2016; Korean Patent Application Number 10-2016-0129382, filed Oct. 6, 2016; and Korean Patent Application Number 10-2017-0090612, filed Jul. 17, 2017. All of these applications are incorporated by reference herein in their entireties. TECHNICAL FIELD The present invention relates to image data encoding and decoding technology, and more particularly, to a method and apparatus for encoding and decoding a 360-degree image for realistic media service. BACKGROUND With the spread of the Internet and mobile terminals and the development of information and communication technology, the use of multimedia data is increasing rapidly. Recently, demand for high-resolution images and high-quality images such as a high definition (HD) image and an ultra high definition (UHD) image is emerging in various fields, and demand for realistic media service such as virtual reality, augmented reality, and the like is increasing rapidly. In particular, since multi-view images captured with a plurality of cameras are processed for 360-degree images for virtual reality and augmented reality, the amount of data generated for the processing increases massively, but the performance of an image processing system for processing a large amount of data is insufficient. As described above, in an image encoding and decoding method and apparatus of the related art, there is a demand for improvement of performance in image processing, particularly, image encoding/decoding. SUMMARY It is an object of the present invention to provide a method for improving an image setting process in initial steps for encoding and decoding. More particularly, the present invention is directed to providing an encoding and decoding method and apparatus for improving an image setting process in consideration of the characteristics of a 360-degree image. According to an aspect of the present invention, there is provided a method of decoding a 360-degree image. Here, the method of decoding a 360-degree image may include receiving a bitstream including an encoded 360-degree image, generating a predicted image with reference to syntax information acquired from the received bitstream, acquiring a decoded image by combining the generated predicted image with a residual image acquired by inversely quantizing and inversely transforming the bitstream, and reconstructing the decoded image into the 360-degree image according to a projection format. Here, the syntax information may include projection format information for the 360-degree image. Here, the projection format information may be information indicating at least one of an Equi-Rectangular Projection (ERP) format in which the 360-degree image is projected into a 2D plane, a CubeMap Projection (CMP) format in which the 360-degree image is projected to a cube, an OctaHedron Projection (OHP) format in which the 360-degree image is projected to an octahedron, and an IcoSahedral Projection (ISP) format in which the 360-degree image is projected to a polyhedron. Here, the reconstructing may include acquiring arrangement information according to region-wise packing with reference to the syntax information and rearranging blocks of the decoded image according to the arrangement information. Here, the generating of the predicted image may include performing image expansion on a reference picture acquired by restoring the bitstream, and generating a predicted image with reference to the reference picture on which the image expansion is performed. Here, the performing of the image expansion may include performing image expansion on the basis of partitioning units of the reference picture. Here, the performing of the image expansion on the basis of the partitioning units may include generating an expanded region individually for each partitioning unit by using the reference pixel of the partitioning unit. Here, the expanded region may be generated using a boundary pixel of a partitioning unit spatially adjacent to a partitioning unit to be expanded or using a boundary pixel of a partitioning unit having image continuity with a partitioning unit to be expanded. Here, the performing