US-20260129182-A1 - METHOD AND APPARATUS FOR PATCH BOOK-BASED ENCODING AND DECODING OF VIDEO DATA

US20260129182A1US 20260129182 A1US20260129182 A1US 20260129182A1US-20260129182-A1

Abstract

Disclosed are a method and an apparatus for patch book-based encoding and decoding of video data. A video data decoding method comprises the steps of: determining a patch book list for a block of video data; and determining one or more index values for the block. Each of the index values indicates a patch book entry having color values corresponding to a subset of pixels in the block, among patch book entries of the patch book list. The decoding method further comprises a step of reconstructing subsets of the pixels on the basis of the patch book list and the index values for the block.

Inventors

Dong Gyu Sim
Joo Hyung Byeon
Sea Nae Park
Seung Wook Park
Wha Pyeong Lim

Assignees

HYUNDAI MOTOR COMPANY
KIA CORPORATION
KWANGWOON UNIVERSITY INDUSTRY-ACADEMIC COLLABORATION FOUNDATION

Dates

Publication Date: 20260507
Application Date: 20251229
Priority Date: 20200804

Claims (20)

1 . A method for decoding video data, the method comprising: determining a patchbook list for a block of video data including patchbook entries, wherein at least one of the patchbook entries contains N×M elements defining colors for N×M pixels, and each of N×M elements has one or more color component values for an individual pixel, where at least one of N or M is an integer greater than or equal to 2; decoding one or more index values for the block of video data, wherein each of the decoded one or more index values indicates a corresponding patchbook entry to be used in predicting a subset in the block of video data among the patchbook entries of the patchbook list, wherein the subset is an array of a plurality of spatially adjacent pixels and within the block of video data, and a size of the array of the subset is the same as a size of the corresponding patchbook entry; and reconstructing subsets of the block of video data subset-wisely, based on the patchbook list for the block of video data and the decoded one or more index values.
2 . The method of claim 1 , wherein reconstructing the subsets includes: generating predicted values for the subsets subset-wisely using patchbook entries indicated by the decoded one or more index values; determining residual values for the subsets from a bitstream; and adding the residual values to the predicted values.
3 . The method of claim 1 , wherein reconstructing the subsets includes generating reconstructed sample values for the subsets from patchbook entries indicated by the index values without determining residual values for the subsets from a bitstream.
4 . The method of claim 1 , further including: determining whether the block contains a subset having escape pixels related to none of the patchbook entries of the patchbook list; obtaining encoded sample values for the subset having escape pixels from a bitstream; and performing inverse-transform and inverse quantization on the encoded sample values to reconstruct the subset having escape pixels.
5 . The method of claim 1 , further including: determining whether the block contains a subset having escape pixels related to none of the patchbook entries of the patchbook list; generating predicted values for the subset having escape pixels from a reconstructed block adjacent to the block; obtaining residual values for the subset having escape pixels from a bitstream; and inverse quantization adding the residual values to the predicted values to reconstruct the subset having escape pixels.
6 . The method of claim 1 , wherein determining a patchbook list for a block of video data includes: determining a patchbook prediction list including one or more candidate patchbooks; and determining at least one entry used in the patchbook list for the block of video data among entries of the patchbook prediction list.
7 . The method of claim 6 , wherein determining a patchbook prediction list includes determining entries of a patchbook list used for another previously reconstructed block of the video data as an entry of the patchbook prediction list.
8 . The method of claim 6 , wherein determining a patchbook prediction list includes, among previously reconstructed blocks of the video data, generating a new entry of the patchbook prediction list from at least one of (1) a reconstruction block of a block encoded in an intra prediction, (2) a reconstruction block of a block encoded in an inter prediction, or (3) a reconstruction block of a block encoded in an intra block copy (IBC) mode.
9 . The method of claim 1 , wherein determining a patchbook prediction list includes generating a new entry of the patchbook prediction list by stitching entries existing in a patchbook prediction list determined for another previously reconstructed block of the video data.
10 . The method of claim 1 , wherein determining a patchbook list for a block of video data includes: identifying a default patchbook list specified by high level syntax elements signaled in a bitstream; and determining at least one entry used for the patchbook list for the block of video data among entries of the default patchbook list.
11 . A method for encoding video data, the method comprising: determining a patchbook list for a block of video data including patchbook entries, wherein at least one of the patchbook entries contains N×M elements defining colors for N×M pixels, and each of N×M elements has one or more color component values for an individual pixel, where at least one of N or M is an integer greater than or equal to 2; determining one or more index values for the block of video data, wherein each of the determined one or more index values indicates a corresponding patchbook entry to be used in predicting a subset in the block of video data among the patchbook entries of the patchbook list, wherein the subset is an array of a plurality of spatially adjacent pixels within the block of video data, and a size of the array of the subset is the same as a size of the corresponding patchbook entry; and encoding the determined one or more index values for the block of video data.
12 . The method of claim 11 , further including: generating predicted values for subsets of the block of video data subset-wisely using patchbook entries indicated by the determined one or more index values; determining residual values for the subsets from predicted values for the subsets; and encoding the residual values for the subsets.
13 . The method of claim 11 , further including: determining whether the block of video data contains a subset having escape pixels related to none of the patchbook entries of the patchbook list; and performing an encoding process on sample values of the subset having escape pixels.
14 . The method of claim 11 , further including: determining whether the block of video data contains a subset having escape pixels related to none of the patchbook entries of the patchbook list; generating predicted values for the subset having escape pixels from a reconstructed block adjacent to the block of video data; generating residual values for the subset having escape pixels based on the predicted values for the subset having escape pixels; and encoding the residual values for the subset having escape pixels.
15 . The method of claim 11 , further including: determining a patchbook prediction list including one or more candidate patchbooks; and encoding a sequence of binary flags indicating at least one entry used in the patchbook list for the block of video data among entries of the patchbook prediction list.
16 . The method of claim 15 , wherein determining a patchbook prediction list includes determining entries of the patchbook list used for another previously reconstructed block of the video data as an entry of the patchbook prediction list.
17 . The method of claim 15 , wherein determining a patchbook prediction list includes, among previously encoded blocks of the video data, generating a new entry from at least one of (1) a reconstructed block of a block encoded in an intra prediction, (2) a reconstructed block of a block encoded in an inter prediction, or (3) a reconstructed block of a block encoded in an intra block copy (IBC) mode.
18 . The method of claim 11 , wherein determining a patchbook prediction list includes generating a new entry by stitching entries existing in a patchbook prediction list determined for another previously reconstructed block of the video data.
19 . The method of claim 11 , wherein determining a patchbook list for a block of video data includes: identifying a default patchbook list specified by high level syntax elements signaled in a bitstream; and determining at least one entry used for the patchbook list for the block of video data among entries of the default patchbook list.
20 . A method for providing video data to a video decoding device, the method comprising: encoding the video data into a bitstream; and transmitting the bitstream to the video decoding device, wherein encoding the video data comprises: determining a patchbook list for a block of the video data including patchbook entries, wherein at least one of the patchbook entries contains N×M elements defining colors for N×M pixels, and each of N×M elements has one or more color component values for an individual pixel, where at least one of N or M is an integer greater than or equal to 2; determining one or more index values for the block of video data, wherein each of the determined one or more index values indicates a corresponding patchbook entry to be used in predicting a subset in the block of video data among the patchbook entries of the patchbook list, wherein the subset is an array of a plurality of spatially adjacent pixels within in the block of video data, and a size of the array of the subset is the same as a size of the corresponding patchbook entry; and encoding the determined one or more index values for the block of video data.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation application of non-provisional U.S. patent application Ser. No. 18/104,984, filed on Feb. 2, 2023, which was a continuation of International Application No. PCT/KR2021/010196, filed on Aug. 4, 2021, which claims priority to Korean Patent Application No. 10-2020-0097613 filed on Aug. 4, 2020, and Korean Patent Application No. 10-2021-0102129 filed on Aug. 3, 2021, the entire disclosures of each of which are incorporated herein by reference. TECHNICAL FIELD The present disclosure relates to encoding and decoding of video data and, more particularly, to a method and an apparatus for patch book-based encoding and decoding of video data. BACKGROUND Since the volume of video data is greater than that of voice data or still image data, storing or transmitting video data without processing for compression requires a lot of hardware resources including memory. Accordingly, in storing or transmitting video data, the video data is generally compressed using an encoder so as to be stored or transmitted. Then, a decoder receives the compressed video data and decompresses and reproduces the video data. The picture size, resolution, and frame rate are gradually increasing, and accordingly the amount of data to be encoded is also increasing. Accordingly, a new compression technique having better encoding efficiency and higher image quality than the existing compression technique is required. In particular, more efficient encoding and decoding techniques are required for screen content videos, such as animation and computer graphics. SUMMARY The present disclosure provides a new coding tool capable of improving encoding efficiency using texture similarity in video data. In accordance with one aspect of the present disclosure, a method for encoding video data comprises determining a patchbook list for a block of video data; determining one or more index values for the block of video data; and encoding the index values for the block of video data. Each index value indicates a patchbook entry having color values corresponding to a subset of pixels in the block of video data among patchbook entries of the patchbook list. The patchbook is a block with a texture that frequently appears in the frame(s) of the video data. The method further comprises generating predicted values for the subsets of pixels from patchbook entries indicated by the index values; determining residual values for the subsets of pixels from predicted values for the subsets of pixels; and encoding the residual values for the subsets of pixels. Alternatively, the subsets of pixels may be reconstructed by a video decoder based on the patchbook list and the index values, without the aid of residual values, so that the above-described steps including encoding the residual values may be omitted. In accordance with another aspect of the present disclosure, a method for decoding video data comprises determining a patchbook list for a block of video data; and determining one or more index values for the block of video data. Each index value indicates a patchbook entry having color values corresponding to a subset of pixels in the block of video data among patchbook entries of the patchbook list. The method further comprises reconstructing the subsets of pixels based on the patchbook list for the block of video data and the index values. The reconstructing subsets of pixels includes generating prediction values predicted values for the subsets of pixels from patchbook entries indicated by the index values, determining residual values for the subsets of pixels from a bitstream; and adding the residual values to the prediction values predicted values. Alternatively, the reconstructing subsets of pixels includes generating reconstructed sample values for the subsets of pixels from patchbook entries indicated by the index values without determining residual values for the subsets of pixels from the a bitstream. The encoding method and the decoding method described above may be performed or implemented by a video encoding apparatus and a video decoding apparatus, respectively. Such apparatus includes a memory and one or more processors, and the one or more processors may be configured to perform the steps of the encoding method described above or the steps of the decoding method described above. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram of a video encoding apparatus that may implement the techniques of the present disclosure. FIG. 2 illustrates a method for partitioning a block using a quadtree plus binarytree ternarytree (QTBTTT) structure. FIG. 3A illustrates a plurality of intra prediction modes. FIG. 3B illustrates a plurality of intra prediction modes including wide-angle intra prediction modes. FIG. 4 is a block diagram of a video decoding apparatus that may implement the techniques of the present disclosure. FIG. 5 is a conceptual diagram illustrating a patchbook list for co