Search

EP-4736421-A1 - SBT APPLIED TO IBC AND INTRATMP

EP4736421A1EP 4736421 A1EP4736421 A1EP 4736421A1EP-4736421-A1

Abstract

Systems, methods, and instrumentalities are disclosed for performing subblock transform (SBT) applied to inter block copy (IBC) and intra template matching prediction (TMP). SBT may be performed in combination with IBC and intra TMP. A coding unit predicted by IBC or intra TMP may benefit from performing SBT (e.g., splitting the coding block into a zero spatial sub-part and a non-zero coded sub-part).

Inventors

  • LE LEANNEC, FABRICE
  • RATH, Gagan Bihari
  • NASER, Karam
  • CHEN, Ya

Assignees

  • InterDigital CE Patent Holdings, SAS

Dates

Publication Date
20260506
Application Date
20240624

Claims (20)

  1. 1 . A video decoding device, comprising: a processor configured to: obtain coding mode information associated with a coding block, wherein the coding mode information indicates to use a block vector based prediction mode; determine to use subblock transform (SBT) on a subblock associated with the coding block; based on the determination to use SBT on the subblock, obtain SBT information associated with the subblock; perform a first decoding of the coding block based on the coding mode information; perform a second decoding of the coding block based on the SBT information; and reconstruct the coding block based on the performed first decoding and the second decoding.
  2. 2. The video decoding device of claim 1 , wherein the block vector based prediction mode is at least one of inter block copy (IBC) mode or intra template matching prediction (TMP) mode.
  3. 3. The video decoding device of claim 1 , wherein the determination to use SBT is performed based on an indication, wherein the indication is signaled at a coding unit (CU) level.
  4. 4. The video decoding device of claim 1 , wherein the SBT information associated with the subblock is obtained based on the coding mode information associated with the coding block, and wherein the SBT information indicates one or more of an SBT split orientation or an SBT mode.
  5. 5. The video decoding device of claim 1 , wherein the processor is further configured to: obtain an indication that indicates an intra TMP index; and determine an intra TMP prediction candidate based on the intra TMP index, wherein the SBT information is obtained based on the determined intra TMP prediction candidate.
  6. 6. The video decoding device of claim 1 , wherein the determination to use SBT on the subblock associated with the coding block is based on a template-based distortion analysis, wherein the templatebased distortion analysis comprises a determination of a template matching cost associated with a neighboring region, and a determination of whether the determined template matching cost associated with the neighboring region is below a threshold, wherein the SBT information associated with the subblock is obtained based on the determination whether the determined template matching cost associated with the neighboring region is below the threshold.
  7. 7. A video encoding device, comprising: a processor configured to: obtain coding mode information associated with a coding block, wherein the coding mode information indicates to use a block vector based prediction mode; determine to use subblock transform (SBT) on a subblock associated with the coding block; based on the determination to use SBT on the subblock, obtain SBT information associated with the subblock; perform a first encoding of the coding block based on the coding mode information; perform a second encoding of the coding block based on the SBT information; and send video data comprising the SBT information and the coding mode information.
  8. 8. The video encoding device of claim 7, wherein the block vector based prediction mode is at least one of inter block copy (IBC) mode or intra template matching prediction (TMP) mode.
  9. 9. The video encoding device of claim 7, wherein the SBT information associated with the subblock is obtained based on the coding mode information associated with the coding block, and wherein the SBT information indicates one or more of an SBT split orientation or an SBT mode.
  10. 10. The video encoding device of claim 7, wherein the processor is further configured to: determine an intra template matching prediction (TMP) prediction candidate based on a TMP index, wherein the SBT information is obtained based on the determined intra TMP prediction candidate.
  11. 11. The video encoding device of claim 7, wherein the determination to use SBT on the subblock associated with the coding block is based on a template-based distortion analysis, wherein the templatebased distortion analysis comprises a determination of a template matching cost associated with a neighboring region, and a determination of whether the determined template matching cost associated with the neighboring region is below a threshold, wherein the SBT information associated with the subblock is obtained based on the determination whether the determined template matching cost associated with the neighboring region is below the threshold.
  12. 12. A video decoding method, the method comprising: obtaining coding mode information associated with a coding block, wherein the coding mode information indicates to use a block vector based prediction mode; determining to use subblock transform (SBT) on a subblock associated with the coding block; based on the determination to use SBT on the subblock, obtain SBT information associated with the subblock; performing a first decoding of the coding block based on the coding mode information; performing a second decoding of the coding block based on the SBT information; and reconstructing the coding block based on the performed first decoding and the second decoding.
  13. 13. The video decoding method of claim 12, wherein the block vector based prediction mode is at least one of inter block copy (IBC) mode or intra template matching prediction (TMP) mode.
  14. 14. The video decoding method of claim 12, wherein the determination to use SBT is performed based on an indication, wherein the indication is signaled at a coding unit (CU) level.
  15. 15. The video decoding method of claim 12, wherein the SBT information associated with the subblock is obtained based on the coding mode information associated with the coding block, and wherein the SBT information indicates one or more of an SBT split orientation or an SBT mode.
  16. 16. The video decoding method of claim 12, wherein the video decoding method further comprises: obtaining an indication that indicates an intra TMP index; and determining an intra TMP prediction candidate based on the intra TMP index, wherein the SBT information is obtained based on the determined intra TMP prediction candidate.
  17. 17. The video decoding method of claim 12, wherein the determination to use SBT on the subblock associated with the coding block is based on a template-based distortion analysis, wherein the templatebased distortion analysis comprises a determination of a template matching cost associated with a neighboring region, and a determination of whether the determined template matching cost associated with the neighboring region is below a threshold, wherein the SBT information associated with the subblock is obtained based on the determination whether the determined template matching cost associated with the neighboring region is below the threshold.
  18. 18. A video encoding method, the video encoding method comprising: obtaining coding mode information associated with a coding block, wherein the coding mode information indicates to use a block vector based prediction mode; determining to use subblock transform (SBT) on a subblock associated with the coding block; based on the determination to use SBT on the subblock, obtaining SBT information associated with the subblock; performing a first encoding of the coding block based on the coding mode information; performing a second encoding of the coding block based on the SBT information; and sending video data comprising the SBT information and the coding mode information.
  19. 19. The video encoding method of claim 18, wherein the block vector based prediction mode is at least one of inter block copy (IBC) mode or intra template matching prediction (TMP) mode.
  20. 20. The video encoding method of claim 18, wherein the SBT information associated with the subblock is obtained based on the coding mode information associated with the coding block, and wherein the SBT information indicates one or more of an SBT split orientation or an SBT mode.

Description

SBT APPLIED TO IBC AND INTRATMP CROSS-REFERENCE TO RELATED APPLICATOINS [0001] The application claims the benefit of European Patent Application Number 23306082.1 , filed June 30, 2023, the contents of which are incorporated by reference in their entirety herein. BACKGROUND [0002] Video coding systems may be used to compress digital video signals, e.g., to reduce the storage and/or transmission bandwidth needed for such signals. Video coding systems may include, for example, block-based, wavelet-based, and/or object-based systems. SUMMARY [0003] Systems, methods, and instrumentalities are disclosed for performing subblock transform (SBT) applied to inter block copy (IBC) and intra template matching prediction (TMP). SBT may be performed in combination with IBC and intra TMP. A coding unit predicted by IBC or intra TMP may benefit from performing SBT (e.g., splitting the coding block into a zero spatial sub-part and a non-zero coded sub-part). [0004] A device (e.g., video decoding device, video encoding device) may perform SBT in combination with IBC or intra TMP. For example, the device may determine whether IBC or intra TMP is used for a coding block. Based on a determination that IBC or intra TMP is used for a coding block, the device may determine to use SBT. The device may determine to use SBT on the coding block (e.g., a subblock associated with the coding block), for example, based on an indication (e.g., via a signaling). Based on a determination that IBC or intra TMP is used for a coding block, the device may encode/decode SBT partitioning information associated with the coding block. The device may determine SBT split orientations and/or SBT modes based on the IBC or intra TMP information. For example, the device may derive or configure the SBT split orientations and/or SBT modes based on an intra TMP prediction candidate (e.g., signaled by an intra TMP index). The device may decode/encode the coding block based on the performed SBT (e.g., in addition to the performed IBC or intra TMP). [0005] Systems, methods, and instrumentalities described herein may involve a decoder. In some examples, the systems, methods, and instrumentalities described herein may involve an encoder. In some examples, the systems, methods, and instrumentalities described herein may involve a signal (e.g., from an encoder and/or received by a decoder). A computer-readable medium may include instructions for causing one or more processors to perform methods described herein. A computer program product may include instructions which, when the program is executed by one or more processors, may cause the one or more processors to carry out the methods described herein. [0006] The device may include a video decoding device. The video decoding device may include a processor. The video decoding device may obtain coding mode information associated with a coding block. The coding mode information may indicate to use a block vector based prediction mode. The block vector based prediction mode may include one or more of an inter block copy (IBC) mode, an intra template matching prediction (TMP) mode, and/or the like. The video decoding device may determine to use subblock transform (SBT) on a subblock associated with the coding block. The determination to use SBT may be based on an indication. The indication may be signaled at a coding unit (CU) level. The video decoding device may obtain SBT information associated with the subblock, for example, based on the determination to use SBT on the subblock. The SBT information may be obtained based on the coding mode information associated with the coding block. The SBT information may indicate one or more of an SBT split orientation, an SBT mode, and/or the like. The video decoding device may perform a first decoding of the coding block, for example, based on the coding mode information. The video decoding device may perform a second decoding of the coding block, for example, based on the SBT information. The video decoding device may reconstruct the coding block, for example, based on the performed first decoding and the second decoding. The video decoding device may obtain an indication that indicates an intra TMP index. The video decoding device may determine an intra TMP prediction candidate, for example, based on the intra TMP index. The SBT information may be obtained, for example, based on the intra TMP prediction candidate. For example, the determination to use SBT on the subblock associated with the coding block may be based on a templatebased distortion analysis. A template-based distortion analysis may include a determination of a template matching cost associated with a neighboring region. The template-based distortion analysis may include a determination of whether the determined template matching cost associated with the neighboring region is below a threshold. The SBT information associated with the subblock may be obtained, for example, based on the determination whether the