Search

EP-4736419-A1 - INTRA TMP AND LIC COMBINATION

EP4736419A1EP 4736419 A1EP4736419 A1EP 4736419A1EP-4736419-A1

Abstract

Systems, methods, and instrumentalities are disclosed for performing intra template matching prediction (TMP) and local illumination compensation (LIC) in combination. LIC may be used in combination with intra TMP. For example, using LIC (e.g., at the CU level) may be adaptively selected for an intra TMP predicted coding block. LIC may be considered (e.g., used) when performing an intra TMP prediction block search. LIC may be used on some (e.g., not all) intra TMP candidates. Whether to use LIC may be signaled. Performing LIC with intra TMP may be mutually exclusive from performing intra TMP with linear filter model (FLM) mode.

Inventors

  • LE LEANNEC, FABRICE
  • NASER, Karam
  • RATH, Gagan Bihari
  • DUMAS, Thierry

Assignees

  • InterDigital CE Patent Holdings, SAS

Dates

Publication Date
20260506
Application Date
20240624

Claims (20)

  1. 1 . A video decoding device, comprising: a processor configured to: obtain an intra prediction mode indication associated with a coding block; determine that intra template matching prediction (TMP) is used for coding the coding block based on the intra prediction mode indication; determine, based on the determination that intra TMP is used for coding the coding block, whether LIC is enabled for the coding block; and decode the coding block using intra TMP, wherein if LIC is enabled for the coding block, the decoding of the coding block comprises a determination of a prediction block using LIC.
  2. 2. The video decoding device of claim 1 , wherein the processor is further configured to: obtain a coding unit level indication that indicates whether LIC is used jointly with intra TMP for the coding block, wherein the determination whether LIC is enabled for the coding block is further based on the obtained coding unit level indication; and based on a determination that LIC is enabled for the coding block, determine an LIC parameter based on a template associated with the coding block, wherein the determination of the prediction block using LIC is based on the determined LIC parameter.
  3. 3. The video decoding device of claim 2, wherein the processor is further configured to: determine an intra TMP prediction candidate, wherein, based on the determination that LIC is enabled for the coding block, the determination of the intra TMP prediction candidate takes LIC into consideration, wherein the determination of the LIC parameter based on the template associated with the coding block is further based on a template area associated with the intra TMP prediction candidate and a template area around the coding block.
  4. 4. The video decoding device of claim 1 , wherein if LIC is not enabled for the coding block, the decoding of the coding block comprises a determination of whether linear filter model (FLM) is used for the coding block, wherein the processor is further configured to: based on a determination that FLM is used for the coding block, determine an FLM filter based on a template associated with the coding block; and apply the FLM filter to generate a predicted block, wherein the coding block is further decoded based on the generated predicted block.
  5. 5. The video decoding device of claim 1 , wherein the determination of whether LIC is enabled for the coding block is further based on LIC information, wherein the LIC information is included in block vector information associated with a previously decoded coding block in a same picture as the coding block, wherein the coding block and the previously decoded coding block is associated with an intra-block copy mode.
  6. 6. The video decoding device of claim 1 , wherein the processor is further configured to: obtain an intra TMP index; determine a first intra TMP prediction candidate and a second intra TMP prediction candidate based on the intra TMP index, wherein the intra TMP index indicates that the first intra TMP prediction candidate has LIC enabled and the second intra TMP prediction candidate does not have LIC enabled; and determine a predicted block based on the first intra TMP prediction candidate and the second intra TMP prediction candidate.
  7. 7. A video encoding device, comprising: a processor configured to: determine to use intra template matching prediction (TMP) for a coding block; determine whether local illumination compensation (LIC) is enabled for the coding block, wherein the determination whether LIC is enabled is for the coding block is based at least on the determination to use intra TMP for the coding block; encode the coding block using intra TMP, wherein if LIC is enabled for the coding block, the encoding of the coding block comprises a determination of a prediction block using LIC; and include in video data an indication that indicates to use intra TMP for the coding block, wherein the indication further indicates that LIC is enabled for the coding block if LIC is enabled for the coding block.
  8. 8. The video encoding device of claim 7, wherein the processor is further configured to: based on a determination that LIC is enabled for the coding block, determine an LIC parameter based on a template associated with the coding block, wherein the determination of the prediction block using LIC is based on the determined LIC parameter, wherein the video data further includes a coding unit level indication that indicates whether LIC is used jointly with intra TMP for the coding block.
  9. 9. The video encoding device of claim 8, wherein the processor is further configured to: determine an intra TMP prediction candidate, wherein, based on the determination that LIC is enabled for the coding block, the determination of the intra TMP prediction candidate takes LIC into consideration, wherein the determination of the LIC parameter based on the template associated with the coding block is further based on a template area associated with the intra TMP prediction candidate and a template area around the coding block.
  10. 10. The video encoding device of claim 7, wherein if LIC is not enabled for the coding block, the encoding of the coding block comprises a determination of whether linear filter model (FLM) is used for the coding block, wherein the processor is further configured to: based on a determination that FLM is used for the coding block, determine an FLM filter based on a template associated with the coding block; and apply the FLM filter to generate a predicted block, wherein the coding block is further decoded based on the generated predicted block.
  11. 11 . The video encoding device of claim 7, wherein the determination of whether LIC is enabled for the coding block is further based on LIC information, wherein the LIC information is included in block vector information associated with a previously encoded coding block in a same picture as the coding block, wherein the coding block and the previously encoded coding block is associated with an intra-block copy mode.
  12. 12. The video encoding device of claim 7, wherein the processor is further configured to: obtain an intra TMP index; determine a first intra TMP prediction candidate and a second intra TMP prediction candidate based on the intra TMP index, wherein the intra TMP index indicates that the first intra TMP prediction candidate has LIC enabled and the second intra TMP prediction candidate does not have LIC enabled; and determine a predicted block based on the first intra TMP prediction candidate and the second intra TMP prediction candidate.
  13. 13. A video decoding method, the video decoding method comprising: obtaining an intra prediction mode indication associated with a coding block; determining that intra template matching prediction (TMP) is used for coding the coding block based on the intra prediction mode indication; determining, based on the determination that intra TMP is used for coding the coding block, whether LIC is enabled for the coding block; and decoding the coding block using intra TMP, wherein if LIC is enabled for the coding block, the decoding of the coding block comprises a determination of a prediction block using LIC.
  14. 14. The video decoding method of claim 13, wherein the video decoding method further comprises: obtaining a coding unit level indication that indicates whether LIC is used jointly with intra TMP for the coding block, wherein the determination whether LIC is enabled for the coding block is further based on the obtained coding unit level indication; and based on a determination that LIC is enabled for the coding block, determining an LIC parameter based on a template associated with the coding block, wherein the determination of the prediction block using LIC is based on the determined LIC parameter.
  15. 15. The video decoding method of claim 14, wherein the video decoding method further comprises: determining an intra TMP prediction candidate, wherein, based on the determination that LIC is enabled for the coding block, the determination of the intra TMP prediction candidate takes LIC into consideration, wherein the determination of the LIC parameter based on the template associated with the coding block is further based on a template area associated with the intra TMP prediction candidate and a template area around the coding block.
  16. 16. The video decoding method of claim 13, wherein if LIC is not enabled for the coding block, the decoding of the coding block comprises a determination of whether linear filter model (FLM) is used for the coding block, wherein the video decoding method further comprises: based on a determination that FLM is used for the coding block, determining an FLM filter based on a template associated with the coding block; and applying the FLM filter to generate a predicted block, wherein the coding block is further decoded based on the generated predicted block.
  17. 17. The video decoding method of claim 13, wherein the determination of whether LIC is enabled for the coding block is further based on LIC information, wherein the LIC information is included in block vector information associated with a previously decoded coding block in a same picture as the coding block, wherein the coding block and the previously decoded coding block is associated with an intra-block copy mode.
  18. 18. The video decoding method of claim 13, wherein the video decoding method further comprises: obtaining an intra TMP index; determining a first intra TMP prediction candidate and a second intra TMP prediction candidate based on the intra TMP index, wherein the intra TMP index indicates that the first intra TMP prediction candidate has LIC enabled and the second intra TMP prediction candidate does not have LIC enabled; and determining a predicted block based on the first intra TMP prediction candidate and the second intra TMP prediction candidate.
  19. 19. A video encoding method, the video encoding method comprising: determining to use intra template matching prediction (TMP) for a coding block; determining whether local illumination compensation (LIC) is enabled for the coding block, wherein the determination whether LIC is enabled is for the coding block is based at least on the determination to use intra TMP for the coding block; encoding the coding block using intra TMP, wherein if LIC is enabled for the coding block, the encoding of the coding block comprises a determination of a prediction block using LIC; and including in video data an indication that indicates to use intra TMP for the coding block, wherein the indication further indicates that LIC is enabled for the coding block if LIC is enabled for the coding block.
  20. 20. The video encoding method of claim 19, wherein the video encoding method further comprises: based on a determination that LIC is enabled for the coding block, determining an LIC parameter based on a template associated with the coding block, wherein the determination of the prediction block using LIC is based on the determined LIC parameter, wherein the video data further includes a coding unit level indication that indicates whether LIC is used jointly with intra TMP for the coding block.

Description

INTRA TMP AND LIC COMBINATION CROSS-REFERENCE TO RELATED APPLICATOINS [0001] The application claims the benefit of European Patent Application Number 23306084.7, filed June 30, 2023 and European Patent Application Number 23307092.9, filed November 30, 2023, the contents of which are incorporated by reference in their entirety herein. BACKGROUND [0002] Video coding systems may be used to compress digital video signals, e.g., to reduce the storage and/or transmission bandwidth needed for such signals. Video coding systems may include, for example, block-based, wavelet-based, and/or object-based systems. SUMMARY [0003] Systems, methods, and instrumentalities are disclosed for performing intra template matching prediction (TMP) and local illumination compensation (LIC) in combination. LIC may be used in combination with intra TMP. For example, using LIC (e.g., at the CU level) may be adaptively selected for an intra TMP predicted coding block. LIC may be considered (e.g., used) when performing an intra TMP prediction block search. LIC may be used on some (e.g., not all) intra TMP candidates. Whether to use LIC may be signaled. Performing LIC with intra TMP may be mutually exclusive from performing intra TMP with linear filter model (FLM) mode. [0004] A device (e.g., decoding device, encoding device, etc.) may be used to perform intra TMP in combination with LIC (e.g., and/or FLM). A device may obtain an intra prediction mode indication associated with a coding block. The device may determine an intra TMP prediction candidate. The intra prediction mode may indicate to use intra TMP for a coding block. The device may obtain an LIC indication associated with the coding block. The LIC indication may indicate whether LIC is enabled for a coding block. The device may determine that LIC is used for a coding block. Based on the determination that LIC is used for a coding block, the device may determine LIC parameters (e.g., based on a template, such as a template area associated with an intra TMP prediction candidate and a template area around the coding block) and the device may determine a predicted block based on the LIC parameters. Based on a determination that LIC is used for a coding block, the intra TMP prediction candidate may be determined based on taking LIC into consideration. The predicted block may be used to decode a residual associated with the coding block. [0005] The device may determine that LIC is not used for a coding block. Based on the determination that LIC is not used for the coding block, the device may determine whether FLM is used for the coding block (e.g., based on parsing the coding block). The device, based on a determination that FLM is used for the coding block, may determine an FLM filter based on a template associated with the coding block. The device may apply the FLM filter to generate a predicted block. The predicted block may be used to decode a residual associated with the coding block. [0006] The device may determine whether to use TMP without LIC or FLM, use TMP with LIC but without FLM, or use TMP with FLM but without LIC. The device may determine which coding mode to use based on costs associated with the respective coding modes. For example, the device may determine a cost associated with each coding mode. The coding mode selected may be the coding mode associated with the lowest cost. [0007] Intra Block Copy (IBC) may be used jointly with LIC. Joint IBC-LIC coding modes may be used. For example, joint IBC-LIC coding modes may include multi-model IntraTM P-LIC, top-only IntraTM P-LIC, or left-only IntraTM P-LIC. A device (e.g., decoder or encoder) may determine that IBC is used jointly with LIC for a coding block. The determination that joint IBC-LIC is used may be based on a received index and/or index value. Parameters associated with joint IBC-LIC coding modes may be determined. [0008] Slope adjustment may be used for CUs. Slope adjustment may introduce adaptivity in linear models supported by IBC-LIC. Signaling may indicate whether slope adjustment is used. For example, slope adjustment may be indicated by a CU-level indication (e.g., flag. [0009] Systems, methods, and instrumentalities described herein may involve a decoder. In some examples, the systems, methods, and instrumentalities described herein may involve an encoder. In some examples, the systems, methods, and instrumentalities described herein may involve a signal (e.g., from an encoder and/or received by a decoder). A computer-readable medium may include instructions for causing one or more processors to perform methods described herein. A computer program product may include instructions which, when the program is executed by one or more processors, may cause the one or more processors to carry out the methods described herein. [0010] The device may include a video decoding device. The video decoding device may include a processor. The video decoding device may obtain an intra prediction mode indication as