EP-4740461-A1 - TEMPLATE-BASED INTRA MODE DERIVATION WITH DIRECTIONAL SAMPLEWISE FUSION

Abstract

A coder determines, based on template-based intra mode derivation (TIMD) being applied for a block, a plurality of costs of a plurality of intra prediction modes (IPMs) applied to predict a template of the block. TIMD modes are determined based on IPMs from the plurality of IPMs. For each of the TIMD modes: a weight is determined based on a cost of the TIMD mode, and a directionality of the TIMD mode is determined based on comparing a first sub-cost, of the cost, for a first region of the template and a second sub-cost, of the cost, for a second region of the template. A TIMD mode predictor is determined based on a linear combination of the TIMD modes having weights that are adjusted according to the directionalities of the TIMD modes. Based on the TIMD mode predictor, the coder generates a prediction block for coding the current block.

Inventors

• ANDRIVON, Pierre Jean
• BLESTEL, Médéric

Assignees

• Ofinno, LLC

Dates

Publication Date

20260513

Application Date

20240710

Claims (1)

1. Docket No.: 23-2034PCT CLAIMS What is claimed is: 1. A method comprising: determining, based on template-based intra mode derivation (TIMD) being applied for a current block, a plurality of costs of a plurality of intra prediction modes (IPMs) applied to predict a template of the current block; determining TIMD modes based on IPMs from the plurality of IPMs; for each of the TIMD modes: determining a weight based on a cost of the TIMD mode; and determining a directionality of the TIMD mode, based on comparing: a first sub-cost, of the cost, for a first region of the template; and a second sub-cost, of the cost, for a second region of the template; and determining a TIMD mode predictor based on a linear combination of the TIMD modes having weights that are adjusted according to the directionalities of the TIMD modes; and generating, based on the TIMD mode predictor, a prediction block for the current block. 2. The method of claim 1, wherein the IPMs have the lowest costs of the plurality of costs. 3. The method of any one of claims 1-2, wherein each cost for a respective IPM, of the plurality of IPMs, is based on differences between: predicted samples, of the template of the current block, generated from reference samples of a reference of the template and using the IPM; and reconstructed samples of the template; 4. The method of any one of claims 1-3, wherein the cost comprises a Sum of Absolute Transformed Difference (SATD) cost, a Sum of Squared Error (SSE) cost, or a Sum of Absolute Difference (SAD) cost. 5. The method of any one of claims 3-4, wherein the reference of the template comprises reconstructed samples above or to the left of the template. 6. The method of claim 5, wherein the reference of the template comprises a first number of rows of reconstructed samples above the template and a second number of columns of reconstructed samples to the left of the template. 7. The method of any one of claims 1-6, wherein the first region is associated with a first directionality and the second region is associated with a second directionality, and wherein the directionality is selected from at least the first directionality or the second directionality. 8. The method of any one of claims 1-7, wherein the first region comprises reconstructed samples to the left of the current block and the second region comprises reconstructed samples above the current block. 9. The method of any one of claims 7-8, wherein the first directionality is horizontal and the second directionality is vertical. Docket No.: 23-2034PCT 10. The method of any one of claims 7-9, wherein the directionality is further selected from directionalities comprising the first directionality, the second directionality, and a diagonal directionality indicating no direction. 11. The method of any one of claims 7-10, wherein the directionality corresponds to the first region based on the first sub-cost being smaller than the second sub-cost for the second region. 12. The method of claim 11, wherein the directionality corresponds to the first region further based on the first sub- cost being smaller than the second sub-cost multiplied by a scaling factor. 13. The method of claim 12, wherein the scaling factor is less than 1. 14. The method of any one of claims 1-13, wherein the first sub-cost and the second sub-cost are normalized into a first normalized sub-cost and a second normalized sub-cost, respectively, before being compared. 15. The method of claim 14, wherein the first sub-cost is normalized to the first normalized sub-cost based on the number of samples in the first region, and wherein the second sub-cost is normalized to the second normalized sub-cost based on the number of samples in the second region. 16. The method of any one of claims 1-15, wherein the cost is equal to a sum of the first sub-cost and the second sub-cost. 17. The method of any one of claims 1-16, further comprising determining, based on a width and a height of the current block, whether to determine the directionality of each of the TIMD mode. 18. The method of claim 17, wherein the determining the directionality of each of the TIMD mode is based on a size of the current block being greater than or equal to a size threshold, wherein the size is based on the width and the height. 19. The method of any one of claims 17-18, wherein the determining the directionality of each of the TIMD mode is based on: the width of the current block being greater than or equal to a width threshold; and the height of the current block being greater than or equal to a height threshold. 20. The method of any one of claims 1-19, wherein the weight is determined further based on a sum of costs of the TIMD modes. 21. The method of claim 20, wherein the weight for the TIMD mode is inversely proportional to the cost of the TIMD mode. 22. The method of any one of claims 1-21, further comprising: determining a range of weight adjustment for adjusting weights of TIMD modes according to the directionalities. 23. The method of claim 22, wherein the range of weight adjustment is a predetermined value. 24. The method of any one of claims 22-23, wherein the range of weight adjustment is determined based on a size of the current block. 25. The method of any one of claims 22-24, wherein the range of weight adjustment is determined based on a resolution of a picture comprising the current block. Docket No.: 23-2034PCT 26. The method of any one of claims 1-25, wherein the generating the prediction block comprises applying the TIMD mode predictor to reference samples in the template to determine predicted samples in the prediction block. 27. The method of claim 26, wherein the weight for a TIMD mode, in the linear combination, is adjusted based on a position of a reference sample and the directionality of the TIMD mode. 28. The method of any one of claims 1-27, wherein the TIMD modes comprise a first TIMD mode and a second TIMD mode that are based on a first IPM and a second IPM, respectively, that have the lowest costs of the plurality of costs. 29. The method of claim 28, wherein the determining the first TIMD mode based on the first IPM comprises: selecting the first TIMD mode from the first IPM and first adjacent IPMs, of the IPM, in a second plurality of IPMs different than the plurality of IPMs, wherein the selecting is based on costs computed for the first IPM and the two adjacent IPMs. 30. The method of claim 29, wherein the determining the second TIMD mode based on the second IPM comprises: selecting the second TIMD mode from the second IPM and second adjacent IPMs, of the second IPM, in the second plurality of IPMs, wherein the selecting is based on costs computed for the second IPM and the second adjacent IPMs. 31. The method of any one of claims 1-30, further comprising receiving, from a bitstream, a first indication that indicates TIMD is applied for the current block, wherein the determining the plurality of costs is based on the first indication. 32. The method of any one of claims 1-31, further comprising receiving, from a bitstream, a second indication that indicates location-dependency weighting is applied to the TIMD modes, wherein the determining the directionality of each of the TIMD mode is further based on the second indication. 33. The method of any one of claims 1-32, further comprising: decoding, from the bitstream, a residual of the current block; and reconstructing the current block based on the prediction block and the residual. 34. The method of claim 33, wherein the reconstructing comprises combining the prediction block with the residual. 35. The method of any one of claims 1-30, further comprising signaling, in a bitstream, a first indication that indicates TIMD is applied for the current block, wherein the determining the plurality of costs is based on the first indication. 36. The method of any one of claims 1-30 or 35, further comprising signaling, in a bitstream, a second indication that indicates location-dependency weighting is applied to the TIMD modes, wherein the determining the directionality of each of the TIMD mode is further based on the second indication. 37. The method of any one of claims 1-30 or 35-36, further comprising: determining a residual based on the prediction block and the current block; and encoding the residual in the bitstream. 38. The method of claim 37, wherein the residual is determined as a difference between the current block and the prediction block. Docket No.: 23-2034PCT 39. An encoder comprising: one or more processors; and memory storing instructions that, when executed by the one or more processors, cause the encoder to perform the method of any one of claims 1-30 or 35-38. 40. A decoder comprising: one or more processors; and memory storing instructions that, when executed by the one or more processors, cause the decoder to perform the method of any one of claims 1-34. 41. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors of an apparatus, cause the apparatus to perform the method of any one of claims 1-38.

Description

Docket No.: 23-2034PCT TITLE Template-based Intra Mode Derivation with Directional Samplewise Fusion CROSS-REFERENCE TO RELATED APPLICATIONS [0001] This application claims the benefits of U.S. Provisional Application No.63/525,936, filed July 10, 2023, and U.S. Provisional Application No.63/542,931, filed October 6, 2023, all of which are hereby incorporated by reference in their entireties. BRIEF DESCRIPTION OF THE DRAWINGS [0002] Some features are shown by way of example, and not by limitation, in the accompanying drawings. In the drawings, like numerals reference similar elements. [0003] FIG.1 shows an example video coding/decoding system in which embodiments of the present disclosure may be implemented. [0004] FIG.2 shows an example encoder in which embodiments of the present disclosure may be implemented. [0005] FIG.3 shows an example decoder in which embodiments of the present disclosure may be implemented. [0006] FIG.4 shows an example quadtree partitioning of a coding tree block (CTB). [0007] FIG.5 shows an example quadtree corresponding to the example quadtree partitioning of the CTB in FIG.4. [0008] FIG.6 show examples of binary tree and ternary tree partitions. [0009] FIG.7 shows an example of combined quadtree and multi-type tree partitioning of a CTB. [0010] FIG.8 shows an example tree corresponding to the combined quadtree and multi-type tree partitioning of the CTB shown in FIG.7. [0011] FIG.9 shows an example set of reference samples determined for intra prediction of a current block. [0012] FIGS.10A and 10B show example intra prediction modes. [0013] FIG.11 shows an example of a current block and corresponding reference samples. [0014] FIG.12 shows an example of applying an intra prediction mode (e.g., an angular mode) for prediction of a current block. [0015] FIG.13A shows an example of inter prediction performed for a current block in a current picture. [0016] FIG.13B shows an example motion vector. [0017] FIG.14 shows an example of bi-prediction performed for a current block. [0018] FIG.15A shows example spatial candidate neighboring blocks relative to a current block being coded. [0019] FIG.15B shows example locations of two temporal, co-located blocks relative to a current block. [0020] FIG.16 shows an example of intra block copy (IBC). [0021] FIG.17A shows an example of decoder-side intra mode derivation (DIMD) for coding a current block, according to some embodiments. [0022] FIG.17B shows an example of template-based intra mode derivation (TIMD) for coding a current block, according to some embodiments. [0023] FIG.18 shows an example of signaling TIMD for decoding a current block, according to some embodiments. Docket No.: 23-2034PCT [0024] FIG.19A shows a flowchart of an example method for applying a TIMD technique with directional samplewise fusion for a current block, according to some embodiments. [0025] FIG.19B shows a flowchart of an example method for applying a TIMD technique with directional samplewise fusion for a current block, according to some embodiments. [0026] FIG.20 shows a flowchart of an example method for applying a TIMD technique with directional samplewise fusion for a current block, according to some embodiments. [0027] FIG.21 shows a flowchart of an example method for applying a TIMD technique with directional samplewise fusion for a current block, according to some embodiments. [0028] FIG.22 shows a block diagram of an example computer system in which embodiments of the present disclosure may be implemented. DETAILED DESCRIPTION [0029] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the disclosure. However, it will be apparent to those skilled in the art that the disclosure, including structures, systems, and methods, may be practiced without these specific details. The description and representation herein are the common means used by those experienced or skilled in the art to most effectively convey the substance of their work to others skilled in the art. In other instances, well-known methods, procedures, components, and circuitry have not been described in detail to avoid unnecessarily obscuring aspects of the disclosure. [0030] References in the specification to “one embodiment,” “an embodiment,” “an example embodiment,” etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described. [0031] Also, it is noted that individual embodiments may be described as