Search

EP-4740480-A1 - CONTROLLED SELECTION OF INPUT PICTURES FOR NEURAL-NETWORK POST- FILTER

EP4740480A1EP 4740480 A1EP4740480 A1EP 4740480A1EP-4740480-A1

Abstract

An apparatus configured to: encode a plural ity of uncompressed pictures to a plurality of encoded pictures; and provide the plurality of encoded pictures and supplemental enhancement information into a bitstream, wherein the supplemental enhancement information comprises at least one of: a first indication indicative that at least one first picture of a plurality of reconstructed pictures corresponding to the plurality of encoded pictures is to be included in input to a filter, or a second indication indicative that at least one second picture of the plurality of reconstructed pictures corresponding to the plurality of encoded pictures is to be treated as missing when generating the input to the filter.

Inventors

  • HANNUKSELA, MISKA MATIAS
  • Cricrì, Francesco

Assignees

  • Nokia Technologies Oy

Dates

Publication Date
20260513
Application Date
20240530

Claims (20)

  1. CLAIMS What is claimed is: 1. An apparatus comprising: at least one processor; and at least one non-transitory memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: encode a plurality of uncompressed pictures to a plurality of encoded pictures; and provide the plurality of encoded pictures and supplemental enhancement information into a bitstream, wherein the supplemental enhancement information comprises at least one of: a first indication indicative that at least one first picture of a plurality of reconstructed pictures corresponding to the plurality of encoded pictures is to be included in input to a filter, or a second indication indicative that at least one second picture of the plurality of reconstructed pictures corresponding to the plurality of encoded pictures is to be treated as missing when generating the input to the filter.
  2. 2. The apparatus of claim 1, wherein the filter comprises a neural-network post-filter.
  3. 3. The apparatus of claim 1 or 2, wherein the supplemental enhancement information further comprises at least one of: an indication indicative of a starting point, in output order, of a set of the plurality of reconstructed pictures to be included in the input to the filter, or an indication indicative of an end point, in inverse output order, of the set of the plurality of reconstructed pictures to be included in the input to the filter.
  4. 4. The apparatus of any of claims 1 through 3, wherein the at least one second picture precedes the at least one first picture in the plurality of encoded pictures in output order.
  5. 5. The apparatus of claim 4, wherein a first coded layer video sequence includes the at least one first picture, wherein a second coded layer video sequence that precedes the first coded layer video sequence in output order includes the at least one second picture, wherein the supplemental enhancement information is associated with the at least one first picture, and wherein the second indication is indicative that pictures from a preceding coded layer video sequence in output order are to be treated as missing when generating the input to the filter.
  6. 6. The apparatus of any of claims 1 through 3, wherein the at least one second picture follows the at least one first picture in the plurality of encoded pictures in output order.
  7. 7. The apparatus of claim 6, wherein a first coded layer video sequence includes the at least one first picture, wherein a second coded layer video sequence that follows the first coded layer video sequence in output order includes the at least one second picture, wherein the supplemental enhancement information is associated with the at least one first picture, and wherein the second indication is indicative that pictures from a following coded layer video sequence in output order are to be treated as missing when generating the input to the filter.
  8. 8. The apparatus of any of claims 1 through 7, wherein the supplemental enhancement information further comprises an indication indicative of a persistence for the supplemental enhancement information.
  9. 9. The apparatus of any of claims 1 through 8, wherein the supplemental enhancement information further comprises an indication indicative of a number of the plurality of reconstructed pictures to be treated as missing when generating the input to the filter.
  10. 10. The apparatus of any of claims 1 through 9, wherein the supplemental enhancement information further comprises an indication indicative of a number of pictures of the at least one second picture.
  11. 11. The apparatus of any of claims 1 through 10, wherein the supplemental enhancement information further comprises an indication indicative to at least one of: disable the filter for the at least one second picture, or treat the at least one second picture as missing from the bitstream.
  12. 12. The apparatus of any of claims 1 through 11, wherein the supplemental enhancement information further comprises at least one hash value for at least one of the plurality of reconstructed pictures corresponding to the plurality of encoded pictures.
  13. 13. The apparatus of claim 12, wherein the at least one memory stores instructions that, when executed by the at least one processor, cause the apparatus to: determine the at least one hash value from respective reconstructed pictures of the plurality of reconstructed pictures.
  14. 14. The apparatus of any of claims 1 through 13, wherein the at least one memory stores instructions that, when executed by the at least one processor, cause the apparatus to: generate the supplemental enhancement information.
  15. 15. The apparatus of any of claims 1 through 13, wherein the at least one memory stores instructions that, when executed by the at least one processor, cause the apparatus to: include at least one of: the indication indicative that the at least one first picture is to be included in the input to the filter, or the indication indicative that the at least one second picture is to be treated as missing when generating the input to the filter in the supplemental enhancement information.
  16. 16. A method comprising: encoding, with a user equipment, a plurality of uncompressed pictures to a plurality of encoded pictures; and providing the plurality of encoded pictures and supplemental enhancement information into a bitstream, wherein the supplemental enhancement information comprises at least one of: a first indication indicative that at least one first picture of a plurality of reconstructed pictures corresponding to the plurality of encoded pictures is to be included in input to a filter, or a second indication indicative that at least one second picture of the plurality of reconstructed pictures corresponding to the plurality of encoded pictures is to be treated as missing when generating the input to the filter.
  17. 17. The method of claim 16, wherein the filter comprises a neural-network post-filter.
  18. 18. The method of claim 16 or 17, wherein the supplemental enhancement information further comprises at least one of: an indication indicative of a starting point, in output order, of a set of the plurality of reconstructed pictures to be included in the input to the filter, or an indication indicative of an end point, in inverse output order, of the set of the plurality of reconstructed pictures to be included in the input to the filter.
  19. 19. The method of any of claims 16 through 18, wherein the at least one second picture precedes the at least one first picture in the plurality of encoded pictures in output order.
  20. 20. The method of claim 19, wherein a first coded layer video sequence includes the at least one first picture, wherein a second coded layer video sequence that precedes the first coded layer video sequence in output order includes the at least one second picture, wherein the supplemental enhancement information is associated with the at least one first picture, and wherein the second indication is indicative that pictures from a preceding coded layer video sequence in output order are to be treated as missing when generating the input to the filter.

Description

CONTROLLED SELECTION OF INPUT PICTURES FOR NEURAL-NETWORK POST-FILTER TECHNICAL FIELD [0001] The example and non-limiting embodiments relate generally to media processing with neural networks and, more particularly, to the input provided to a neural-network post- filter (NNPF). BACKGROUND [0002] It is known, in encoding of media content, to provide supplemental enhancement information (SEI). SUMMARY [0003] The following summary is merely intended to be illustrative. The summary is not intended to limit the scope of the claims. [0004] In accordance with one aspect, an apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: encode a plurality of uncompressed pictures to a plurality of encoded pictures; and provide the plurality of encoded pictures and supplemental enhancement information into a bitstream, wherein the supplemental enhancement information comprises at least one of: a first indication indicative that at least one first picture of a plurality of reconstructed pictures corresponding to the plurality of encoded pictures is to be included in input to a filter, or a second indication indicative that at least one second picture of the plurality of reconstructed pictures corresponding to the plurality of encoded pictures is to be treated as missing when generating the input to the filter. [0005] In accordance with one aspect, a method comprising: encoding, with a user equipment, a plurality of uncompressed pictures to a plurality of encoded pictures; and providing the plurality of encoded pictures and supplemental enhancement information into a bitstream, wherein the supplemental enhancement information comprises at least one of: a first indication indicative that at least one first picture of a plurality of reconstructed pictures corresponding to the plurality of encoded pictures is to be included in input to a filter, or a second indication indicative that at least one second picture of the plurality of reconstructed pictures corresponding to the plurality of encoded pictures is to be treated as missing when generating the input to the filter. [0006] In accordance with one aspect, an apparatus comprising means for: encoding a plurality of uncompressed pictures to a plurality of encoded pictures; and providing the plurality of encoded pictures and supplemental enhancement information into a bitstream, wherein the supplemental enhancement information comprises at least one of: a first indication indicative that at least one first picture of the plurality of a plurality of reconstructed pictures corresponding to encoded pictures is to be included in input to a filter, or a second indication indicative that at least one second picture of the plurality of reconstructed pictures corresponding to the plurality of encoded pictures is to be treated as missing when generating the input to the filter. [0007] In accordance with one aspect, a non-transitory computer-readable medium comprising program instructions stored thereon for performing at least the following: encoding a plurality of uncompressed pictures to a plurality of encoded pictures; and providing the plurality of encoded pictures and supplemental enhancement information into a bitstream, wherein the supplemental enhancement information comprises at least one of: a first indication indicative that at least one first picture of a plurality of reconstructed pictures corresponding to the plurality of encoded pictures is to be included in input to a filter, or a second indication indicative that at least one second picture of the plurality of reconstructed pictures corresponding to the plurality of encoded pictures is to be treated as missing when generating the input to the filter. [0008] In accordance with one aspect, an apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: encode a plurality of uncompressed pictures to a plurality of encoded pictures; and provide the plurality of encoded pictures and supplemental enhancement information into a bitstream, wherein the supplemental enhancement information comprises at least one of: a first indication indicative that at least one first picture of a plurality of reconstructed pictures corresponding to the plurality of encoded pictures is to be included in input to a filter, wherein the first indication comprises a picture order count delta value, or a second indication indicative that at least one second picture of the plurality of reconstructed pictures corresponding to the plurality of encoded pictures is to be treated as missing when generating the input to the filter. [0009] In accordance with one aspect, a method comprising: encoding, with a user equipment, a plurality of uncompressed pictures to a plurality of encoded pictures; and providing the plurality of