Search

CN-121986476-A - Post-processing filtering process based on post-processing filter bank

CN121986476ACN 121986476 ACN121986476 ACN 121986476ACN-121986476-A

Abstract

A mechanism for processing video data is disclosed. The mechanism includes determining to obtain a list of output pictures from a processing chain, wherein the list of output pictures may include more pictures than output by a last post-processing filter (PPF) in the processing chain. The PPF may be a neural network post-processing filter (NNPF). Conversion between the visual media data and the bitstream is performed based on a processing chain.

Inventors

  • WANG YEKUI
  • JIA WEI
  • XU JIZHENG
  • ZHANG LI

Assignees

  • 字节跳动有限公司

Dates

Publication Date
20260505
Application Date
20241004
Priority Date
20240403

Claims (20)

  1. 1. A method of processing media data, comprising: Determining a list of output pictures obtained from a processing chain, wherein the list of output pictures may include more pictures than output by a last post-processing filter (PPF) in the processing chain, and Conversion between the visual media data and the bitstream is performed based on the processing chain.
  2. 2. The method of claim 1, wherein the list may include cropped decoded pictures that do not have corresponding output pictures output by PPF of any application.
  3. 3. The method of any of claims 1-2, wherein the list may include PPF output pictures that are output by PPFs that are applied to non-last PPFs of a current picture.
  4. 4. A method according to any of claims 1-3, wherein when a picture in the list is replaced, the properties of the picture are also updated as part of the list, and the updated properties include picture width in units of luma samples, picture height in units of luma samples, bit depth for a luma sample array of the picture, bit depth for a chroma sample array of the picture, a chroma format indicator of the picture, or a combination thereof.
  5. 5. The method of any of claims 1-4, wherein when a picture is inserted into the list, the properties of the picture are also updated as part of the list, and the updated properties include a picture width in units of luma samples, a picture height in units of luma samples, a bit depth for a luma sample array of the picture, a bit depth for a chroma sample array of the picture, a chroma format indicator of the picture, or a combination thereof.
  6. 6. The method of any of claims 1-5, wherein after applying PPF, the list is updated by 1) replacing each of the pictures in the list having a corresponding PPF output picture of the PPF with the corresponding PPF output picture, and 2) inserting interpolated or extrapolated pictures (if any) into the list and placing the interpolated or extrapolated pictures such that all pictures in the updated list are arranged in output order.
  7. 7. The method of any of claims 1-6, wherein the list is initialized to include a list of cropped decoded pictures in output order resulting from decoding the bitstream before any PPF is applied, and the list is updated each time PPF is applied.
  8. 8. The method of any of claims 1-7, wherein after applying a PPF, the list is updated by replacing each of the pictures in the list having a corresponding PPF output picture of the PPF with the corresponding PPF output picture.
  9. 9. The method of any of claims 1-8, wherein after application of the PPF, the list is updated by inserting interpolated pictures (if any) into the list and placing the interpolated pictures such that all pictures in the updated list are arranged in output order.
  10. 10. The method of any of claims 1-9, wherein after applying PPF, the list is updated by replacing each of the pictures in the list having a corresponding PPF output picture of the PPF with the corresponding PPF output picture, and inserting interpolated pictures (if any) into the list, and placing the interpolated pictures such that all pictures in the updated list are arranged in output order.
  11. 11. The method of any of claims 1-10, wherein PPF can be indicated by a Supplemental Enhancement Information (SEI) message indicating a post-processing operation and for which a payload type (payloadType) value is included in a list SEI processing order SEI list (SeiProcessingOrderSeiList) specified by a SEI Processing Order (SPO) SEI message.
  12. 12. The method of claims 1-11, wherein the processing chain is applied by decoding a list of cropped decoded pictures in output order, selecting the processing chain, applying each PPF in the processing chain to each cropped decoded picture in the list, and replacing the cropped decoded picture in the list with a processed picture.
  13. 13. The method of any of claims 1-12, wherein the converting comprises encoding the visual media data into the bitstream.
  14. 14. The method of any of claims 1-12, wherein the converting comprises decoding the visual media data from the bitstream.
  15. 15. A non-transitory computer readable recording medium storing a bitstream of video generated by a method performed by a video processing apparatus, wherein the method comprises: Determining a list of output pictures obtained from a processing chain, wherein the list of output pictures may include more pictures than output by a last post-processing filter (PPF) in the processing chain, and A bitstream is generated based on the determination.
  16. 16. The non-transitory computer-readable recording medium of claim 15, wherein the list may include cropped decoded pictures that do not have corresponding output pictures output by the PPF of any application.
  17. 17. The non-transitory computer readable recording medium of any one of claims 15-16, wherein the list may include PPF output pictures that are output by PPFs that are applied to non-last PPFs of a current picture.
  18. 18. A method of storing a bitstream of video, comprising: Determining a list of output pictures from a processing chain, wherein the list of output pictures may include more pictures than output by a last post-processing filter (PPF) in the processing chain; Generating a bit stream based on the determination, and The bit stream is stored in a non-transitory computer readable recording medium.
  19. 19. The method of claim 18, wherein the list may include cropped decoded pictures that do not have corresponding output pictures output by PPF of any application.
  20. 20. The method of any of claims 18-19, wherein the list may include PPF output pictures that are output by PPFs that are applied to non-last PPFs of a current picture.

Description

Post-processing filtering process based on post-processing filter bank Cross Reference to Related Applications The present application claims the priority and benefit of U.S. provisional patent application 63/588,273 filed on 5 th 10 th 2023 and U.S. provisional patent application 63/574,088 filed on 3 th 4 th 2024. All of the foregoing patent applications are incorporated herein by reference in their entirety. Technical Field This patent document relates to the generation, storage and use of digital audio video media information in a file format. Background Digital video occupies the maximum bandwidth used on the internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth requirements for digital video usage may continue to increase. Disclosure of Invention A first aspect relates to a method of processing video data, comprising obtaining a list of output pictures from a processing chain, wherein the list of output pictures may comprise more pictures than output by a last post-processing filter (PPF) in the processing chain, and performing a conversion between visual media data and a bitstream based on the processing chain. A second aspect relates to a non-transitory computer readable recording medium storing a bitstream of video generated by a method performed by a video processing apparatus, wherein the method comprises determining to obtain a list of output pictures from a processing chain, wherein the list of output pictures may include more pictures than output by a last post-processing filter (PPF) in the processing chain, and generating the bitstream based on the determining. A third aspect relates to a method of storing a bitstream of a video, comprising determining to obtain a list of output pictures from a processing chain, wherein the list of output pictures may include more pictures than output by a last post-processing filter (PPF) in the processing chain, generating the bitstream based on the determining, and storing the bitstream in a non-transitory computer readable recording medium. A fourth aspect relates to an apparatus for processing video data, comprising a processor, and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform any of the above aspects. A fifth aspect relates to a non-transitory computer readable medium comprising a computer program product for use by a video codec device, the computer program product comprising computer executable instructions stored on the non-transitory computer readable medium such that when executed by a processor cause the video codec device to perform the method of any of the above aspects. For clarity, any of the above embodiments may be combined with any one or more of the other above embodiments to create new embodiments within the scope of the present disclosure. These and other features will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings and claims. Drawings For a more complete understanding of the present disclosure, reference is now made to the following brief description, taken in connection with the accompanying drawings and detailed description, wherein like reference numerals represent like parts. Fig. 1 is a block diagram illustrating an example video processing system. Fig. 2 is a block diagram of an example video processing apparatus. Fig. 3 is a flow chart of an example method of video processing. Fig. 4 is a block diagram illustrating an example video codec system. Fig. 5 is a block diagram illustrating an example encoder. Fig. 6 is a block diagram illustrating an example decoder. Fig. 7 is a schematic diagram of an example encoder. Fig. 8 is a flow chart of an example method of video processing. Detailed Description It should be understood at the outset that although an illustrative implementation of one or more embodiments are provided below, the disclosed systems and/or methods may be implemented using any number of techniques, whether currently known or in existence of development. The disclosure should in no way be limited to the illustrative embodiments, figures, and techniques illustrated below, including the exemplary designs and embodiments shown and described herein, but may be modified within the scope of the appended claims along with their full scope of equivalents. The section headings are used in this document for ease of understanding, and are not intended to limit the applicability of the techniques and embodiments disclosed in each section to that section only. Furthermore, the H.266 term is used in some descriptions merely for ease of understanding and is not intended to limit the scope of the disclosed technology. Thus, the techniques described herein are also applicable to other video codec protocols and designs. In this document, edit changes are shown textua