Search

EP-4740479-A1 - SIGNALING OF PROCESSING ORDER FOR METADATA MESSAGING IN VIDEO CODING

EP4740479A1EP 4740479 A1EP4740479 A1EP 4740479A1EP-4740479-A1

Abstract

Methods, systems, and bitstream syntax are described for determining a preferred processing order of metadata messaging, such as supplemental enhancement information (SEI) messaging in MPEG video coding. Examples are provided to address issues related to backwards compatibility with legacy systems. For example, proposed messaging allows encoders to isolate messages critical to a decoder's implementation and assign importance to each SEI message so that backwards compatibility is preserved.

Inventors

  • SULLIVAN, GARY J.

Assignees

  • Dolby Laboratories Licensing Corporation

Dates

Publication Date
20260513
Application Date
20240703

Claims (13)

  1. 1. A method to determine processing order among multiple metadata messages, the method comprising: receiving an input video bitstream and corresponding input metadata messages defining supplemental processing to be applied to decoded video pictures of the input video bitstream; parsing the input metadata messages to identify a processing order of metadata messaging (POM) message defining a preferred processing order of selected input metadata messages; and upon detecting the POM message (305): for an i-th input metadata message among the selected input metadata messages: detecting (310) whether a wrapping flag is set to 1 for the i-th input metadata message, and if the wrapping flag is set to 1, then reading messaging information for the i-th input metadata message; and determining a preferred processing order for the i-th input metadata message (320).
  2. 2. A method to determine processing order among multiple metadata messages, the method comprising: receiving an input video bitstream and corresponding input metadata messages defining supplemental processing to be applied to decoded video pictures of the input video bitstream; parsing the input metadata messages to identify a processing order of metadata messaging (POM) message defining a preferred processing order of selected input metadata messages; and upon detecting the POM message (305): for an i-th input metadata message among the selected input metadata messages: reading an importance flag for the i-th input metadata message, and if the importance flag classifies the i-th input metadata message as important and a decoder does not recognize or support it, then the decoder skips processing the POM message and ignores any identified processing order in the POM message.
  3. 3. A method to determine processing order among multiple metadata messages, the method comprising: receiving an input video bitstream and corresponding input metadata messages defining supplemental processing to be applied to decoded video pictures of the input video bitstream; parsing the input metadata messages to identify a processing order of metadata messaging (POM) message defining a preferred processing order of selected input metadata messages; and upon detecting the POM message (305): for an i-th input metadata message among the selected input metadata messages: detecting (335) whether an importance flag is set to 1 for the i-th input metadata message; detecting (310) whether a wrapping flag is set to 1 for the i-th input metadata message, and if the wrapping flag is set to 1, then reading messaging information for the i-th input metadata message; and determining a preferred processing order for the i-th input metadata message (320).
  4. 4. The method of claim 1 or claim 3, further comprising: for a decoded video picture of the input video bitstream, applying the selected input metadata messages with their preferred processing order.
  5. 5. The method of claim 1 or claim 3, further comprising: if the wrapping flag for the i-th input metadata message is set to 1, then if its preferred processing order (po_sei_processing_order[i]) is not 0, then input for the i-th input metadata message is intended to be output of the input metadata message which has the preferred processing order equal to po_sei_processing_order[ i ] - 1.
  6. 6. The method of claim 3, further comprising, if a decoder cannot interpret or does not support any indicated input metadata message with its importance flag set to important, then the determined preferred processing order is ignored for all the selected input metadata messages.
  7. 7. A method to determine processing order among multiple metadata messages, the method comprising: receiving an input video bitstream and corresponding input metadata messages defining supplemental processing to be applied to decoded video pictures of the input video bitstream, wherein the input metadata messages comprise a first class of input metadata messages for a first class of decoders and a second class of input metadata for a second class of decoders; parsing the input metadata messages to identify a processing order of metadata messaging (POM) message defining a preferred processing order of selected input metadata messages; and upon detecting the POM message: if a decoder identifies itself as one of the first class of decoders, then ignoring the POM message and all input metadata messages defined in the POM message, else, if the decoder identifies itself as one of the second class of decoders, then processing the POM message to determine a preferred processing order for selected input metadata messages in the POM message.
  8. 8. The method of claim 7, wherein the first class of decoders represents a legacy decoder.
  9. 9. The method of claim 7, wherein processing the POM message to determine a preferred processing order further comprises: reading a first processing-order parameter indicating a total number of the selected input metadata messages; for an i-th input metadata message among the selected input metadata messages: reading a preferred processing order for the i-th input metadata message; and reading messaging information for the i-th input metadata message; reading a second processing-order parameter indicating a total number of property indications present that apply after completing processing the selected input metadata messages; and reading messaging information for each of the input metadata messages specified by the second processing-order parameter.
  10. 10. The method of claim 7, wherein processing the POM message to determine a preferred processing order further comprises: reading a first processing-order parameter indicating a total number of selected input metadata messages; for an i-th input metadata message among the selected input metadata messages: reading messaging information for the i-th input metadata message; reading a processing order for the i-th input metadata message; reading a second processing-order parameter indicating a total number of other input metadata messages with property indications present that apply after completing processing the i-th input metadata message; and reading messaging information for each of the other input metadata messages specified by the second processing-order parameter.
  11. 11. The method of any one of claims 1-10, wherein the input metadata messages comprise supplemental enhancement information (SEI) messages or video user information (VUI) messages.
  12. 12. A non-transitory computer-readable storage medium having stored thereon computerexecutable instructions for executing with one or more processors a method in accordance with any one of the claims 1-11.
  13. 13. An apparatus comprising a processor and configured to perform any one of the methods recited in claims 1-11.

Description

SIGNALING OF PROCESSING ORDER FOR METADATA MESSAGING IN VIDEO CODING CROSS-REFERENCE TO RELATED APPLICATIONS [0001] This patent application claims the benefit of priority to U.S. Provisional patent application Ser. No. 63/588,188, filed on 05 October 2023, and U.S. Provisional patent application Ser. No. 63/511,800, filed on 03 July 2023, each of which is incorporated by reference in its entirety. TECHNOLOGY [0002] The present document relates generally to images and video. More particularly, an embodiment of the present invention relates to signaling a processing order for metadata messaging in images and video sequences. BACKGROUND [0003] As described in Annex D of the AVC and HEVC standards, or in H.274 (also referred to as VSEI) (Ref. [1-4]), Supplemental Enhancement Information (SEI) messages in a coded video bitstream assist in processes related to decoding, display, or other purposes in a video processing pipeline. Despite their extensive use, at least up to now, conforming decoders (e.g., an AVC, HEVC, or VVC decoder ) are not required to process any SEI messaging to comply with the picture decoding process specifications of any of the MPEG video coding standards (such as AVC, HEVC, and VVC), although such messages may be used for system control functionality such as indicating the buffering and timing information for the video bitstream. [0004] In a typical video bitstream, multiple SEI messages may co-exist; however, none of the existing video coding standards define the processing order of such messaging. For some SEI messages, outside of information embedded in syntax elements, there is no specific processing defined in the standards. Examples of such messages include SEI messaging defining a mastering display color volume or content light-level information. For some other SEI messages, such as those describing film grain characteristics (FGC) or color remapping information, in addition to the syntax elements, additional post-processing may also be defined. For the later, the final video output may vary depending on the processing order of these SEI messages. As appreciated by the inventors, improved techniques for signaling the processing order of SEI messaging are described herein. [0005] The term “metadata” herein relates to any auxiliary information transmitted either as part of the coded bitstream or along with it that assists a decoder to render or interpret one or more decoded images. Such metadata may include, but are not limited to, color space or gamut information, reference display parameters, and film grain modeling parameters, as those described herein. While examples presented herein may refer to SEI messaging as it relates to MPEG-based video coding standards, a person of ordinary skill would appreciate that the techniques discussed herein are applicable to any such metadata messaging and any audio or video coding format (e.g., AVI, AVS, VC-1 and the like). [0006] The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section. Similarly, issues identified with respect to one or more approaches should not assume to have been recognized in any prior art on the basis of this section, unless otherwise indicated. BRIEF DESCRIPTION OF THE DRAWINGS [0007] An embodiment of the present invention is illustrated by way of example, and not in way by limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar elements and in which: [0008] FIG. 1A and FIG IB depict examples of video processing pipelines when metadata includes multiple SEI messages and their post-processing order may affect video output; [0009] FIG. 2 depicts an example processing pipeline, according to prior art, when an SEI Priority (or Processing) Order of Messaging (POM) message is available in a decoder; [00010] FIG. 3A depicts an example processing pipeline with an SEI processing order message using an additional wrapper loop, according to an embodiment of this invention; and [00011] FIG. 3B depicts an example processing pipeline with an SEI processing order message using a wrapper loop and importance flags, according to an embodiment of this invention. DESCRIPTION OF EXAMPLE EMBODIMENTS [00012] Example embodiments that relate to a processing order of metadata are described herein. In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the various embodiments of present invention. It will be apparent, however, that the various embodiments of the present invention may be practiced without these specific details. In other instances, well- known structures and devices are not described in exhaustive