US-12621470-B2 - Techniques for generation of a conformant output sub-bitstream

Abstract

Examples of video encoding methods and apparatus and video decoding methods and apparatus are described. An example method of video processing includes performing a conversion between a video including multiple layers and a bitstream of the video according to a rule, wherein the rule specifies that, in a first process of sub-bitstream extraction to output a first output sub-bitstream, the first output sub-bitstream is extracted without removing network abstraction layer (NAL) units of a particular type and having a particular NAL unit header identifier value, and wherein the particular type includes an access unit delimiter (AUD) NAL unit.

Inventors

• Ye-Kui Wang

Assignees

• BYTEDANCE INC.

Dates

Publication Date

20260505

Application Date

20241209

Claims (20)

1. 1 . A method of processing video data, comprising: performing a conversion between a video including one or more layers and a bitstream of the video according to a rule, wherein the rule specifies, in a process of sub-bitstream extraction in which a first sub-bitstream is used to derive an output sub-bitstream, whether to generate non-scalable-nested supplementary enhancement information (SEI) messages by extracting scalable-nested SEI messages from a scalable nesting SEI message is based on a first flag denoted as sn_ols_flag and a second flag denoted as sn_subpic_flag, wherein the scalable nesting SEI message is contained in a first SEI network abstraction layer (NAL) unit contained in the first sub-bitstream, and wherein in a case that the non-scalable-nested SEI messages are generated, a second SEI NAL unit is generated, and the scalable-nested SEI messages are extracted from the scalable nesting SEI message and are included in the second SEI NAL unit as the non-scalable-nested SEI messages.
2. 2 . The method of claim 1 , wherein the second SEI NAL unit is included immediately after the first SEI NAL unit in a picture unit (PU) containing the first SEI NAL unit in the first sub-bitstream.
3. 3 . The method of claim 2 , wherein the first SEI NAL unit is removed from the first sub-bitstream to generate the output sub-bitstream.
4. 4 . The method of claim 1 , wherein the rule specifies to generate the non-scalable-nested SEI messages when a set of conditions are satisfied, wherein the set of conditions comprise that the first SEI NAL unit contain the scalable nesting SEI message with the sn_ols_flag equal to 1 and the sn_subpic_flag that applies to a target output layer set (OLS) is equal to 0.
5. 5 . The method of claim 4 , wherein the set of conditions further comprise that a list of NAL unit header identifier values in an OLS with a target OLS index does not include all values of NAL unit header identifiers in all video coding layer (VCL) NAL units in the first sub-bitstream.
6. 6 . The method of claim 5 , wherein the rule further specifies that, in response to that the list of NAL unit header identifier values in the OLS does not include all values of NAL unit header identifiers in all VCL NAL units in the bitstream that are input to the process of sub-bitstream extraction, the following further applies: 1) all SEI NAL units that contain a non-scalable-nested SEI message with payload type equal to 0 or 130 are removed from the output sub-bitstream, and 2) when general_same_pic_timing_in_all_ols_flag is equal to 0, all SEI NAL units that contain a non-scalable-nested SEI message with payload type equal to 1 are removed from the output sub-bitstream.
7. 7 . The method of claim 1 , wherein the conversion includes encoding the video into the bitstream.
8. 8 . The method of claim 1 , wherein the conversion includes decoding the video from the bitstream.
9. 9 . An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to: perform a conversion between a video including one or more layers and a bitstream of the video according to a rule, wherein the rule specifies, in a process of sub-bitstream extraction in which a first sub-bitstream is used to derive an output sub-bitstream, whether to generate non-scalable-nested supplementary enhancement information (SEI) messages by extracting scalable-nested SEI messages from a scalable nesting SEI message is based on a first flag denoted as sn_ols_flag and a second flag denoted as sn_subpic_flag, wherein the scalable nesting SEI message is contained in a first SEI network abstraction layer (NAL) unit contained in the first sub-bitstream, and wherein in a case that the non-scalable-nested SEI messages are generated, a second SEI NAL unit is generated, and the scalable-nested SEI messages are extracted from the scalable nesting SEI message and are included in the second SEI NAL unit as the non-scalable-nested SEI messages.
10. 10 . The apparatus of claim 9 , wherein the second SEI NAL unit is included immediately after the first SEI NAL unit in a picture unit (PU) containing the first SEI NAL unit in the first sub-bitstream, and wherein the first SEI NAL unit is removed from the first sub-bitstream to generate the output sub-bitstream.
11. 11 . The apparatus of claim 9 , wherein the rule specifies to generate the non-scalable-nested SEI messages when a set of conditions are satisfied, wherein the set of conditions comprise that the first SEI NAL unit contain the scalable nesting SEI message with the sn_ols_flag equal to 1 and the sn_subpic_flag that applies to a target output layer set (OLS) is equal to 0.
12. 12 . The apparatus of claim 11 , wherein the set of conditions comprise that a list of NAL unit header identifier values in an OLS with a target OLS index does not include all values of NAL unit header identifiers in all video coding layer (VCL) NAL units in the first sub-bitstream.
13. 13 . The apparatus of claim 12 , wherein the rule further specifies that, in response to that the list of NAL unit header identifier values in the OLS does not include all values of NAL unit header identifiers in all VCL NAL units in the bitstream that are input to the process of sub-bitstream extraction, the following further applies: 1) all SEI NAL units that contain a non-scalable-nested SEI message with payload type equal to 0 or 130 are removed from the output sub-bitstream, and 2) when general_same_pic_timing_in_all_ols_flag is equal to 0, all SEI NAL units that contain a non-scalable-nested SEI message with payload type equal to 1 are removed from the output sub-bitstream.
14. 14 . A non-transitory computer-readable storage medium storing instructions that cause a processor to: perform a conversion between a video including one or more layers and a bitstream of the video according to a rule, wherein the rule specifies, in a process of sub-bitstream extraction in which a first sub-bitstream is used to derive an output sub-bitstream, whether to generate non-scalable-nested supplementary enhancement information (SEI) messages by extracting scalable-nested SEI messages from a scalable nesting SEI message is based on a first flag denoted as sn_ols_flag and a second flag denoted as sn_subpic_flag, wherein the scalable nesting SEI message is contained in a first SEI network abstraction layer (NAL) unit contained in the first sub-bitstream, and wherein in a case that the non-scalable-nested SEI messages are generated, a second SEI NAL unit is generated, and the scalable-nested SEI messages are extracted from the scalable nesting SEI message and are included in the second SEI NAL unit as the non-scalable-nested SEI messages.
15. 15 . The non-transitory computer-readable storage medium of claim 14 , wherein the second SEI NAL unit is included immediately after the first SEI NAL unit in a picture unit (PU) containing the first SEI NAL unit in the first sub-bitstream, and wherein the first SEI NAL unit is removed from the first sub-bitstream to generate the output sub-bitstream.
16. 16 . The non-transitory computer-readable storage medium of claim 14 , wherein the rule specifies to generate the non-scalable-nested SEI messages when a set of conditions are satisfied, wherein the set of conditions comprise that the first SEI NAL unit contain the scalable nesting SEI message with the sn_ols_flag equal to 1 and the sn_subpic_flag that applies to a target output layer set (OLS) is equal to 0.
17. 17 . The non-transitory computer-readable storage medium of claim 16 , wherein the set of conditions comprise that a list of NAL unit header identifier values in an OLS with a target OLS index does not include all values of NAL unit header identifiers in all video coding layer (VCL) NAL units in the first sub-bitstream.
18. 18 . A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises: generating the bitstream of the video including one or more layers according to a rule, wherein the rule specifies, in a process of sub-bitstream extraction in which a first sub-bitstream is used to derive an output sub-bitstream, whether to generate non-scalable-nested supplementary enhancement information (SEI) messages by extracting scalable-nested SEI messages from a scalable nesting SEI message is based on a first flag denoted as sn_ols_flag and a second flag denoted as sn_subpic_flag, wherein the scalable nesting SEI message is contained in a first SEI network abstraction layer (NAL) unit contained in the first sub-bitstream, and wherein in a case that the non-scalable-nested SEI messages are generated, a second SEI NAL unit is generated, and the scalable-nested SEI messages are extracted from the scalable nesting SEI message and are included in the second SEI NAL unit as the non-scalable-nested SEI messages.
19. 19 . The non-transitory computer-readable recording medium of claim 18 , wherein the second SEI NAL unit is included immediately after the first SEI NAL unit in a picture unit (PU) containing the first SEI NAL unit in the first sub-bitstream, and wherein the first SEI NAL unit is removed from the first sub-bitstream to generate the output sub-bitstream.
20. 20 . The non-transitory computer-readable recording medium of claim 18 , wherein the rule specifies to generate the non-scalable-nested SEI messages when a set of conditions are satisfied, wherein the set of conditions comprise that the first SEI NAL unit contain the scalable nesting SEI message with the sn_ols_flag equal to 1 and the sn_subpic_flag that applies to a target output layer set (OLS) is equal to 0.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation of U.S. patent application Ser. No. 18/488,692, filed on Oct. 17, 2023, which is a continuation of U.S. patent application Ser. No. 17/992,176, filed on Nov. 22, 2022 (U.S. Pat. No. 11,825,102, issued Nov. 21, 2023), which is a continuation of International Patent Application No. PCT/US2021/033648, filed on May 21, 2021, which claims the priority to and benefits of U.S. Provisional Patent Application No. 63/029,308, filed on May 22, 2020. All the aforementioned patent applications are hereby incorporated by reference in their entireties. TECHNICAL FIELD The present disclosure relates to image and video coding and decoding. BACKGROUND Digital video accounts for the largest bandwidth use on the internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, it is expected that the bandwidth demand for digital video usage will continue to grow. SUMMARY The present disclosure discloses embodiments that can be used by video encoders and decoders to perform video encoding or decoding. In one example aspect, a video processing method is disclosed. The method includes performing a conversion between a video comprising multiple layers and a bitstream of the video according to a rule, wherein the rule specifies a maximum allowed value of a temporal layer identifier value of a sub-bitstream that is extracted from the bitstream. In another example aspect, a video processing method is disclosed. The method includes performing a conversion between a video and a bitstream of the video according to a rule, wherein the rule defines network abstraction layer (NAL) units to be extracted from the bitstream during a sub-bitstream extraction process to output a sub-bitstream, and wherein the rule specifies to derive the sub-bitstream based on whether a list of NAL unit header identifier values in an output layer set (OLS) with a target OLS index does not include all values of NAL unit header identifiers in all video coding layer (VCL) NAL units in the bitstream that is input to the sub-bitstream extraction process. In another example aspect, a video processing method is disclosed. The method includes performing a conversion between a video and a bitstream of the video according to a rule, wherein the rule defines network abstraction layer (NAL) units to be extracted from the bitstream during a sub-bitstream extraction process to output a sub-bitstream, and wherein the rule specifies, responsive to a payload type of a first supplemental enhancement information (SEI) message included in an SEI network abstraction layer (NAL) unit, to disallow the SEI NAL unit to contain an SEI message with a certain payload type. In another example aspect, a video processing method is disclosed. The method includes performing a conversion between a video and a bitstream of the video according to a format rule, wherein the format rule specifies that the bitstream includes a flag specifying whether one or more non-scalable nested supplemental enhancement information (SEI) messages with one or more particular payload types apply to all output layer sets referenced by a coding layer. In another example aspect, a video processing method is disclosed. The method includes performing a conversion between a video and a bitstream of the video according to a rule, wherein the rule specifies a sub-bitstream extraction process by which an output sub-bitstream is extracted from the bitstream without removing network abstraction layer (NAL) units of a particular type and having a particular NAL unit header identifier value, wherein the particular type includes an access unit delimiter (AUD) NAL unit. In another example aspect, a video processing method is disclosed. The method includes performing a conversion between a video including one or more layers and a bitstream of the video according to a rule, wherein the rule specifies, in a process of sub-bitstream extraction, to remove network abstraction layer (NAL) units that include a scalable nesting supplementary enhancement information (SEI) message applied to layers that are not included in a target output layer set (OLS). In another example aspect, a video processing method is disclosed. The method includes performing a conversion between a video including one or more layers and a bitstream of the video according to a rule, wherein the rule specifies, in a process of sub-bitstream extraction, a non-scalable nested supplementary enhancement information (SEI) message is generated by extracting an SEI message that is scalably-nested from a scalable nesting SEI message based on a first flag indicating whether the SEI message applies to specific output layer sets (OLSs) and a second flag indicating whether the SEI message applies to all subpictures or only to specific subpictures. In another example aspect, a video processing method is disclosed. The method incl