US-12621435-B2 - Constraints on reference picture lists

Abstract

Methods and apparatus for processing of video are described. The processing may include video encoding, decoding, or transcoding. One example video processing method includes performing a conversion between a video including one or more pictures including one or more subpictures and a bitstream of the video. The bitstream conforms to a format rule that specifies that a subpicture cannot be a random access type of subpicture in response to the subpicture not being a leading subpicture of an intra random access point subpicture. The leading subpicture precedes the intra random access point subpicture in output order.

Inventors

• Ye-Kui Wang

Assignees

• BYTEDANCE INC.

Dates

Publication Date

20260505

Application Date

20231204

Claims (20)

1. 1 . A method of video processing, comprising: performing a conversion between a video comprising one or more pictures comprising one or more subpictures and a bitstream of the video, wherein the bitstream conforms to a format rule that specifies that a subpicture cannot be a random access decodable leading subpicture or a random access skipped leading subpicture when the subpicture is not a leading subpicture of an intra random access point subpicture, wherein the leading subpicture precedes the intra random access point subpicture in output order, wherein the intra random access point subpicture is a subpicture for which all video coding layer (VCL) network abstraction layer (NAL) units have a same value of NAL unit type in a range of IDR_W_RADL to CRA_NUT, inclusive, wherein the random access decodable leading subpicture is a subpicture for which each VCL NAL unit has NAL unit type equal to RADL_NUT, and wherein the random access skipped leading is a subpicture for which each VCL NAL has NAL unit type equal to RASL_NUT.
2. 2 . The method of claim 1 , wherein the format rule further specifies that a first subpicture precedes a second subpicture in a recovery point picture in an output order when: the first subpicture and the second subpicture having a same layer identifier and a same subpicture index, and the first subpicture preceding the second subpicture in a decoding order; and wherein the layer identifier specifies the identifier of the layer to which a VCL NAL unit belongs or the identifier of a layer to which a non-VCL NAL unit applies.
3. 3 . The method of claim 1 , wherein when a current subpicture is a clean random access subpicture, the format rule disallows a picture referred to by an entry in a reference picture list that precedes, in a first order, any preceding picture in a second order, containing an intra random access point subpicture with a same layer identifier and a same subpicture index as the current subpicture, wherein the clean random access subpicture is an intra random access point subpicture for which each Video Coding Layer (VCL) NAL unit has nal_unit_type equal to CRA_NUT; and wherein the layer identifier specifies the identifier of the layer to which a VCL NAL unit belongs or the identifier of a layer to which a non-VCL NAL unit applies.
4. 4 . The method of claim 3 , wherein the first order comprises a decoding order or an output order, wherein the second order comprises a decoding order, and wherein the reference picture list includes a reference picture list 0 or a reference picture list 1.
5. 5 . The method of claim 1 , wherein the format rule specifies an order by which pictures are indicated in the bitstream, wherein the format rule disallows an active entry in a reference picture list of a current slice from including a first picture that precedes a second picture according to a first order, wherein the second picture comprises an intra random access point subpicture having a same layer identifier and a same subpicture index as a current subpicture, wherein the current subpicture follows the intra random access point subpicture in a second order; and wherein the layer identifier specifies the identifier of the layer to which a VCL NAL unit belongs or the identifier of a layer to which a non-VCL NAL unit applies.
6. 6 . The method of claim 5 , wherein the first order comprises a decoding order or an output order, and the second order comprises a decoding order and an output order, wherein the active entry corresponds to an entry that is available for use as a reference index in an inter prediction of the current slice, and wherein the reference picture list includes a reference picture list 0 or a reference picture list 1.
7. 7 . The method of claim 1 , wherein the format rule specifies an order by which pictures are indicated in the bitstream, wherein the format rule disallows an entry in a reference picture list of a current slice from including a first picture that precedes a second picture according to a first order or a second order, wherein the second picture comprises an intra random access point subpicture having zero or more associated leading subpictures and having a same layer identifier and a same subpicture index as a current subpicture, wherein the current subpicture follows the intra random access point subpicture and the zero or more associated leading subpictures in the first order and the second order; and wherein the layer identifier specifies the identifier of the layer to which a VCL NAL unit belongs or the identifier of a layer to which a non-VCL NAL unit applies.
8. 8 . The method of claim 7 , wherein the first order comprises a decoding order, and the second order comprises an output order, and wherein the reference picture list includes a reference picture list 0 or a reference picture list 1.
9. 9 . The method of claim 1 , wherein the format rule further specifies that when a current subpicture is a random access decodable leading subpicture, an active entry of a reference picture list of a current slice is disallowed from including any one or more of: a first picture including a random access skipped leading subpicture having a same subpicture index as that of the current subpicture, and a second picture that precedes a third picture including an intra random access point subpicture associated with the random access decodable leading subpicture in a decoding order.
10. 10 . The method of claim 9 , wherein the active entry corresponds to an entry that is available for use as a reference index in an inter prediction of the current slice, and wherein the reference picture list includes a reference picture list 0 or a reference picture list 1.
11. 11 . The method of claim 1 , wherein the performing the conversion includes encoding the video into the bitstream.
12. 12 . The method of claim 1 , wherein the performing the conversion includes decoding the video from the bitstream.
13. 13 . An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to: perform a conversion between a video comprising one or more pictures comprising one or more subpictures and a bitstream of the video, wherein the bitstream conforms to a format rule that specifies that a subpicture cannot be a random access decodable leading subpicture or a random access skipped leading subpicture when the subpicture is not a leading subpicture of an intra random access point subpicture, wherein the leading subpicture precedes the intra random access point subpicture in output order, wherein the intra random access point subpicture is a subpicture for which all video coding layer (VCL) network abstraction layer (NAL) units have a same value of NAL unit type in a range of IDR_W_RADL to CRA_NUT, inclusive, wherein the random access decodable leading subpicture is a subpicture for which each VCL NAL unit has NAL unit type equal to RADL_NUT, and wherein the random access skipped leading is a subpicture for which each VCL NAL has NAL unit type equal to RASL_NUT.
14. 14 . The apparatus of claim 13 , wherein the format rule further specifies that a first subpicture precedes a second subpicture in a recovery point picture in an output order when: the first subpicture and the second subpicture has a same layer identifier and a same subpicture index, and the first subpicture precedes the second subpicture in a decoding order; and wherein the layer identifier specifies the identifier of the layer to which a VCL NAL unit belongs or the identifier of a layer to which a non-VCL NAL unit applies.
15. 15 . The apparatus of claim 13 , wherein when a current subpicture is a clean random access subpicture, the format rule disallows a picture referred to by an entry in a reference picture list that precedes, in a first order, any preceding picture, in a second order, containing an intra random access point subpicture with a same layer identifier and a same subpicture index as the current subpicture, wherein the layer identifier specifies the identifier of the layer to which a VCL NAL unit belongs or the identifier of a layer to which a non-VCL NAL unit applies; wherein the clean random access subpicture is an intra random access point subpicture for which each Video Coding Layer (VCL) NAL unit has nal_unit_type equal to CRA_NUT; wherein the first order comprises a decoding order or an output order, wherein the second order comprises a decoding order, and wherein the reference picture list includes a reference picture list 0 or a reference picture list 1.
16. 16 . The apparatus of claim 13 , wherein the format rule specifies an order by which pictures are indicated in the bitstream, wherein the format rule disallows an active entry in a reference picture list of a current slice from including a first picture that precedes a second picture according to a first order, wherein the second picture comprises an intra random access point subpicture having a same layer identifier and a same subpicture index as a current subpicture, wherein the layer identifier specifies the identifier of the layer to which a VCL NAL unit belongs or the identifier of a layer to which a non-VCL NAL unit applies, and wherein the current subpicture follows the intra random access point subpicture in a second order; wherein the first order comprises a decoding order or an output order, and the second order comprises a decoding order and an output order, wherein the active entry corresponds to an entry that is available for use as a reference index in an inter prediction of the current slice, and wherein the reference picture list includes a reference picture list 0 or a reference picture list 1.
17. 17 . A non-transitory computer-readable storage medium storing instructions that cause a processor to: perform a conversion between a video comprising one or more pictures comprising one or more subpictures and a bitstream of the video, wherein the bitstream conforms to a format rule that specifies that a subpicture cannot be a random access decodable leading subpicture or a random access skipped leading subpicture when the subpicture is not a leading subpicture of an intra random access point subpicture, wherein the leading subpicture precedes the intra random access point subpicture in output order, wherein the intra random access point subpicture is a subpicture for which all video coding layer (VCL) network abstraction layer (NAL) units have a same value of NAL unit type in a range of IDR_W_RADL to CRA_NUT, inclusive, wherein the random access decodable leading subpicture is a subpicture for which each VCL NAL unit has NAL unit type equal to RADL_NUT, and wherein the random access skipped leading is a subpicture for which each VCL NAL has NAL unit type equal to RASL_NUT.
18. 18 . A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises: generating the bitstream of the video comprising one or more pictures comprising one or more subpictures, wherein the bitstream conforms to a format rule that specifies that a subpicture cannot be a random access decodable leading subpicture or a random access skipped leading subpicture when the subpicture is not a leading subpicture of an intra random access point subpicture, wherein the leading subpicture precedes the intra random access point subpicture in output order, wherein the intra random access point subpicture is a subpicture for which all video coding layer (VCL) network abstraction layer (NAL) units have a same value of NAL unit type in a range of IDR_W_RADL to CRA_NUT, inclusive, wherein the random access decodable leading subpicture is a subpicture for which each VCL NAL unit has NAL unit type equal to RADL_NUT, and wherein the random access skipped leading is a subpicture for which each VCL NAL has NAL unit type equal to RASL_NUT.
19. 19 . The non-transitory computer-readable storage medium of claim 17 , wherein the format rule further specifies that a first subpicture precedes a second subpicture in a recovery point picture in an output order when: the first subpicture and the second subpicture having a same layer identifier and a same subpicture index, and the first subpicture preceding the second subpicture in a decoding order; and wherein the layer identifier specifies the identifier of the layer to which a VCL NAL unit belongs or the identifier of a layer to which a non-VCL NAL unit applies.
20. 20 . The non-transitory computer-readable recording medium of claim 18 , wherein the format rule further specifies that a first subpicture precedes a second subpicture in a recovery point picture in an output order when: the first subpicture and the second subpicture having a same layer identifier and a same subpicture index, and the first subpicture preceding the second subpicture in a decoding order; and wherein the layer identifier specifies the identifier of the layer to which a VCL NAL unit belongs or the identifier of a layer to which a non-VCL NAL unit applies.

Description

CROSS REFERENCE TO RELATED APPLICATIONS This application is a continuation of U.S. application Ser. No. 17/969,976, filed on Oct. 20, 2022, which is a continuation of International Patent Application No. PCT/US2021/027963 filed on Apr. 19, 2021, which claims the priority to and benefits of U.S. Provisional Patent Application No. 63/012,713 filed on Apr. 20, 2020. All the aforementioned patent applications are hereby incorporated by reference in their entireties. TECHNICAL FIELD The present disclosure relates to image and video coding and decoding. BACKGROUND Digital video accounts for the largest bandwidth use on the internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, it is expected that the bandwidth demand for digital video usage will continue to grow. SUMMARY The present disclosure discloses techniques that can be used by video encoders and decoders for processing coded representation of video using various rules of syntax. In one example aspect, a video processing method is disclosed. The method includes performing a conversion between a video comprising one or more pictures comprising one or more subpictures and a bitstream of the video, wherein the bitstream conforms to a format rule that specifies that a subpicture cannot be a random access type of subpicture in response to the subpicture not being a leading subpicture of an intra random access point subpicture, and wherein the leading subpicture precedes the intra random access point subpicture in output order. In another example aspect, a video processing method is disclosed. The method includes performing a conversion between a video comprising one or more pictures comprising a plurality of subpictures and a bitstream of the video, wherein the bitstream conforms to a format rule that specifies that a first subpicture precedes a second subpicture in a recovery point picture in an output order in response to: the first subpicture and the second picture having a same layer identifier of a network abstraction layer (NAL) unit and a same subpicture index, and the first subpicture preceding the second subpicture in a decoding order. In another example aspect, a video processing method is disclosed. The method includes performing a conversion between a video comprising a current picture comprising a current subpicture comprising a current slice and a bitstream of the video, wherein the bitstream conforms to a format rule, wherein the format rule specifies an order by which pictures are indicated in the bitstream, wherein the format rule disallows an entry in a reference picture list of the current slice from including a first picture that precedes, according to a first order, a second picture that precedes, according to a second order, the current picture, wherein the second picture comprises an intra random access point subpicture having a same layer identifier of a network abstraction unit (NAL) unit and a same subpicture index as the current subpicture, and wherein the current subpicture is a clean random access subpicture. In another example aspect, a video processing method is disclosed. The method includes performing a conversion between a video comprising a current picture comprising a current subpicture comprising a current slice and a bitstream of the video, wherein the bitstream conforms to a format rule, wherein the format rule specifies an order by which pictures are indicated in the bitstream, wherein the format rule disallows an active entry in a reference picture list of the current slice from including a first picture that precedes a second picture according to a first order, wherein the second picture comprises an intra random access point subpicture having a same layer identifier of a network abstraction unit (NAL) unit and a same subpicture index as the current subpicture, and wherein the current subpicture follows the intra random access point subpicture in a second order. In another example aspect, a video processing method is disclosed. The method includes performing a conversion between a video comprising a current picture comprising a current subpicture comprising a current slice and a bitstream of the video, wherein the bitstream conforms to a format rule, wherein the format rule specifies an order by which pictures are indicated in the bitstream, wherein the format rule disallows an entry in a reference picture list of the current slice from including a first picture that precedes a second picture according to a first order or a second order, wherein the second picture comprises an intra random access point subpicture having zero or more associated leading subpictures and having a same layer identifier of a network abstraction unit (NAL) unit and a same subpicture index as the current subpicture, and wherein the current subpicture follows the intra random access point subpicture and the zero or more associated leading subpictures in the f