US-12627799-B2 - CCSO with downsampling filters methods

Abstract

Various implementations described herein include methods and systems for coding video. In one aspect, a video bitstream includes a current image frame and a first syntax element for a cross-component sample offset (CCSO) mode. The first syntax element indicates whether to generate a first sample offset of a first color sample of the current image frame based on one or more luma samples. An electronic device identifies a first luma sample that is collocated with the first color sample and neighboring luma sample(s) s of the first luma sample. The first luma sample and the one neighboring luma sample(s) are downsampled to generate downsampled luma sample(s) for the CCSO mode. The electronic device determines the first sample offset of the first color sample based on the downsampled luma sample(s), and reconstructs the current image frame at least by adjusting the first color sample based on the first sample offset.

Inventors

• Han Gao
• Xin Zhao
• Liang Zhao
• Shan Liu
• Jing Ye

Assignees

• Tencent America LLC

Dates

Publication Date

20260512

Application Date

20240509

Claims (20)

1. 1 . A method for decoding video data, comprising: receiving a video bitstream including a current image frame, wherein the video bitstream includes a first syntax element indicating a downsampling index for a cross-component sample offset (CCSO) mode in which a first sample offset of a first color sample of the current image frame is generated based on one or more luma samples; when the downsampling index is zero, reconstructing the current image frame without applying the CCSO mode; and when the downsampling index is non-zero: identifying a first luma sample that is collocated with the first color sample and one or more neighboring luma samples of the first luma sample; downsampling, using a filter indicated by the downsampling index, the first luma sample and the one or more neighboring luma samples to generate one or more downsampled luma samples for the CCSO mode; determining the first sample offset of the first color sample based on the one or more downsampled luma samples; and reconstructing the current image frame at least by adjusting the first color sample based on the first sample offset.
2. 2 . The method of claim 1 , wherein determining the first sample offset of the first color sample further comprises: generating one or more quantized values based on the one or more downsampled luma samples; and classifying the first color sample based on the one or more quantized values to determine the first sample offset of the first color sample.
3. 3 . The method of claim 2 , wherein the CCSO mode corresponds to at least an edge offset classifier, and generating one or more quantized values based on the one or more downsampled luma samples further comprises, based on the edge offset classifier: determining that the one or more downsampled luma samples include a first downsampled luma sample and one or more neighboring downsampled luma samples; and determining one or more downsampled difference values between the one or more neighboring downsampled luma samples and the first downsampled luma sample; wherein the one or more quantized values are generated based on the one or more downsampled difference values and applied by the edge offset classifier to classify the first color sample.
4. 4 . The method of claim 2 , wherein: the CCSO mode corresponds to an edge offset classifier and an alternative classifier; the one or more quantized values include a first quantized value that is generated, and further applied by the edge offset classifier to classify the first color sample, based on the one or more downsampled luma samples; and the one or more quantized values further include a second quantized value that is generated, and further applied by the alternative classifier to classify the first color sample, based on a subset of the first luma sample and the one or more neighboring luma samples; and the first sample offset is determined based on an edge classification result and an alternative classification result.
5. 5 . The method of claim 2 , wherein the CCSO mode corresponds to at least a band offset classifier, and generating one or more quantized values based on the one or more downsampled luma samples further comprises, based on the band offset classifier: determining that the one or more downsampled luma samples include a first downsampled luma sample and one or more neighboring downsampled luma samples; wherein the one or more quantized values are generated based on the one or more downsampled luma samples, and applied by the band offset classifier, to classify the first color sample.
6. 6 . The method of claim 2 , wherein: the CCSO mode corresponds to a band offset classifier and an alternative classifier; the one or more quantized values include a first quantized value that is generated, and further applied by the band offset classifier to classify the first color sample, based on the one or more downsampled luma samples; and the one or more quantized values further include a second quantized value that is generated, and further applied by the alternative classifier to classify the first color sample, based on a subset of the first luma sample and the one or more neighboring luma samples; and the first sample offset is determined based on an edge classification result and an alternative classification result.
7. 7 . The method of claim 2 , wherein the CCSO mode corresponds to a plurality of offset classifiers, and generating one or more quantized values based on the one or more downsampled luma samples further comprises: determining that the one or more downsampled luma samples include a first downsampled luma sample and one or more neighboring downsampled luma samples; wherein the one or more quantized values are generated based on the one or more downsampled luma samples, and applied by at least one of the plurality of offset classifiers, to classify the first color sample and generate a respective classification result; wherein the first sample offset is determined based on a plurality of classification results including the respective classification result of at least one of the plurality of offset classifiers.
8. 8 . The method of claim 7 , wherein the plurality of offset classifiers include a band offset classifier and an edge offset classifier.
9. 9 . The method of claim 1 , further comprising: applying a chroma-from-luma (CfL) filter in a cross-component intra prediction (CCIP) mode to determine a first chroma sample, which is collocated with the first luma sample, based on a subset of the first luma sample and the one or more neighboring luma samples.
10. 10 . The method of claim 1 , wherein the first luma sample and the one or more neighboring luma samples are downsampled using a first downsampling filter for the CCSO mode, the method further comprising: applying a second downsampling filter to the first luma sample or the first color sample in a coding module distinct from the CCSO mode, wherein a type of the first downsampling filter matches a type of the second downsampling filter.
11. 11 . The method of claim 1 , wherein the video bitstream further includes a second high-level syntax element selecting one or more classifiers applied in the CCSO mode to generate the first sample offset of the first color sample based on one or more luma samples, the second high-level syntax element having a first predefined value indicating that the one or more classifiers includes only a band offset classifier and a second predefined value indicating that the one or more classifiers includes both the band offset classifier and an edge offset classifier.
12. 12 . The method of claim 11 , wherein a first quantized value is generated, and further applied by the band offset classifier to classify the first color sample, based on (1) the one or more downsampled luma samples or (2) a subset of the first luma sample and the one or more neighboring luma samples, the method further comprising: classifying the first color sample based on at least the first quantized value to determine the first sample offset of the first color sample.
13. 13 . The method of claim 1 , wherein: the video bitstream further includes a third high-level syntax element for a downsampling mode when only a band offset classifier is applied in the CCSO mode; the third high-level syntax element has a plurality of predefined values including a first predefined value and one or more second predefined values; the first predefined value indicates that one or more quantized values are generated, and further applied only by the band offset classifier to classify the first color sample, based on a subset of the first luma sample and the one or more neighboring luma samples; and the one or more second predefined values correspond to one or more types of a downsampling filter, and each second predefined value indicates that the one or more quantized values are generated, and further applied to classify the first color sample, based on the one or more downsampled luma samples generated by a respective type of the downsampling filter.
14. 14 . The method of claim 1 , wherein the video bitstream further includes a fourth high-level syntax element for a downsampling filter mode indicating application conditions of the one or more downsampled luma samples to a band offset classifier and an edge offset classifier, and the fourth high-level syntax element corresponds to four predefined values that are uniquely associated with four application conditions including: the band offset classifier alone, a combination of the band offset classifier and the edge offset classifier, the edge offset classifier alone, and no application of either the band offset classifier or an edge offset classifier.
15. 15 . The method of claim 1 , further comprising: determining, on one of a high level and a block level, application conditions of the one or more downsampled luma samples to a band offset classifier and an edge offset classifier based on coded information, wherein the coded information includes one or more of: a distinct coded syntax value, a prediction mode, a block size, a quantization parameter, whether the current image frame is intra or inter coded, whether the first color sample is coded using intra block coding mode, and a palette coding mode; and wherein based on the application conditions, one or more quantized values are generated, and further applied to classify the first color sample, based on the one or more downsampled luma samples.
16. 16 . The method of claim 1 , wherein the first color sample is one of the first luma sample and a first chroma sample that is collocated with the first luma sample, and wherein each of the first luma sample and the one or more neighboring luma samples includes a respective reconstructed luma sample, and the first chroma sample includes a first reconstructed chroma sample.
17. 17 . A computing system, comprising: control circuitry; and memory storing one or more programs configured to be executed by the control circuitry, the one or more programs further comprising instructions for: receiving video data comprising a current image frame; when a cross-component sample offset (CCSO) mode is not to be used for the current image frame, signaling, via a video bitstream, a syntax element with a value of zero; when the CCSO mode is to be used for the current image frame: encoding the current image frame using the CCSO mode to generate a first sample offset of a first color sample of the current image frame based on one or more luma samples, wherein: in the CCSO mode, the first sample offset of the first color sample is determined based on one or more downsampled luma samples that are generated by downsampling a first luma sample and one or more neighboring luma samples, and the first luma sample is collocated with the first color sample; transmitting the encoded current image frame via the video bitstream; and signaling, via the video bitstream, the syntax element with a value to indicate a filter to be used to perform the downsampling.
18. 18 . The computing system of claim 17 , wherein determining the first sample offset of the first color sample comprises: generating one or more quantized values based on the one or more downsampled luma samples; and classifying the first color sample based on the one or more quantized values to determine the first sample offset of the first color sample.
19. 19 . A method of generating a video bitstream, the method comprising: when a cross-component sample offset (CCSO) mode is not to be used for a current image frame of video data, setting a syntax element with a value of zero; and when the CCSO mode is to be used for the current image frame: encoding the current image frame using the CCSO mode to generate a first sample offset of a first color sample of the current image frame based on one or more luma samples, wherein, in the CCSO mode, the first sample offset of the first color sample is determined based on one or more downsampled luma samples that are generated by downsampling a first luma sample and one or more neighboring luma samples, and the first luma sample is collocated with the first color sample; and setting the syntax element with a non-zero value that indicates a filter to be used to perform the downsampling; and transmitting the video bitstream, wherein the video bitstream comprises the encoded current image frame and the syntax element.
20. 20 . The method of claim 19 , wherein determining the first sample offset of the first color sample comprises: generating one or more quantized values based on the one or more downsampled luma samples; and classifying the first color sample based on the one or more quantized values to determine the first sample offset of the first color sample.

Description

RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 63/545,885, entitled “CCSO with Downsampling Filters Methods,” filed Oct. 26, 2023, which is hereby incorporated by reference in its entirety. TECHNICAL FIELD The disclosed embodiments relate generally to video coding, including but not limited to systems and methods for loop filtering (e.g., cross-component offset filtering) of video data. BACKGROUND Digital video is supported by a variety of electronic devices, such as digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video gaming consoles, smart phones, video teleconferencing devices, video streaming devices, etc. The electronic devices transmit and receive or otherwise communicate digital video data across a communication network, and/or store the digital video data on a storage device. Due to a limited bandwidth capacity of the communication network and limited memory resources of the storage device, video coding may be used to compress the video data according to one or more video coding standards before it is communicated or stored. The video coding can be performed by hardware and/or software on an electronic/client device or a server providing a cloud service. Video coding generally utilizes prediction methods (e.g., inter-prediction, intra-prediction, or the like) that take advantage of redundancy inherent in the video data. Video coding aims to compress video data into a form that uses a lower bit rate, while avoiding or minimizing degradations to video quality. Multiple video codec standards have been developed. For example, High-Efficiency Video Coding (HEVC/H.265) is a video compression standard designed as part of the MPEG-H project. ITU-T and ISO/IEC published the HEVC/H.265 standard in 2013 (version 1), 2014 (version 2), 2015 (version 3), and 2016 (version 4). Versatile Video Coding (VVC/H.266) is a video compression standard intended as a successor to HEVC. ITU-T and ISO/IEC published the VVC/H.266 standard in 2020 (version 1) and 2022 (version 2). AOMedia Video 1 (AV1) is an open video coding format designed as an alternative to HEVC. On Jan. 8, 2019, a validated version 1.0.0 with Errata 1 of the specification was released. SUMMARY As mentioned above, encoding (compression) reduces the bandwidth and/or storage space requirements. As described in detail later, both lossless compression and lossy compression can be employed. Lossless compression refers to techniques where an exact copy of the original signal can be reconstructed from the compressed original signal via a decoding process. Lossy compression refers to coding/decoding process where original video information is not fully retained during coding and not fully recoverable during decoding. When using lossy compression, the reconstructed signal may not be identical to the original signal, but the distortion between original and reconstructed signals is made small enough to render the reconstructed signal useful for the intended application. The amount of tolerable distortion depends on the application. For example, users of certain consumer video streaming applications may tolerate higher distortion than users of cinematic or television broadcasting applications. The compression ratio achievable by a particular coding algorithm can be selected or adjusted to reflect various distortion tolerance: higher tolerable distortion generally allows for coding algorithms that yield higher losses and higher compression ratios. The present disclosure describes methods, systems, and non-transitory computer-readable storage media for applying a loop filter for video (image) compression. A video codec includes a plurality of function modules for one or more of: intra/inter prediction, transform coding, quantization, entropy coding, and in-loop filtering. In-loop filtering technologies are applied to adjust reconstructed picture samples to further reduce a reconstruction error. A cross-component offset filtering method is implemented to apply a co-located reconstructed sample and associated neighboring reconstructed samples of a first color component to derive an offset value that is added on a current sample of a second color component, thereby adjusting a reconstruction value of the current sample. Examples of the first color component is a luma color component, and examples of the second color component is a chroma color component. In some implementations, the first color component and the second color component correspond to the same color component, e.g., luma sample. In various embodiments of this application, a downsampling filter (also called a chroma from luma filter) is applied to samples of a first color component to generate downsampled samples of a first color component, and the downsampled samples are processed by a cross-component offset filter to determine an offset value that is added on a sample of a