US-12627797-B2 - Preprocessing of input data for adaptive loop filter in video coding

US12627797B2US 12627797 B2US12627797 B2US 12627797B2US-12627797-B2

Abstract

A device for decoding video data can be configured to determine a prediction block for a current block of the video data; determine a residual block for the current block of the video data; process residual data of the residual block to determine processed residual data; store the processed residual data; add the prediction block to the residual block to determine a reconstructed block; and perform adaptive loop filtering (ALF) on the reconstructed block, based on the processed residual data.

Inventors

Han Huang
Cheng-Teh Hsieh
Nan Hu
Vadim SEREGIN
Marta Karczewicz

Assignees

QUALCOMM INCORPORATED

Dates

Publication Date: 20260512
Application Date: 20240228

Claims (14)

1 . A method of decoding video data, the method comprising: determining a prediction block for a current block of the video data; determining a residual block for the current block of the video data; subsampling sample values of the residual block to determine processed residual data, the processed residual data comprising the subsampled sample values of the residual block; storing the processed residual data; adding the prediction block and the residual block to determine a reconstructed block; determining a filter, from a set of filters, based on the processed residual data; and performing adaptive loop filtering (ALF) on the reconstructed block, based on the determined filter.
2 . The method of claim 1 , wherein the processed residual data comprises less data than the samples of the residual block.
3 . The method of claim 1 , wherein performing the ALF on the reconstructed block comprises performing ALF on the reconstructed block after performing one or both of deblock filtering and sample adaptive offset filtering on the reconstructed block.
4 . The method of claim 3 , further comprising: storing the processed residual data before performing the one or both of the deblock filtering and the sample adaptive offset filtering on the reconstructed block.
5 . The method of claim 1 , wherein the method of decoding the video data is performed as part of a video encoding process.
6 . A device for decoding video data, the device comprising: a memory configured to store video data; one or more processors implemented in circuitry and configured to: determine a prediction block for a current block of the video data; determine a residual block for the current block of the video data; subsample sample values of the residual block to determine processed residual data, the processed residual data comprising the subsampled sample values of the residual block; store the processed residual data; add the prediction block and the residual block to determine a reconstructed block; determine a filter, from a set of filters, based on the processed residual data; and perform adaptive loop filtering (ALF) on the reconstructed block, based on the determined filter.
7 . The device of claim 6 , wherein the processed residual data comprises less data than the samples of the residual block.
8 . The device of claim 6 , wherein to perform the ALF on the reconstructed block, the one or more processors are further configured to perform ALF on the reconstructed block after performing one or both of deblock filtering and sample adaptive offset filtering on the reconstructed block.
9 . The device of claim 8 , wherein the one or more processors are further configured to: store the processed residual data before performing the one or both of the deblock filtering and the sample adaptive offset filtering on the reconstructed block.
10 . The device of claim 6 , further comprising a display configured to display decoded video data.
11 . The device of claim 6 , wherein the device comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.
12 . The device of claim 6 , wherein the device comprises a video decoder.
13 . The device of claim 6 , wherein the device comprises a video encoder.
14 . A non-transitory computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors to: determine a prediction block for a current block of video data; determine a residual block for the current block of the video data; subsample sample values of the residual block to determine processed residual data, the processed residual data comprising the subsampled sample values of the residual block; store the processed residual data; add the prediction block to the residual block to determine a reconstructed block; determine a filter, from a set of filters, based on the processed residual data; and perform adaptive loop filtering (ALF) on the reconstructed block, based on the determined filter.

Description

This application claims the benefit of U.S. Provisional Patent Application No. 63/487,698, filed 1 Mar. 2023, the entire contents of which is incorporated herein by reference. TECHNICAL FIELD This disclosure relates to video encoding and video decoding. BACKGROUND Digital video capabilities can be incorporated into a wide range of devices, including digital televisions, digital direct broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, e-book readers, digital cameras, digital recording devices, digital media players, video gaming devices, video game consoles, cellular or satellite radio telephones, so-called “smart phones,” video teleconferencing devices, video streaming devices, and the like. Digital video devices implement video coding techniques, such as those described in the standards defined by MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264/MPEG-4, Part 10, Advanced Video Coding (AVC), ITU-T H.265/High Efficiency Video Coding (HEVC), ITU-T H.266/Versatile Video Coding (VVC), and extensions of such standards, as well as proprietary video codecs/formats such as AOMedia Video 1 (AV1) that was developed by the Alliance for Open Media. The video devices may transmit, receive, encode, decode, and/or store digital video information more efficiently by implementing such video coding techniques. Video coding techniques include spatial (intra-picture) prediction and/or temporal (inter-picture) prediction to reduce or remove redundancy inherent in video sequences. For block-based video coding, a video slice (e.g., a video picture or a portion of a video picture) may be partitioned into video blocks, which may also be referred to as coding tree units (CTUs), coding units (CUs) and/or coding nodes. Video blocks in an intra-coded (I) slice of a picture are encoded using spatial prediction with respect to reference samples in neighboring blocks in the same picture. Video blocks in an inter-coded (P or B) slice of a picture may use spatial prediction with respect to reference samples in neighboring blocks in the same picture or temporal prediction with respect to reference samples in other reference pictures. Pictures may be referred to as frames, and reference pictures may be referred to as reference frames. SUMMARY Some implementations of adaptive loop filtering (ALF) utilize one or both of residual samples or reconstructed samples to determine a filter used at the ALF stage. ALF is generally performed after deblocking filtering and SAO filtering, while residual samples and reconstructed samples are determined before deblocking filtering and SAO filtering. Thus, in order to use the residual samples or reconstructed samples for ALF, the residual samples and reconstructed samples need to be buffered while deblocking filtering and SAO are performed. This buffering requires increased hardware complexity. To reduce the amount of required buffering, this disclosure describes techniques to process residual data of a residual block to determine processed residual data and store the processed residual data rather than the residual data itself. The processed residual data may have less data, e.g., fewer bits, than the residual data and, thus, require less memory to buffer. In some examples, the processing may also modify the residual data, such that the residual data is more likely to result in a better filter being selected during ALF. According to an example of this disclosure, a method of decoding video data includes determining a prediction block for a current block of the video data; determining a residual block for the current block of the video data; processing residual data of the residual block to determine processed residual data; storing the processed residual data; adding the prediction block to the residual block to determine a reconstructed block; and performing adaptive loop filtering (ALF) on the reconstructed block, based on the processed residual data. According to an example of this disclosure, a device for decoding video data includes a memory configured to store video data and one or more processors implemented in circuitry and configured to determine a prediction block for a current block of the video data; determine a residual block for the current block of the video data; process residual data of the residual block to determine processed residual data; store the processed residual data; add the prediction block to the residual block to determine a reconstructed block; and perform adaptive loop filtering (ALF) on the reconstructed block, based on the processed residual data. According to an example of this disclosure, a computer-readable storage medium stores instructions that, when executed, cause one or more processors to determine a prediction block for a current block of video data; determine a residual block for the current block of the video data; process residual data of the residual block to determine processed residual dat