Search

EP-4740462-A1 - BILATERAL FILTER

EP4740462A1EP 4740462 A1EP4740462 A1EP 4740462A1EP-4740462-A1

Abstract

A method for filtering input samples. The method includes obtaining a first input sample, S IN_1 , belonging to a first transform unit (TU) (e.g., one sample included in the TU) having a first dimension (e.g., width) and a second dimension (e.g., height). The method also includes using a first TU scale factor, C TU_1 , associated with the first TU and a bilateral filter, BIF, function to calculate a first BIF offset, δ BIF_1 , for the first input sample. The method further includes calculating a first output sample, S out_1 , using δ BIF_1 and S IN_1 . The first TU scale factor is a function of: 1) a two-dimensional, 2D, lookup table, LUT, and the magnitudes of the first and second dimensions of the first TU and/or 2) an activity score (e.g., MAD, MSD) for the first TU, and/or the BIF function is a sampled piecewise linear function.

Inventors

  • SHCHUKIN, Vladislav
  • WENNERSTEN, PER
  • STRÖM, Jacob

Assignees

  • Telefonaktiebolaget LM Ericsson (publ)

Dates

Publication Date
20260513
Application Date
20240701

Claims (20)

  1. 1. An apparatus (600) for encoding a picture, the apparatus configured to: obtain a first input sample, SIN_1, belonging to a first transform unit, TU, having a first dimension and a second dimension; use a first TU scale factor, CTU_1, associated with the first TU and a bilateral filter, BIF, function to calculate a first BIF offset, δ BIF_1, for the first input sample; and calculate a first output sample, Sout_1, using δBIF_1 and SIN_1, wherein: the first TU scale factor is a function of: 1) a two-dimensional, 2D, lookup table, LUT, and the magnitudes of the first and second dimensions of the first TU and/or 2) an activity score for the first TU, and/or the BIF function is a sampled piecewise linear function.
  2. 2. The apparatus of claim 1, wherein the TU scale factor is a function of the 2D LUT and the magnitudes of the first and second dimensions of the first TU.
  3. 3. The apparatus of claim 2, wherein the apparatus is further configured to: obtain a first look-up coordinate using the magnitude of the first dimension of the TU; obtain a second look-up coordinate using the magnitude of the second dimension of the TU; and use the look-up coordinates to select a scale factor from the 2D LUT, wherein C TU is a function of the scale factor selected from the 2D LUT.
  4. 4. The apparatus of claim 3, wherein the first look-up coordinate, x, is a function of the magnitude of the first dimension; the second look-up coordinate, y, is a function of the magnitude of the second dimension.
  5. 5. The apparatus of claim 4, wherein x = log 2 (W), were W is the magnitude of the first dimension of the first TU, and y = log2(H), were H is the magnitude of the second dimension of the first TU.
  6. 6. The apparatus of any one of claims wherein the first TU scale factor is a function of the activity score for the first TU.
  7. 7. The apparatus of claim 6, wherein the first TU scale factor is a function of a scale factor selected from a one-dimensional, 1D, LUT using a look-up index, j, that is a function of the activity score, AS.
  8. 8. The apparatus of claim 7, wherein j is equal to min(AS >> s, L), where L is the length of the 1D LUT, and s is a defined value.
  9. 9. The apparatus of claim 1, wherein the first TU scale factor is equal to the sum of SF1 and SF2, SF1 is a function of the 2D LUT and the magnitudes of the first and second dimensions of the first TU, and SF2 is a function of the activity score for the first TU.
  10. 10. The apparatus of claim 9, wherein the BIF function is a sampled piecewise linear function.
  11. 11. A method (500) for filtering input samples, the method comprising: obtaining (s502) a first input sample, SIN_1, belonging to a first transform unit, TU, having a first dimension and a second dimension; using (s504) a first TU scale factor, CTU_1, associated with the first TU and a bilateral filter, BIF, function to calculate a first BIF offset, δ BIF_1, for the first input sample; and calculating (s506) a first output sample, Sout_1, using δBIF_1 and SIN_1, wherein: the first TU scale factor is a function of: 1) a two-dimensional, 2D, lookup table, LUT, and the magnitudes of the first and second dimensions of the first TU and/or 2) an activity score for the first TU, and/or the BIF function is a sampled piecewise linear function.
  12. 12. The method of claim 11, wherein the TU scale factor is a function of the 2D LUT and the magnitudes of the first and second dimensions of the first TU.
  13. 13. The method of claim 12, wherein the method further comprises: obtaining a first look-up coordinate using the magnitude of the first dimension of the TU; obtaining a second look-up coordinate using the magnitude of the second dimension of the TU; and using the look-up coordinates to select a scale factor from the 2D LUT, wherein CTU is a function of the scale factor selected from the 2D LUT.
  14. 14. The method of claim 13, wherein the first look-up coordinate, x, is a function of the magnitude of the first dimension; the second look-up coordinate, y, is a function of the magnitude of the second dimension.
  15. 15. The method of claim 14, wherein x = log 2 (W), were W is the magnitude of the first dimension of the first TU, and y = log2(H), were H is the magnitude of the second dimension of the first TU.
  16. 16. The method of any one of claims 11-15, wherein the first TU scale factor is a function of the activity score for the first TU.
  17. 17. The method of claim 16, wherein the first TU scale factor is a function of a scale factor selected from a one-dimensional, 1D, LUT using a look-up index, j, that is a function of the activity score, AS.
  18. 18. The method of claim 17, wherein j is equal to min(AS >> s, L), where L is the length of the 1D LUT, and s is a defined value.
  19. 19. The method of claim 11, wherein the first TU scale factor is equal to the sum of SF1 and SF2, SF1 is a function of the 2D LUT and the magnitudes of the first and second dimensions of the first TU, and SF2 is a function of the activity the first TU.
  20. 20. The method of claim 19, wherein the BIF function is a sampled piecewise linear function.

Description

BILATERAL FILTER TECHNICAL FIELD [0001] This disclosure relates to bilateral filtering. BACKGROUND [0002] VVC and ECM [0003] Versatile Video Coding (VVC), also known as H.266, is a block-based video codec developed by the Joint Video Experts Team (JVET), a team of experts from ITU-T VCEG and ISO/IEC MPEG, and first published in 2020. [0004] JVET continues study on future video codec providing coding capabilities beyond VVC. The exploratory codec is called Enhanced Compression Model (ECM). The current ECM version is ECM-9.0. ECM is used for video compression. [0005] Video Sequence, Picture, Color Components [0006] A video (a.k.a., “video sequence”) comprises of a series of pictures. In VVC, each picture is identified with a picture order count (POC) value. The POC value also represents the display order of the picture. A picture with a smaller POC value is displayed before another picture with a larger POC value. [0007] Each picture consists of 2-dimensional arrays (a.k.a., blocks) of color components. Usually, each picture has three components: one luma component (Y) and two chroma components (Cb, Cr). Other color spaces can also be used, such as I, Cp, Ct. Usually, the chroma components are half the size in both the x and y dimension compared to the luma component. [0008] CTU, CU and TU [0009] In VVC and ECM, each component is split into square top-level blocks called Coding Tree Units (CTUs). The maximum supported size of a CTU is 256x256 in ECM-9.0. Each CTU is split into rectangular Coding Units (CUs). CU size varies from 4x4 to 256x256 in ECM-9.0. A CU consists of one or more Transform Units (TUs). Each TU is transform coded and typically varies in size from 4x4 to 128x128. If the size of the CU is higher than the maximal supported transform length, then the CU is split into two or more TUs. [0010] Parameter Sets and Headers [0011] ECM specifies three types of parameter sets: The picture parameter set (PPS), the sequence parameter set (SPS), and the video parameter set (VPS). The PPS contains data that is common for a whole picture, the SPS contains data that is common for a coded layer video sequence (CLVS), and the VPS contains data that is common for multiple CLVSs. [0012] A picture is split into slices that consists of several CTUs. The decoding of one slice in the picture is independent of other slices of the same picture. Pictures, slices and CTUs have headers comprising syntax elements used by the decoder. Correspondingly, coding parameters in the picture header are common for all slices in the picture, coding parameters in the slice header are common for all CTUs in the slice. [0013] Intra/Inter Prediction [0014] Each CU is either intra predicted or inter predicted. If it is intra predicted, it is predicted using previously decoded blocks in the same picture. An intra picture is a picture where all CUs must be intra predicted. In inter prediction, the CU is predicted using previously decoded pictures. A picture that allows inter predicted blocks is called an inter picture. [0015] Transform, Quantization, Entropy Coding [0016] The difference between the original samples and predicted samples of a CU is called the residual. The residual is transformed into the frequency domain using one of the transforms supported by ECM, for instance the Discrete Cosine Transform (DCT) or the Discrete Sine Transform (DST). The resulting transform coefficients are quantized and entropy coded and the resulting bits are put into the bitstream. The level of quantization depends on the QP parameter used for the CU. In some settings, all CUs belonging to the same picture will have the same QP parameter value. [0017] In-Loop Filters [0018] FIG.1 illustrates a filtering flow. Because transform coefficients are quantized, video encoding produces losses. To reduce some quantization artifacts and improve video quality, in-loop filters are introduced. In-loop filtering is done on the decoder side as well as on the encoder side to use filtered pictures for inter-prediction. There are four in-loop filters in ECM-9.0: the deblocking filter (DBF), the sample adaptive offset (SAO), the bilateral filter (BIF), and the adaptive loop filter (ALF). and ALF are supplemented by cross- component (CC) counterparts. [0019] Bilateral filter (BIF) [0020] A BIF is designed to reduce the ringing artifacts caused by quantization of high frequency coefficients. The Gaussian form of BIF is introduced in reference [1] where its ability to improve image quality is demonstrated. The use for video coding and insertion into in-loop filtering in parallel with SAO is developed in reference [2]. State-of-the-art BIF included into ECM-9.0 is presented in a standardization contribution reference [3] by Ericsson and Huawei. [0021] SAO, CC-SAO, and BIF use the samples output from the DBF as input samples, where each input sample belongs to a TU (e.g., each input sample is a member of a TU, which has two-dimensions (width, length) and is associated with