US-12621433-B2 - Prediction-based extrapolation of pixels for improved video compression

US12621433B2US 12621433 B2US12621433 B2US 12621433B2US-12621433-B2

Abstract

Methods and systems for generating missing reference pixels for intra prediction of coding units are described. A pattern amongst a plurality of available reference pixel samples from a set of reference pixel samples is computed. The pattern can be determined based on a computed difference between actual pixel values of the available reference pixel samples. The patterns are learned based on a comparison of the computed difference between the actual pixel values to a predetermined threshold. The unavailable pixel values are then generated based on the learned pattern. Further, one or more image effects corresponding to the available reference pixel samples are automatically replicated in the generated pixels as well.

Inventors

Sumit Johar

Assignees

XILINX, INC.

Dates

Publication Date: 20260505
Application Date: 20231220

Claims (20)

1 . A processor comprising: circuitry configured to: compute a pattern amongst a set of available reference pixel samples from a plurality of reference pixel samples; and generate pixel values for one or more unavailable reference pixel samples, based at least in part on the pattern.
2 . The processor as claimed in claim 1 , wherein the circuitry is configured to compute the pattern amongst the set of available reference pixel samples based at least in part on a computed difference between actual pixel values of the set of available reference pixel samples.
3 . The processor as claimed in claim 2 , wherein the pattern corresponds to at least one pattern type from a plurality of pattern types, and wherein the plurality of pattern types are learned at least based in part on comparison of the computed difference to a threshold.
4 . The processor as claimed in claim 1 , wherein the circuitry is configured to automatically replicate one or more image effects corresponding to the set of available reference pixel samples in the generated pixel values.
5 . The processor as claimed in claim 1 , wherein the one or more unavailable reference pixel samples correspond to a current coding unit of video data and the set of available reference pixel samples correspond to one or more neighboring coding units to the current coding unit.
6 . The processor as claimed in claim 5 , wherein the circuitry is configured to perform intra prediction for the current coding unit, based at least in part on the generated pixel values, using an angular intra prediction mode.
7 . The processor as claimed in claim 6 , wherein the circuitry is configured to encode the current coding unit of the video data based on the angular intra prediction mode.
8 . A method comprising: computing, by a prediction circuitry, a pattern amongst a set of available reference pixel samples from a plurality of reference pixel samples; and generating, by the prediction circuitry, pixel values for one or more unavailable reference pixel samples, based at least in part on the pattern.
9 . The method as claimed in claim 8 , further comprising computing, by the prediction circuitry, the pattern amongst the set of available reference pixel samples based at least in part on a computed difference between actual pixel values of the set of available reference pixel samples.
10 . The method as claimed in claim 9 , wherein the pattern corresponds to at least one pattern type from a plurality of pattern types, and wherein the plurality of pattern types are learned at least based in part on comparison of the computed difference to a threshold.
11 . The method as claimed in claim 8 , further comprising automatically replicating, by the prediction circuitry, one or more image effects corresponding to the set of available reference pixel samples in the generated pixel values.
12 . The method as claimed in claim 8 , wherein the one or more unavailable reference pixel samples correspond to a current coding unit of video data and the set of available reference pixel samples correspond to one or more neighboring coding units to the current coding unit.
13 . The method as claimed in claim 12 , further comprising performing, by the prediction circuitry, intra prediction for the current coding unit, based at least in part on the generated pixel values, using an angular intra prediction mode.
14 . The method as claimed in claim 13 , further comprising encoding, by an encoding circuitry, the current coding unit of the video data based on the angular intra prediction mode.
15 . A system comprising: processing circuitry; and encoding circuitry configured to: compute a pattern amongst a set of available reference pixel samples from a plurality of reference pixel samples; and generate pixel values for one or more unavailable reference pixel samples, based at least in part on the pattern.
16 . The system as claimed in claim 15 , wherein the encoding circuitry is configured to compute the pattern amongst the set of available reference pixel samples based at least in part on a computed difference between actual pixel values of the set of available reference pixel samples.
17 . The system as claimed in claim 16 , wherein the pattern corresponds to at least one pattern type from a plurality of pattern types, and wherein the plurality of pattern types are learned at least based in part on comparison of the computed difference to a threshold.
18 . The system as claimed in claim 15 , wherein the encoding circuitry is configured to automatically replicate one or more image effects corresponding to the set of available reference pixel samples in the generated pixel values.
19 . The system as claimed in claim 15 , wherein the one or more unavailable reference pixel samples correspond to a current coding unit of video data and the set of available reference pixel samples correspond to one or more neighboring coding units to the current coding unit.
20 . The system as claimed in claim 19 , wherein the encoding circuitry is configured to perform intra prediction for the current coding unit, based at least in part on the generated pixel values, using an angular intra prediction mode.

Description

BACKGROUND Description of the Related Art Digital video functionalities can be integrated into a wide array of devices, including digital televisions, digital direct broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, digital cameras, and similar gadgets. These digital video devices make use of video encoding methods, as outlined in the standards set by MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264/MPEG-4, Part 10, Advanced Video Coding (AVC), the High Efficiency Video Coding (HEVC) standard, and extensions of these standards. By incorporating such video encoding techniques, these devices can transmit, receive, encode, decode, and/or store digital video content in a more efficient manner. Video coding methods include spatial (intra-picture) prediction and/or temporal (inter-picture) prediction to diminish or eliminate the repetitiveness found in video sequences. In the context of block-based video coding, a video segment (for instance, a video frame or a segment of a video frame) can be divided into video units, which might also be denoted as tree blocks, coding units (CUs), or coding blocks. Spatial or temporal prediction produces a predictive block that serves as a basis for encoding a target block. The residual data represents the variations in pixel values between the original block and the predictive block. To enhance compression, the residual data can undergo a transformation from the pixel domain to a transform domain, yielding residual transformation coefficients that can subsequently be subjected to quantization. To achieve additional compression, entropy coding techniques may be employed. Intra Prediction helps in video compression by exploiting spatial correlation between the pixels within a frame. It generates predicting pixels for a coding unit from the boundary pixels belonging to one or more of its neighboring blocks (e.g., top, left, top-left, etc.). Further, intra prediction can be performed in multiple modes to generate predicted pixels, such as Direct Current (DC) Mode, Angular Modes, etc. supported by different video compression standards. When these neighboring blocks are not available, those pixels are substituted with the last available pixel. However, this may be a sub-optimal way of generating such pixels, since merely replicating unavailable pixels with available pixels does not consider the relationship between the available pixels, e.g., existing patterns such as fading or shadows. Therefore, there exists a need for an improved pixel prediction for intra prediction in coding units. BRIEF DESCRIPTION OF THE DRAWINGS The advantages of the methods and mechanisms described herein may be better understood by referring to the following description in conjunction with the accompanying drawings, in which: FIG. 1 is a block diagram of one implementation of a computing system. FIG. 2 illustrates a schematic representation of a video encoder. FIG. 3 is a conceptual illustration depicting a sample of intra prediction modes and their corresponding mode indices. FIG. 4 is an example block diagram illustrating generation of unavailable pixels using reference pixels during intra prediction for a coding unit. FIG. 5 illustrates an example block diagram showing extrapolation of unavailable pixels using available reference pixels, based on identified patterns between the available pixels. FIG. 6 illustrates an example lookup table indicating pixel availability determination. FIG. 7 illustrates an example method for generating missing reference pixel values based on available pixel values. DETAILED DESCRIPTION OF IMPLEMENTATIONS In the following description, numerous specific details are set forth to provide a thorough understanding of the methods and mechanisms presented herein. However, one having ordinary skill in the art should recognize that the various implementations may be practiced without these specific details. In some instances, well-known structures, components, signals, computer program instructions, and techniques have not been shown in detail to avoid obscuring the approaches described herein. It will be appreciated that for simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn to scale. For example, the dimensions of some of the elements may be exaggerated relative to other elements. Systems, apparatuses, and methods for generating missing pixel values for intra prediction are disclosed. When reconstructing a block that has undergone intra prediction, the decoding process involves creating a prediction block by utilizing neighboring pixels from the same block. This prediction block is then combined with the residual signal that was sent by the encoding process. To generate a prediction sample for the current block based on the chosen prediction mode, it is necessary to obtain reference samples from the neighboring pixels. In situations where certain neighboring reference pixel samp