US-20260129187-A1 - Application accelerator

US20260129187A1US 20260129187 A1US20260129187 A1US 20260129187A1US-20260129187-A1

Abstract

A system for video encoding includes an acceleration device to select from a video stream a plurality of target video frames and one or more reference frames, to divide each of the target video frames into multiple blocks, and to compute at least a first map, including respective first motion vectors between the blocks in the a target video frame and corresponding blocks in one of the reference frames, and a second map, including respective second motion vectors between the blocks in a second target video frame and corresponding blocks in the one of the reference frames. A control unit is to encode one of the first and second target video frames based on the one of the reference frames by selecting motion vectors from among the first motion vectors in the first map or the second motion vectors in the second map.

Inventors

Dotan David Levi
Assaf Weissman
Kobi Pines
Noam Bloch
Erez Yaacov
Ariel Naftali Cohen

Assignees

MELLANOX TECHNOLOGIES, LTD.

Dates

Publication Date: 20260507
Application Date: 20251231

Claims (8)

1 . A system for video encoding, comprising: an acceleration device, to select from a video stream a plurality of target video frames, including at least first and second target video frames, and one or more reference frames to be applied in encoding the target video frames, to divide each of the target video frames into multiple blocks, and to compute at least a first map, comprising respective first motion vectors between the blocks in the first target video frame and corresponding blocks in one of the reference frames, and a second map, comprising respective second motion vectors between the blocks in the second target video frame and corresponding blocks in the one of the reference frames; and a control unit, to encode one of the first and second target video frames based on the one of the reference frames by selecting motion vectors from among the first motion vectors in the first map or the second motion vectors in the second map.
2 . The system according to claim 1 , wherein the acceleration device is to compute for the first target video frame multiple first maps with respect to two or more different reference frames.
3 . The system according to claim 1 , wherein the acceleration device is to compute first and second result maps, which comprise entries each comprising a respective first or second motion vector and respective reference scores for the motion vectors in the entries, and wherein the control unit is to encode the target video frame by selecting the motion vectors from the result maps responsively to the reference scores.
4 . The system according to claim 3 , wherein the control unit is to encode a plurality of adjacent blocks having equal motion vectors by aggregating the plurality of adjacent blocks into a larger block.
5 . A method for video encoding, comprising: selecting from a video stream a plurality of target video frames, including at least first and second target video frames, and one or more reference frames to be applied in encoding the target video frames; dividing each of the target video frames into multiple blocks; computing at least a first map, comprising respective first motion vectors between the blocks in the first target video frame and corresponding blocks in one of the reference frames, and a second map, comprising respective second motion vectors between the blocks in the second target video frame and corresponding blocks in the one of the reference frames; and encoding one of the first and second target video frames based on the one of the reference frames by selecting motion vectors from among the first motion vectors in the first map or the second motion vectors in the second map.
6 . The method according to claim 5 , wherein computing at least the first map comprises computing for the first target video frame multiple first maps with respect to two or more different reference frames.
7 . The method according to claim 5 , and comprising computing first and second result maps, which comprise entries each comprising a respective first or second motion vector and respective reference scores for the motion vectors in the entries, and wherein encoding the one of the first and second target video frames comprises selecting the motion vectors from the result maps responsively to the reference score.
8 . The method according to claim 7 , wherein encoding the one of the first and second target video frames comprises encoding a plurality of adjacent blocks having equal motion vectors by aggregating the plurality of adjacent blocks into a larger block.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS The present application is a continuation of U.S. patent application Ser. No. 18/738,013, filed Jun. 9, 2024, which is a continuation of U.S. patent application Ser. No. 16/291,023, filed Mar. 4, 2019, which claims the benefit of U.S. Provisional Patent Application 62/695,063 of Levi et al, filed 8 Jul. 2018; and U.S. Provisional Patent Application 62/726,446 of Levi et al, filed 4 Sep. 2018. FIELD OF THE INVENTION The present invention relates generally to the fields of computer architecture and application acceleration. BACKGROUND Video encoding is described herein as one example of a workload which creates heavy stresses (by way of non-limiting example, in the cloud and/or in the data center; the same applies in other equipment and/or locations that are part of the video flow). With respect to the particular non-limiting example of preparing video for an edge computing device (by way of non-limiting example, for a smart phone, a TV set etc.), data center workload includes millions of video streams which have to be encoded and re-encoded during every minute, in order to stream the streams to multiple devices (generally having different video codecs) in a very efficient way. Persons skilled in the art will appreciate that adaptive streaming additionally requires the video to be encoded to multiple bitrates. SUMMARY OF THE INVENTION Persons skilled in the art will appreciate the following: 1. Providing solutions to issues mentioned in the present application (including in the Background thereof) in exemplary embodiments of the present invention have applicability to various use cases, and not only to data center and/or cloud use cases.2. Solutions provided in exemplary embodiments of the present invention have applicability to (by way of non-limiting example): video compression; motion estimation; video deblocking filter; Current Picture Referencing (CPR) and video transform. Non-limiting examples of relevant systems to which exemplary embodiments of the present invention may be applicable include, by way of non-limiting example: HEVC/H.265 (high efficiency video coding); AVC/H.264 (advanced video coding); VP9; AV-1 (AOMedia Video 1); and VVC (versatile video coding). While, generally, uses of exemplary embodiments of the present invention are described in the context of motion estimation, such descriptions are not meant to be limiting, and persons skilled in the art will appreciate, in light of the description herein, how to provide solutions in at least the other cases mentioned above in the present paragraph. The load of a single encoding task itself is generally too big for a single CPU. Reference is now made to FIG. 9, which is a simplified graphical illustration depicting a relationship between CPU power and various encoding tasks; to FIG. 10, which is a simplified pictorial illustration depicting an approximate timeline view of various past, present, and future encoding standards; and to FIG. 11, which is a simplified tabular illustration depicting complexity of various encoding standards. FIG. 9 shows that, for quite some time, a single CPU has not been able to keep up with a single encoding task; this problem, it is believed, will become more severe in the future, as future standards are adopted and become more widely used (see FIG. 10) and as the complexity of those future standards is expected to be higher, as shown in FIG. 11. In addition, the amount of information/data in every stream is increasing, as more and more information/data is produced and streamed into the edge devices at higher resolution. Reference is now additionally made to FIG. 12, which is a simplified partly pictorial, partly graphical illustration depicting relationships between resolution and complexity. FIG. 12 show a very conservative forecast of resolution mixture along the years, and also shows the nominal complexity involved with the fact that there are more pixels in every stream. Reference is now additionally made to FIG. 13, which is a simplified graphical illustration depicting relationships between video complexity, CPU capability, and computational effort. FIG. 13 shows that the real video complexity (complexity per pixel X number of pixels) is suffering from a growing gap between CPU capability and the computational effort needed. The consideration of the problem presented up to this point does not include the major increase, which is expected to continue, in the number of video streams that must be simultaneously processed in the data center. Having considered the above, it is fair to ask why the art, as known to the inventors of the present invention, does not yet include any acceleration device offering video encoding acceleration for the data center. Without limiting the generality of the present invention, the term “device” may be used in the present description to describe implementations of exemplary embodiments of the present invention, as well as (in the preceding s