CN-121985177-A - Low latency video streaming-modified frame rate

Abstract

The present disclosure relates to low latency video streaming-modified frame rates. In an example, a device may include logic to receive a plurality of video frames from a video source, logic to process the plurality of video frames, logic to provide at least some of the plurality of video frames to a video receiver, and logic to reduce a display latency of each of the video frames provided to the video receiver by changing a frame rate of some or all of the frames.

Inventors

• RAJESH MAMIDWAR
• CHEN XUEMIN
• BRIAN HENG

Assignees

• 安华高科技股份有限公司

Dates

Publication Date

20260505

Application Date

20251020

Priority Date

20241030

Claims (20)

1. 1. An apparatus, comprising: Logic for receiving a plurality of video frames from a video source at a fixed frame rate, the plurality of received video frames comprising a first plurality of subframes; Logic for processing at least some of the plurality of video frames to generate a plurality of processed video frames, wherein: Processing at least some of the plurality of video frames includes processing at least some of the first plurality of subframes, and Processing at least some of the plurality of video frames to decouple the fixed frame rate of the received video frames from a flexible frame rate of the processed video frames, and Logic for providing at least some of the plurality of processed video frames including at least some of the processed first plurality of subframes to a video receiver at the flexible frame rate.
2. 2. The apparatus of claim 1, wherein: processing at least some of the plurality of video frames includes encoding the at least some of the plurality of video frames with a video encoder/decoder CODEC for transmission over a network.
3. 3. The apparatus of claim 1, wherein: The received plurality of video frames comprising a plurality of encoded video frames encoded with a video CODEC, and Processing at least some of the plurality of video frames includes: the at least some of the plurality of video frames are decoded to produce a plurality of decoded video frames.
4. 4. A device according to claim 3, wherein: providing each of the plurality of processed frames at the flexible frame rate includes: providing a first set of the plurality of decoded video frames to the video receiver via a dedicated multimedia interface, and A second set of the plurality of decoded video frames is provided to the video receiver via a local area network.
5. 5. The apparatus of claim 4, wherein: providing at least some of the processed first plurality of subframes to the video receiver includes: not all of the processed first plurality of subframes are provided to the video receiver.
6. 6. The apparatus of claim 1, wherein: Processing at least some of the first plurality of subframes includes: processing not all of the first plurality of subframes.
7. 7. The apparatus of claim 1, wherein: providing at least some of the processed first plurality of subframes to the video receiver includes: at least some of the processed subframes are provided as part of separate streams.
8. 8. The apparatus of claim 1, wherein: providing at least some of the processed first plurality of subframes to the video receiver includes: the processed subframes are provided as part of a single stream.
9. 9. The apparatus of claim 1, wherein: providing the plurality of processed video frames to the video receiver includes: generating a processed video frame from at least some of the processed first plurality of subframes and a previous processed video frame, and The processed video frames are provided to the video receiver.
10. 10. The apparatus of claim 1, wherein: Decoding at least some of the plurality of encoded video frames to generate a plurality of decoded video frames includes: two or more of the first plurality of subframes are decoded in parallel.
11. 11. The apparatus of claim 1, wherein: Receiving the plurality of encoded video frames from the video source includes: at least some of the first plurality of subframes are received as separate streams.
12. 12. The apparatus of claim 1, wherein: receiving the first plurality of encoded video frames from the video source includes: The first plurality of subframes is received as a single stream.
13. 13. The apparatus of claim 1, wherein: receiving the plurality of video frames from the video source includes: receiving a full video frame; processing at least some of the plurality of video frames includes: Dividing the full video frame into a second plurality of subframes, and Providing each of the plurality of processed video frames to the video receiver at the flexible frame rate includes: At least some of the second plurality of subframes are provided to the video receiver.
14. 14. The apparatus of claim 1, wherein: Processing at least some of the plurality of received video frames includes: processing the at least some of the plurality of received video frames at a rate higher than the fixed frame rate, and Providing each of the plurality of processed video frames to the video receiver at the flexible frame rate includes: Each of the plurality of processed video frames is provided to the video receiver at the rate at which each respective frame has been processed and no post-processing buffering is required.
15. 15. The apparatus of claim 1, wherein the apparatus comprises a set-top box, a component of a set-top box, or a system on a chip SoC.
16. 16. The apparatus of claim 1, wherein the apparatus is a television.
17. 17. The apparatus of claim 1, wherein: Each of the received plurality of frames includes a decoded picture having a first picture size; Each of the processed video frames includes a display picture having a second picture size smaller than the first picture size, the display picture including a portion of the decoded picture, and The portion of the decoded picture comprised by the display picture comprises a change between subsequent frames.
18. 18. The device of claim 1, further comprising: Logic for selectively disabling the decoupling of the fixed frame rate from the flexible frame rate and providing the processed video frames at the flexible frame rate based on: An application associated with the video frame; Configuration setting or And (5) controlling a user.
19. 19. A method, comprising: receiving a plurality of video frames from a video source at a fixed frame rate, the plurality of received video frames comprising a first plurality of subframes; processing at least some of the plurality of video frames to produce a plurality of processed video frames, wherein: Processing at least some of the plurality of video frames includes processing at least some of the first plurality of subframes, and Processing at least some of the plurality of video frames to decouple the fixed frame rate of the received video frames from a flexible frame rate of the processed video frames, and At least some of the plurality of processed video frames including at least some of the processed first plurality of subframes are provided to a video receiver at the flexible frame rate.
20. 20. A set top box, comprising: an input interface configured to receive a plurality of video frames from a video source at a fixed frame rate, the plurality of received video frames comprising a first plurality of subframes; a processor configured to process at least some of the plurality of video frames to generate a plurality of processed video frames, wherein: Processing at least some of the plurality of video frames includes processing at least some of the first plurality of subframes, and Processing at least some of the plurality of video frames decouples the fixed frame rate of the received video frames from a flexible frame rate of the processed video frames; An output interface configured to provide at least some of the processed video frames including at least some of the plurality of processed subframes to a video receiver at the flexible frame rate.

Description

Low latency video streaming-modified frame rate Cross reference to related applications This disclosure may be related to U.S. patent application Ser. No. 18/932,328 (attorney docket No. 5009.220197US01) entitled "Low latency video stream-reduced frame buffer (Low Latency Video Streaming-Reducing Frame Buffers)" and U.S. patent application Ser. No. 18/932,361 (attorney docket No. 5009.220197US02) entitled "Low latency video stream-partial frame (Low Latency Video Streaming-PARTIAL FRAMES"), each filed concurrently herewith by its inventor. The respective disclosures of each of these applications are incorporated herein by reference for all purposes. Technical Field This document relates generally to video streaming and, more particularly, to low latency video streaming by reducing the display latency of video frames provided to a video receiver. Background Traditional video streaming, whether broadcast or IP-based, has in the past relied on complex network infrastructure to provide a smooth video experience to end users. This typically involves the video player decoding each video and audio frame and sending it timely to the TV. Such conventional approaches rely on complex network architectures and extensive buffering to ensure consistent frame delivery. This generally results in higher costs as bandwidth and storage requirements increase. To address these challenges, various video coding standards (e.g., MPEG2, AVC, HEVC, VP, AV 1) have been developed for efficient compression, but this is typically at the cost of increased computational complexity. Furthermore, to ensure smooth video playback, buffering mechanisms have been implemented at different stages of the video pipeline, including cloud servers, networks, and video players (e.g., set top boxes or on-the-fly (OTT) clients). For example, popular streaming services like YouTube and Netflix typically buffer 10 to 40 seconds of video frames. Emerging applications such as cloud gaming, video conferencing, and virtual reality require low latency performance. These applications are pushing new network, encoder and system standards to evolve. As network speeds increase, cloud games have begun to become popular. In this model, the actual game server resides in the cloud, while the local game controller sends commands to the cloud server. The game is presented on a cloud server and the encoded video is transmitted through a video pipeline to the user's device (e.g., TV or set-top box) for display. In addition, video conferencing applications have become an indispensable need for tele-office, online education, and telemedicine. Low latency is critical to a seamless experience. Many users are turning to OTT devices or set-top boxes for larger screen displays rather than traditional conferencing equipment. Furthermore, virtual reality experiences typically require high resolution video and low latency. Cloud-based rendering may provide the required processing power, while local devices may focus on displaying rendered content. For applications such as gaming, video conferencing, and virtual reality and the like, end-to-end latency is more important than fluent video. Conventional set-top boxes and OTT devices designed for streaming media video typically rely on fixed frame rates and buffering at multiple stages of the video pipeline. In other words, the video pipeline typically includes frame buffers at different stages, such as encoder, decoder, video processing, high Definition Multimedia Interface (HDMI) input, and within the TV itself. This buffering ensures smooth playback but also causes delay because each stage buffers multiple frames to ensure that full frame data is available at the input before feeding into the output stage. Conventional pipelines typically need to transmit an entire frame regardless of the number of pixels that have changed. This approach also introduces a significant amount of latency that is not desirable for latency sensitive applications such as cloud gaming, video conferencing, and virtual reality. In other examples, at the HDMI interface, the frame rate is typically fixed and cannot be dynamically adjusted during playback. For example, in the 60 FPS configuration, one full frame of data must be sent every 1/60 second. If the next frame is not ready in time (i.e., an underflow condition), then the previous frame is repeated, resulting in potential visual artifacts. This fixed frame rate requirement may limit the ability to reduce latency in applications requiring real-time responses. Disclosure of Invention In one aspect, the present disclosure provides an apparatus comprising logic for receiving a plurality of video frames from a video source at a fixed frame rate, the plurality of received video frames comprising a first plurality of subframes, logic for processing at least some of the plurality of video frames to produce a plurality of processed video frames, wherein processing at least some of the plurality of video frames co