Search

KR-102961996-B1 - METHOD, SYSTEM, AND COMPUTER READABLE RECORD MEDIUM TO MINIMIZE DELAY IN REAL-TIME LIVE STREAMING

KR102961996B1KR 102961996 B1KR102961996 B1KR 102961996B1KR-102961996-B1

Abstract

A method, system, and computer-readable recording medium for minimizing latency in real-time live streaming are disclosed. At least two groups of pictures (GOPs) are cached for content received from a streamer in a real-time live streaming environment, and in response to a request from a client, the GOPs completed through the cache are delivered to the client as packets starting with an I-frame.

Inventors

  • 김영환
  • 최해성
  • 조희성

Assignees

  • 라인플러스 주식회사

Dates

Publication Date
20260508
Application Date
20200903

Claims (20)

  1. Regarding the method of execution on a server implemented on a computer, The above server includes at least one processor configured to execute computer-readable instructions contained in memory, and The above method is, The step of caching at least two groups of pictures (GOPs) for content received from a streamer in a real-time live streaming environment by the at least one processor; and A step of transmitting to the client, from the GOP completed through the cache as a packet starting with an I-frame in response to a client request, by the at least one processor above. Includes, The GOP cache size is determined based on the network status between the server and the client, but, If there are multiple completed GOPs through the above cache, the GOP to be delivered is determined among the multiple completed GOPs based on the network status between the server and the client and the time of connection of the client, and The above-mentioned transmitting step is, If the network bandwidth is above a certain threshold at the time of connection of the client, the most recently completed GOP among the plurality of completed GOPs is determined as the GOP to be delivered; and if the network bandwidth is below the certain threshold at the time of connection of the client, the GOP completed before the most recently completed GOP among the plurality of completed GOPs is determined as the GOP to be delivered. A method including
  2. delete
  3. In paragraph 1, The above-mentioned transmitting step is, A step of clearing some frames and delivering the remaining frames based on the GOP reception status of the above client. A method including
  4. Regarding the method of execution on a server implemented on a computer, The above server includes at least one processor configured to execute computer-readable instructions contained in memory, and The above method is, The step of caching at least two groups of pictures (GOPs) for content received from a streamer in a real-time live streaming environment by the at least one processor; and A step of transmitting to the client, from the GOP completed through the cache as a packet starting with an I-frame in response to a client request, by the at least one processor above. Includes, The GOP cache size is determined based on the network status between the server and the client, but, If there are multiple completed GOPs through the above cache, the GOP to be delivered is determined among the multiple completed GOPs based on the network status between the server and the client and the time of connection of the client, and The above-mentioned transmitting step is, A step of clearing some frames and delivering the remaining frames based on the GOP reception status of the above client. Includes, The step of transmitting the remaining frames mentioned above is, If it is determined that the completed GOP is not received normally at the above client side, a step of erasing a frame of a first length from the GOP to be transmitted to the above client side; If it is determined that the client side cannot normally receive the GOP in which the frame of the first length has been erased, the step of erasing a frame of a second length greater than the first length from the GOP to be transmitted to the client side; and The step of delivering the GOP with the frame of the second length erased to the client. A method including
  5. Regarding the method of execution on a server implemented on a computer, The above server includes at least one processor configured to execute computer-readable instructions contained in memory, and The above method is, The step of caching at least two groups of pictures (GOPs) for content received from a streamer in a real-time live streaming environment by the at least one processor; and A step of transmitting to the client, from the GOP completed through the cache as a packet starting with an I-frame in response to a client request, by the at least one processor above. Includes, The GOP cache size is determined based on the network status between the server and the client, but, If there are multiple completed GOPs through the above cache, the GOP to be delivered is determined among the multiple completed GOPs based on the network status between the server and the client and the time of connection of the client, and The above-mentioned transmitting step is, A step of clearing some frames and delivering the remaining frames based on the GOP reception status of the above client. Includes, A method for adjusting the length of some frames to be erased according to the monitoring cycle of the GOP reception status of the above client.
  6. delete
  7. In paragraph 1, The above caching step is, A step of finding audio segments corresponding to GOPs through analysis of the video and audio streams of the above content and aligning the video and audio data to the timeline. A method including
  8. In paragraph 1, The above caching step is, A step of analyzing data types for each video frame and grouping video and audio data by matching each I-frame with the audio timestamp corresponding to that I-frame. A method including
  9. In paragraph 1, The above-mentioned transmitting step is, Reconstructing and transmitting video and audio data in a form where packet-unit data intersects A method characterized by
  10. In Paragraph 9, The number of packet crossovers is determined based on the size of the video data and the audio data. A method characterized by
  11. A computer-readable recording medium having a program recorded thereon for executing the method of any one of paragraphs 1, 3 through 5 and 7 through 10 on a computer.
  12. In a server implemented on a computer, At least one processor configured to execute computer-readable instructions contained in memory Includes, The above-mentioned at least one processor is, A process of caching at least two GOPs for content received from a streamer in a real-time live streaming environment; and The process of delivering to the client, starting with the GOP completed via cache, as a packet initiated by an I-frame in response to a client's request. Process, The GOP cache size is determined based on the network status between the server and the client, but, If there are multiple completed GOPs through the above cache, the GOP to be delivered is determined among the multiple completed GOPs based on the network status between the server and the client and the time of connection of the client, and The above-mentioned at least one processor is, A server that, if the network bandwidth is above a certain threshold at the time of connection of the client, determines the most recently completed GOP among the plurality of completed GOPs as the GOP to be delivered, and if the network bandwidth is below the certain threshold at the time of connection of the client, determines the GOP completed before the most recently completed GOP among the plurality of completed GOPs as the GOP to be delivered.
  13. delete
  14. In Paragraph 12, The above-mentioned at least one processor is, Transmitting the remaining frames after discarding some frames based on the GOP reception status of the above client A server featuring
  15. In a server implemented on a computer, At least one processor configured to execute computer-readable instructions contained in memory Includes, The above-mentioned at least one processor is, A process of caching at least two GOPs for content received from a streamer in a real-time live streaming environment; and The process of delivering to the client, starting with the GOP completed via cache, as a packet initiated by an I-frame in response to a client's request. Process, The GOP cache size is determined based on the network status between the server and the client, but, If there are multiple completed GOPs through the above cache, the GOP to be delivered is determined among the multiple completed GOPs based on the network status between the server and the client and the time of connection of the client, and The above-mentioned at least one processor is, Depending on the GOP reception status of the above client, some frames are erased and the remaining frames are delivered, and The above-mentioned at least one processor is, If it is determined that the completed GOP is not received normally by the above client side, frames of a first length are erased from the GOP to be transmitted to the above client side, and If it is determined that the client side cannot normally receive the GOP in which frames of the first length have been erased, frames of a second length greater than the first length are erased from the GOP to be transmitted to the client side, and the GOP in which frames of the second length have been erased is delivered to the client. A server featuring
  16. In a server implemented on a computer, At least one processor configured to execute computer-readable instructions contained in memory Includes, The above-mentioned at least one processor is, A process of caching at least two GOPs for content received from a streamer in a real-time live streaming environment; and The process of delivering to the client, starting with the GOP completed via cache, as a packet initiated by an I-frame in response to a client's request. Process, The GOP cache size is determined based on the network status between the server and the client, but, If there are multiple completed GOPs through the above cache, the GOP to be delivered is determined among the multiple completed GOPs based on the network status between the server and the client and the time of connection of the client, and The above-mentioned at least one processor is, Depending on the GOP reception status of the above client, some frames are erased and the remaining frames are delivered, and The length of some frames to be erased is adjusted according to the monitoring cycle of the GOP reception status of the above client. A server featuring
  17. delete
  18. In Paragraph 12, The above-mentioned at least one processor is, Finding audio segments corresponding to GOPs through analysis of the video and audio streams of the above content and aligning the video and audio data to the timeline A server featuring
  19. In Paragraph 12, The above-mentioned at least one processor is, Analyzing data types for each video frame and grouping video and audio data by matching each I-frame with the audio timestamp corresponding to that I-frame. A server featuring
  20. In Paragraph 12, The above-mentioned at least one processor is, Reconstructing and transmitting video and audio data in a form where packet-unit data intersects A server featuring

Description

Method, system, and computer-readable record medium for minimizing delay in real-time live streaming The following description concerns technology that improves the quality experienced by users in a real-time live streaming environment. Real-time live streaming service can refer to a technology that processes various transmitted multimedia data like a ceaseless and continuous flow of water. Real-time live streaming technology is becoming increasingly important with the growth of the internet, and one of the reasons for this is that most users do not have internet connections fast enough to download large amounts of content instantly. In such situations, if streaming technology is used, the client player can start displaying the data even before the entire file is transmitted. For example, Korean Published Patent Application No. 10-2006-0068547 (published on June 21, 2006) discloses a technology that provides a real-time streaming service considering the transmission rate between a streaming server and a client. FIG. 1 is a drawing illustrating an example of a network environment according to an embodiment of the present invention. FIG. 2 is a block diagram illustrating the internal configuration of an electronic device and a server in an embodiment of the present invention. FIG. 3 is a diagram illustrating an example of a real-time live streaming environment according to an embodiment of the present invention. FIG. 4 is a block diagram illustrating an example of a component that may be included in a processor of a server according to an embodiment of the present invention. FIG. 5 is a flowchart illustrating an example of a method that a server can perform according to an embodiment of the present invention. FIGS. 6 to 10 are exemplary drawings for explaining the GOP delivery process through a stream cache in an embodiment of the present invention. FIG. 11 is an example drawing for explaining the process of matching and storing video and audio in an embodiment of the present invention. FIG. 12 is an example diagram illustrating the process of reconstructing and transmitting video and audio in packet units in an embodiment of the present invention. Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. Embodiments of the present invention relate to technology that supports optimal Quality of Experience (QoE) and Quality of Service (QoS) in a real-time live streaming environment. Embodiments including those specifically disclosed in this specification can minimize the delay time from the moment content on the platform is selected until it is displayed on the player's actual screen, which is one of the important qualities perceived by a user in a real-time live streaming platform. FIG. 1 is a diagram illustrating an example of a network environment according to an embodiment of the present invention. The network environment of FIG. 1 is an example including a plurality of electronic devices (110, 120, 130, 140), a plurality of servers (150, 160), and a network (170). FIG. 1 is an example for explaining the invention, and the number of electronic devices or servers is not limited to that shown in FIG. 1. Multiple electronic devices (110, 120, 130, 140) may be fixed terminals or mobile terminals implemented as computer systems. Examples of multiple electronic devices (110, 120, 130, 140) include smartphones, mobile phones, navigation systems, computers, laptops, digital broadcasting terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), tablet PCs, game consoles, wearable devices, IoT (Internet of Things) devices, VR (Virtual Reality) devices, AR (Augmented Reality) devices, etc. For example, FIG. 1 shows the shape of a smartphone as an example of an electronic device (110), but in embodiments of the present invention, the electronic device (110) may refer to one of various physical computer systems capable of communicating with other electronic devices (120, 130, 140) and/or servers (150, 160) through a network (170) using a substantially wireless or wired communication method. The communication method is not limited and may include not only communication methods utilizing communication networks that the network (170) may include (e.g., mobile communication networks, wired internet, wireless internet, broadcasting networks, satellite networks, etc.) but also short-range wireless communication between devices. For example, the network (170) may include any one or more networks such as a PAN (personal area network), LAN (local area network), CAN (campus area network), MAN (metropolitan area network), WAN (wide area network), BBN (broadband network), and the Internet. Additionally, the network (170) may include any one or more network topologies such as a bus network, star network, ring network, mesh network, star-bus network, tree or hierarchical network, but is not limited thereto. Each of the servers (150, 160) may be imp