Search

CN-122027872-A - Live broadcast synchronous control method and system

CN122027872ACN 122027872 ACN122027872 ACN 122027872ACN-122027872-A

Abstract

The application discloses a live broadcast synchronous control method and a live broadcast synchronous control system. The method comprises the steps that a target client receives live broadcast data which are sent by a service end and comprise video frames and audio frames based on a CDN, an encoding time stamp based on NTP when encoding of a current video frame is completed is determined periodically, target receiving time delay of the target client is determined according to the encoding time stamp and current local time of the target client, the target receiving time delay is sent to a synchronous control module based on an RTC channel, the receiving synchronous control module receives target synchronous time delay which is sent by the RTC channel, the target synchronous time delay is determined by the synchronous control module according to the receiving time delay of a plurality of clients in a virtual room where the target client is located, a playing strategy is determined according to deviation between the target synchronous time delay and the target receiving time delay, and the live broadcast data are played according to the playing strategy in a current period. The method and the system solve the technical problem that the live broadcast progress among a plurality of clients under different network conditions is asynchronous, and the watching experience of users is affected.

Inventors

  • Sheng Jibin

Assignees

  • 湖南快乐阳光互动娱乐传媒有限公司

Dates

Publication Date
20260512
Application Date
20260224

Claims (12)

  1. 1. The live broadcast synchronous control method is applied to a target client and is characterized by comprising the following steps of: Receiving live broadcast data sent by a server based on a content distribution network, wherein the live broadcast data comprises video frames and audio frames; Periodically determining a first coding time stamp of a current video frame, and determining a target receiving time delay of the target client according to the first coding time stamp and the current local time of the target client, wherein the first coding time stamp is a time stamp of video frame coding completion based on a network time protocol; The target receiving time delay is sent to a synchronous control module based on a real-time communication channel, and the target synchronous time delay sent by the synchronous control module based on the real-time communication channel is received, wherein the target synchronous time delay is determined by the synchronous control module according to the receiving time delays of a plurality of clients in a target virtual room to which the target client belongs; And determining a playing strategy according to the time delay deviation between the target synchronous time delay and the target receiving time delay, and playing the live broadcast data according to the playing strategy in the current period.
  2. 2. The method of claim 1, wherein periodically determining the first encoding time stamp for the current video frame comprises: Analyzing the supplemental enhancement information under the condition that the current video frame carries the supplemental enhancement information, and obtaining a first coding time stamp of the current video frame; and under the condition that the current video frame does not carry the supplementary enhancement information, determining a historical video frame which is closest to the current video frame and carries the supplementary enhancement information, and predicting the first coding time stamp of the current video frame according to the second coding time stamp and the second display time stamp of the historical video frame and the first display time stamp of the current video frame.
  3. 3. The method of claim 1, wherein determining the target client's target reception delay based on the first encoded timestamp and the target client's current local time comprises: Acquiring a clock offset between a local clock of the target client and a standard network clock; And determining a first sum value of the current local time of the target client and the clock offset, determining a first difference value between the first sum value and the first coding timestamp, and taking the first difference value as the target receiving delay of the target client.
  4. 4. The method of claim 1, wherein transmitting the target receive delay to a synchronization control module based on a real-time communication channel and receiving the target synchronization delay issued by the synchronization control module based on the real-time communication channel comprises: Transmitting the target receiving time delay to the synchronous control module based on a real-time communication channel, wherein the synchronous control module operates on the server side or any client side in the target virtual room; And receiving the target synchronous time delay issued by the synchronous control module based on a real-time communication channel, wherein the target synchronous time delay is determined by the synchronous control module according to the maximum value or the target quantile in the receiving time delays of a plurality of clients in the target virtual room, or the target synchronous time delay is preset time delay set by the synchronous control module.
  5. 5. A method according to claim 3, wherein determining a play strategy from a delay deviation between the target synchronization delay and the target reception delay comprises: under the condition that the time delay deviation is smaller than a first preset time delay threshold value, determining that the playing strategy of the current period is original speed playing; under the condition that the time delay deviation is not smaller than a first preset time delay threshold value and smaller than a second preset time delay threshold value, determining a playing strategy of the current period to play the video speed change and the audio speed change without tone change, and determining a playing speed adjusting coefficient according to the time delay deviation; And under the condition that the time delay deviation is not smaller than a second preset time delay threshold value, determining the playing strategy of the current period to be frame skip playing.
  6. 6. The method of claim 5, wherein playing the live data according to the play policy during the current period comprises: Under the condition that the playing strategy is original speed playing, playing the live broadcast data according to a preset playing speed in a current period; Under the condition that the playing strategy is video speed changing and audio speed changing is matched with audio speed changing and tone-unchanged playing, adjusting the preset playing speed according to the playing speed adjusting coefficient to obtain a target playing speed, processing an audio frame in live broadcast data in a current period by utilizing a time telescoping algorithm in combination with the target playing speed, and playing a video frame and a processed audio frame in the live broadcast data according to the target playing speed in the current period; And under the condition that the playing strategy is frame skipping playing, determining a target video frame and a corresponding target audio frame, wherein the encoding timestamp of the target video frame is the second sum value of the first sum value and the target synchronous delay, and playing the live broadcast data according to a preset playing speed from the target video frame and the target audio frame in the current period.
  7. 7. The method of claim 2, wherein predicting the first encoding time stamp for the current video frame based on the second encoding time stamp and the second display time stamp for the historical video frame and the first display time stamp for the current video frame comprises: Predicting a first encoding timestamp of a current video frame according to the following formula: In the formula, 、 Representing the first encoding time stamp and the second encoding time stamp respectively, 、 Representing the first display time stamp and the second display time stamp respectively, As a cumulative time error of the periodic update, For a periodically updated play-speed-related adjustment factor, where the historical video frame is a video frame in the first period, , , Indicating a preset play speed.
  8. 8. The live broadcast synchronous control method is applied to a server and is characterized by comprising the following steps of: Acquiring live broadcast data from a live broadcast source, wherein the live broadcast data comprises video frames and audio frames; adding supplemental enhancement information for key video frames in the live broadcast data, and adding supplemental enhancement information for non-key video frames in the live broadcast data according to a preset frame interval, wherein the supplemental enhancement information comprises a first encoding time stamp based on a network time protocol when encoding of corresponding video frames is completed; The live data is sent to a plurality of clients in a target virtual room based on a content distribution network.
  9. 9. The live broadcast synchronous control method is applied to a synchronous control module, and the synchronous control module operates on a server side or any client side in a target virtual room, and is characterized by comprising the following steps: Periodically receiving time delays sent by a plurality of client sides in the target virtual room based on a real-time communication channel, wherein the receiving time delays are determined by each client side based on a first encoding time stamp based on a network time protocol when encoding of video frames in received live broadcast data is completed and local time for receiving the video frames; Determining a target synchronization delay according to the receiving delays of the plurality of clients; and transmitting the target synchronization delay to the plurality of clients based on a real-time communication channel, wherein the target synchronization delay is used for guiding each client to determine a playing strategy of the current period.
  10. 10. The live broadcast synchronous control system is characterized by comprising a server, a plurality of clients in a target virtual room and a synchronous control module running on the server or any client, wherein, The server side is used for acquiring live broadcast data from a live broadcast source, wherein the live broadcast data comprises video frames and audio frames, adding supplemental enhancement information for key video frames in the live broadcast data, and adding supplemental enhancement information for non-key video frames in the live broadcast data according to preset frame intervals, wherein the supplemental enhancement information comprises a first coding time stamp based on a network time protocol when the coding of the corresponding video frames is finished; each client is used for periodically determining a first coding time stamp of a current video frame and determining target receiving time delay of the client according to the first coding time stamp and the current local time of the client, transmitting the target receiving time delay to the synchronous control module based on a real-time communication channel and receiving the target synchronous time delay issued by the synchronous control module based on the real-time communication channel, wherein the target synchronous time delay is determined by the synchronous control module according to the receiving time delays of a plurality of clients, determining a playing strategy according to time delay deviation between the target synchronous time delay and the target receiving time delay, and playing the live broadcast data according to the playing strategy in the current period, and the playing strategy comprises one of original speed playing, video speed changing, audio speed changing, constant tone playing and frame skip playing.
  11. 11. A computer program product comprising a computer program, wherein the computer program when executed by a processor implements the live synchronization control method of any of claims 1 to 9.
  12. 12. An electronic device comprising a memory and a processor, wherein the memory has a computer program stored therein, the processor being configured to execute the live synchronization control method of any of claims 1 to 9 by means of the computer program.

Description

Live broadcast synchronous control method and system Technical Field The application relates to the technical field of streaming media transmission, in particular to a live broadcast synchronous control method and a live broadcast synchronous control system. Background With the rapid development of internet technology, in an interactive live broadcast scene, the demand of users for high-quality and low-delay real-time interactive experience is becoming stronger. In particular, in a social viewing scenario such as "watching together", the fusion of CDN (Content Delivery Network ) and RTC (Real-Time Communication, real-time communication) technologies has become an industry hotspot. The CDN can carry large-scale users to watch at the same time due to its distributed nodes and efficient caching mechanism, and is especially suitable for services with high bandwidth requirements such as live video broadcast, but the delay characteristic (usually 3-15 s) of the CDN means that live streams received by viewers are not instant, which is a frustrating problem in interactive scenarios requiring instant feedback, such as real-time chat among viewers or synchronization problems in game interaction. RTC technology is very suitable for realizing instant interaction, such as online conferences, online education and multiplayer games, with low latency (less than 400 ms) and two-way communication capability, however, the image quality of RTC streams is generally inferior to CDN streams, and the peer-to-peer communication manner is difficult to cope with large-scale concurrent viewing scenarios. The combined use of the two technologies can solve the problem of coexistence of large-scale concurrency and low-delay requirement in interactive live broadcast, but also leads to new problems, namely 'transparent to the play' and 'disordered interaction'. The former happens when audience A sees events a few seconds earlier than audience B, such as the moment of goal, and share through RTC channel, destroyed the surprise that the latter watched, the latter is that the host-side question and audience answer can't correspond in real time, cause interactive experience to decline. In the related solutions, there are obvious disadvantages such as depending on the progress of the player, the local system time or the video PTS (Presentation TIME STAMP) information/DTS (Decoding TIME STAMP) information. Because the progress of the player is affected by buffering, local time is easy to deviate, video time stamps can be reset in CDN processing to lose absolute time reference value, and simple SEI (Supplemental Enhancement Information, supplementary enhancement information) embedding can assist element synchronization in streams, but can not solve the problem of frame-level synchronization among multiple ends. In view of the above problems, no effective solution has been proposed at present. Disclosure of Invention The embodiment of the application provides a live broadcast synchronous control method and a live broadcast synchronous control system, which are used for at least solving the technical problem that the live broadcast progress of a plurality of clients under different network conditions is not synchronous, so that the watching experience of a user is affected. According to one aspect of the embodiment of the application, a live broadcast synchronous control method applied to a target client is provided, and the live broadcast synchronous control method comprises the steps of receiving live broadcast data sent by a server based on a content distribution network, periodically determining a first coding time stamp of a current video frame and determining target receiving time delay of the target client according to the first coding time stamp and the current local time of the target client, wherein the first coding time stamp is a time stamp of video frame coding completion based on a network time protocol, sending target receiving time delay to a synchronous control module based on a real-time communication channel and receiving target synchronous time delay sent by the synchronous control module based on the real-time communication channel, wherein the target synchronous time delay is determined by the synchronous control module according to the receiving time delay of a plurality of clients in a target virtual room to which the target client belongs, determining a playing strategy according to time delay deviation between the target synchronous time delay and the target receiving time delay, and playing live broadcast data according to the playing strategy in the current period. Optionally, periodically determining the first coding time stamp of the current video frame comprises analyzing the supplemental enhancement information to obtain the first coding time stamp of the current video frame when the current video frame carries the supplemental enhancement information, and determining the historical video frame which carries the supp