Search

CN-122027610-A - Real-time communication RTC method, device, system, equipment, medium and product

CN122027610ACN 122027610 ACN122027610 ACN 122027610ACN-122027610-A

Abstract

The method is applied to a sending end of the RTC, and comprises the steps of determining codec information supported by the sending end and a receiving end together in a session description protocol negotiation stage, indicating codec groups and priorities of the codec groups, wherein each codec group comprises a corresponding encoder and a corresponding decoder, selecting a first codec group in the session stage, encoding media data to be sent by the first encoder, sending the media stream obtained by encoding to the receiving end, selecting a second codec group according to codec information if a user-defined request message fed back by the receiving end is received, and sending the media stream obtained by encoding again to the receiving end after the receiving end fails to decode the media stream by the first encoder.

Inventors

  • ZHANG MINQI

Assignees

  • 支付宝(杭州)数字服务技术有限公司

Dates

Publication Date
20260512
Application Date
20260228

Claims (16)

  1. 1. A method for real-time communication RTC, the method being applied to a transmitting end of the RTC, the method comprising: In the session description protocol negotiation SDP stage, determining the codec information commonly supported by the sender and the receiver of the RTC, wherein the codec information indicates a plurality of codec groups and the priority of each codec group, and each codec group comprises an encoder and a decoder which are mutually corresponding; in the RTC session phase, Selecting a first coder-decoder group according to the priority indicated by the coder-decoder information, coding the media data to be transmitted by using a first coder in the first coder-decoder group, transmitting the media stream obtained by coding to the receiving end, If a user-defined request message fed back by the receiving end is received, selecting a second codec group according to the codec information, wherein the request message is constructed after the receiving end fails to decode the media stream by using a first decoder in the first codec group, and And after the media data is encoded again by utilizing a second encoder in the second encoder-decoder group, transmitting a media stream obtained by re-encoding to the receiving end.
  2. 2. The method of claim 1, wherein the request message is received over a media plane channel in the RTC or the request message is received over a signaling system channel in the RTC.
  3. 3. The method according to claim 1 or 2, wherein the request message comprises a preset switch flag for indicating a switch encoder; and if the user-defined request message fed back by the receiving end is received, selecting a second codec group according to the codec information, including: analyzing the received data from the receiving end, and if the analyzed data contains the preset switching mark, selecting a second codec group according to the codec information.
  4. 4. A method for real-time communication RTC, the method being applied to a receiving end of the RTC, the method comprising: Determining the codec information commonly supported by the receiving end and the sending end of the RTC in the session description protocol negotiation SDP stage, wherein the codec information indicates a plurality of codec groups and the priority of each codec group, and each codec group comprises an encoder and a decoder which are mutually corresponding; In the RTC session phase, After receiving the media stream sent by the sending end, determining a first decoder corresponding to a first encoder adopted by the media stream according to the codec information, and decoding the media stream by using the first decoder; under the condition that the decoding failure of the first decoder is determined, constructing a self-defined request message, and transmitting the request message to the transmitting end; The request message is used for triggering the sending end to switch the first encoder to a standby second encoder, and recoding is carried out by utilizing the second encoder to obtain a media stream, and the second encoder is selected based on the coder-decoder information.
  5. 5. The method of claim 4, wherein the media stream contains video frames, and wherein the first decoder fails to decode is determined by: and if the first decoder is monitored to decode the preset number of video frames which are continuously arrived, determining that the decoding of the first decoder fails.
  6. 6. The method of claim 4, wherein the media stream contains video frames, and wherein the first decoder fails to decode is determined by: if the first decoder fails to decode one or more continuous multiple video frames, sending an image loss indication PLI or a complete frame request FIR message to the sending end through a media plane channel; if the first decoder fails to decode the key frame after receiving the key frame sent by the sender in response to the PLI or FIR message, determining that the first decoder fails to decode.
  7. 7. The method of claim 4, wherein transmitting the request message to the sender comprises: transmitting the request message to the sender through a media plane channel in the RTC or, And transmitting the request message to the sending end through a signaling system channel in the RTC.
  8. 8. The method according to claim 4 or 7, wherein constructing the customized request message and transmitting the request message to the sender comprises: generating the request message containing a preset switching mark based on a feedback message format defined by a real-time transmission control protocol (RTCP), transmitting the request message to the sending end through a media plane channel in the RTC, and/or, And generating the request message containing a preset switching mark based on an application layer message format defined by a signaling system channel protocol, and transmitting the request message to the sending end through a signaling system channel in the RTC.
  9. 9. The method of claim 7, wherein constructing a custom request message and transmitting the request message to the sender comprises: firstly, transmitting the request message to the transmitting end through the media plane channel; And if the sending end still sends the media stream coded by the first coder within the preset timeout time, the request message is transmitted to the sending end again through the signaling system channel.
  10. 10. The method according to claim 4, wherein the method further comprises: and identifying a payload type field contained in a header of a received media stream sent by the sender; if the identified payload type field corresponds to the second codec set, creating a second decoder in the second codec set, and decoding the received media stream using the second decoder.
  11. 11. A transmitting apparatus in a real-time communication RTC, the apparatus comprising: a negotiation unit, configured to determine codec information commonly supported by the transmitting device and the receiving device of the RTC in a session description protocol negotiation SDP stage, where the codec information indicates a plurality of codec groups and priorities of the codec groups, and each codec group includes an encoder and a decoder that correspond to each other; And the RTC session unit is used for selecting a first codec group according to the priority indicated by the codec information in the RTC session stage, encoding the media data to be transmitted by using a first encoder in the first codec group, transmitting the media stream obtained by encoding to the receiving device, and selecting a second codec group according to the codec information if a self-defined request message fed back by the receiving device is received, wherein the request message is constructed after the receiving device fails to decode the media stream by using the first encoder in the first codec group, and transmitting the media stream obtained by encoding again to the receiving device after encoding the media data again by using a second encoder in the second codec group.
  12. 12. A receiving apparatus in a real-time communication RTC, the apparatus comprising: A negotiation unit, configured to determine codec information supported by the receiving device and the sending device of the RTC together in a session description protocol negotiation SDP stage, where the codec information indicates a plurality of codec groups and priorities of the codec groups, and each codec group includes an encoder and a decoder that correspond to each other; The RTC session unit is used for determining a first decoder corresponding to a first encoder adopted by the media stream according to the codec information after receiving the media stream sent by the sending device in an RTC session stage, decoding the media stream by using the first decoder, constructing a customized request message and transmitting the request message to the sending device under the condition that the decoding of the first decoder is determined to be failed, wherein the request message is used for triggering the sending device to switch the first encoder to a standby second encoder, and recoding the second encoder to obtain the media stream, and the second encoder is selected based on the codec information.
  13. 13. A communication system, characterized in that the system comprises a transmitting end for performing the method of any of claims 1 to 3 and/or in that the system comprises a receiving end for performing the steps of the method of any of claims 4 to 10.
  14. 14. An electronic device comprising a processor, a memory for storing processor-executable instructions, wherein the processor is configured to implement the steps of the method of any one of claims 1 to 10 by executing the executable instructions.
  15. 15. A computer readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the steps of the method of any of claims 1 to 10.
  16. 16. A computer program product comprising computer programs/instructions which, when executed by a processor, implement the steps of the method of any of claims 1 to 10.

Description

Real-time communication RTC method, device, system, equipment, medium and product Technical Field One or more embodiments of the present disclosure relate to the field of communications technologies, and in particular, to a method, an apparatus, a system, a device, a medium, and a product for real-time communication RTC. Background In a Real-time audio-video communication (Real-Time Communication, RTC) scenario, both parties may determine supported codecs and priorities through SDP negotiation. After the call is established, the sender may use the highest priority codec (e.g., h.265) coding. When network packet loss occurs, the decoder at the receiving end fails to decode, and the PLI/FIR request key frame can be sent through RTCP to recover. If the continued decoding fails, the only standard method is to initiate a time-consuming SDP renegotiation, which typically results in a communication interruption of several seconds, and the user experience is poor. Disclosure of Invention In view of this, one or more embodiments of the present disclosure provide the following technical solutions: according to a first aspect of one or more embodiments of the present specification, there is provided a method for real-time communication RTC, the method being applied to a transmitting end of the RTC, the method including: In the session description protocol negotiation SDP stage, determining the codec information commonly supported by the sender and the receiver of the RTC, wherein the codec information indicates a plurality of codec groups and the priority of each codec group, and each codec group comprises an encoder and a decoder which are mutually corresponding; in the RTC session phase, Selecting a first coder-decoder group according to the priority indicated by the coder-decoder information, coding the media data to be transmitted by using a first coder in the first coder-decoder group, transmitting the media stream obtained by coding to the receiving end, If a user-defined request message fed back by the receiving end is received, selecting a second codec group according to the codec information, wherein the request message is constructed after the receiving end fails to decode the media stream by using a first decoder in the first codec group, and And after the media data is encoded again by utilizing a second encoder in the second encoder-decoder group, transmitting a media stream obtained by re-encoding to the receiving end. According to a second aspect of one or more embodiments of the present specification, there is provided a method of real-time communication RTC, the method being applied to a receiving end of the RTC, the method including: Determining the codec information commonly supported by the receiving end and the sending end of the RTC in the session description protocol negotiation SDP stage, wherein the codec information indicates a plurality of codec groups and the priority of each codec group, and each codec group comprises an encoder and a decoder which are mutually corresponding; In the RTC session phase, After receiving the media stream sent by the sending end, determining a first decoder corresponding to a first encoder adopted by the media stream according to the codec information, and decoding the media stream by using the first decoder; under the condition that the decoding failure of the first decoder is determined, constructing a self-defined request message, and transmitting the request message to the transmitting end; The request message is used for triggering the sending end to switch the first encoder to a standby second encoder, and recoding is carried out by utilizing the second encoder to obtain a media stream, and the second encoder is selected based on the coder-decoder information. According to a third aspect of one or more embodiments of the present specification, there is provided a transmitting apparatus in a real-time communication RTC, the apparatus comprising: a negotiation unit, configured to determine codec information commonly supported by the transmitting device and the receiving device of the RTC in a session description protocol negotiation SDP stage, where the codec information indicates a plurality of codec groups and priorities of the codec groups, and each codec group includes an encoder and a decoder that correspond to each other; And the RTC session unit is used for selecting a first codec group according to the priority indicated by the codec information in the RTC session stage, encoding the media data to be transmitted by using a first encoder in the first codec group, transmitting the media stream obtained by encoding to the receiving device, and selecting a second codec group according to the codec information if a self-defined request message fed back by the receiving device is received, wherein the request message is constructed after the receiving device fails to decode the media stream by using the first encoder in the first codec group, and transmit