CN-122027812-A - Interaction method and device based on screen connection, electronic equipment and storage medium

Abstract

The disclosure relates to an interaction method and device based on a continuous screen, electronic equipment and a storage medium. The method comprises the steps of responding to creation operation of a host account for a combined virtual space, creating the combined virtual space, determining a target combined account from at least one combined account, pulling multimedia data streams of the target combined accounts from a server, collecting local audio streams in real time, merging the local audio streams and the multimedia data streams of the target combined accounts according to a combined mode, obtaining a virtual space multimedia data stream, and playing the virtual space multimedia data stream in the combined virtual space. The method can improve the flexibility of the multicast operation.

Inventors

• YANG ZHAOCHENG
• TAN PEIQIANG

Assignees

• 北京达佳互联信息技术有限公司

Dates

Publication Date

20260512

Application Date

20260113

Claims (20)

1. 1. The interaction method based on the continuous screen is characterized by being applied to a first terminal and comprising the following steps of: Creating a combined virtual space in response to a creation operation of a host account for the combined virtual space, wherein the combined virtual space comprises at least one combined account; determining a target combined account from the at least one combined account, and pulling a multimedia data stream of each target combined account from a server; And collecting local audio streams in real time, converging the local audio streams and the multimedia data streams of each target combined account according to a combined mode to obtain virtual space multimedia data streams, and playing the virtual space multimedia data streams in the combined virtual space.
2. 2. The method of claim 1, wherein the target multimedia data stream comprises a video stream and an audio stream, and wherein the merging the local audio stream and the multimedia data streams of each target combined account according to the combined mode to obtain the virtual space multimedia data stream comprises: acquiring the video stream and the audio stream from the multimedia data stream aiming at the multimedia data stream of any target combined account; According to the picture layout corresponding to the combination mode, rendering video streams corresponding to the target combination accounts to obtain aggregated video streams, extracting speaking audio data from audio streams corresponding to the target combination accounts according to an audio control strategy corresponding to the combination mode, and aggregating the speaking audio data and the local audio streams to obtain aggregated audio streams; And packaging the aggregate video stream and the aggregate audio stream to obtain a virtual space multimedia data stream.
3. 3. The method according to claim 2, wherein the method further comprises: Packaging the aggregate video stream and the local audio stream to obtain the data of a director control stream, wherein the local audio stream comprises speaking data of the account of the director and background music data played by the account of the director; and sending a control instruction to the server, wherein the control instruction comprises the combined control stream data, and the control instruction is used for indicating the server to send the combined control stream data to each combined account.
4. 4. The method of claim 1, wherein pulling the multimedia data stream of each of the target synopsis accounts from the server comprises: For any target compliance account, sending a pull stream verification request for the target compliance account to a server, wherein the pull stream verification request is used for indicating the server to verify the pull stream authority of the host account; Receiving verification feedback sent by the server, wherein the verification feedback comprises a pull stream address and an aging token corresponding to the target compliance account, and the pull stream address is associated with the pull stream address of the target compliance account; And pulling the multimedia data stream of the target combined account from the server based on the pulling address and the aging token.
5. 5. The interaction method based on the continuous screen is characterized by being applied to a second terminal and comprising the following steps of: in response to a synopsis participation operation for a synopsis virtual space, adding a user account as a synopsis account into the synopsis virtual space; And collecting multimedia data streams including local audio streams and video streams in real time, and continuously pushing the collected multimedia data streams to a server side so that a host account of the concurrent virtual space pulls the multimedia data streams from the server side and merges the multimedia data streams into a virtual space multimedia data stream, and then playing the virtual space multimedia data stream in the concurrent virtual space.
6. 6. The method according to claim 5, wherein the capturing in real time the multimedia data stream including the local audio stream and the video stream and continuously pushing the captured multimedia data stream to the server comprises: a push address request is sent to a server, and the push address request is used for indicating the server to distribute a push address for the deduction account; Receiving a push address sent by the server; And acquiring the multimedia data stream comprising the local audio stream and the video stream in real time, and continuously transmitting the acquired multimedia data stream to the push stream address.
7. 7. The method according to claim 5 or 6, characterized in that the method further comprises: Receiving the director control stream data sent by the server, wherein the director control stream data comprises an aggregate video stream and an audio stream of a host account in the director virtual space, and the audio stream of the host account comprises speaking data of the host account and background music played by the host account; And playing the combined control stream data through a first showing position in the virtual space interface of the user account, and showing the currently acquired video stream through a second showing position in the virtual space interface of the user account.
8. 8. The method according to claim 5 or 6, characterized in that the method further comprises: Receiving the synopsis virtual space data sent by the server, wherein the synopsis virtual space data comprises at least one of comment data, list data and virtual resource data; And displaying the combined virtual space data in a virtual space interface of the user account.
9. 9. The interactive method based on the continuous screen is characterized by being applied to a server and comprising the following steps of: creating a synopsis virtual space in response to a creation operation for the synopsis virtual space, wherein the synopsis virtual space comprises a host account and at least one synopsis account; Receiving and storing multimedia data streams pushed by the combined accounts; And responding to a pulling request of the host account for a target concurrent account, sending a multimedia data stream of the target concurrent account to the host account, and playing the virtual space multimedia data stream in the concurrent virtual space after the multimedia data of the target concurrent account are merged into the virtual space multimedia data stream at the host account side.
10. 10. The method of claim 9, wherein receiving and storing the multimedia data streams pushed by each of the synopsis accounts comprises: Receiving a plug flow address request sent by the combined account; Distributing a plug address and a pull address for the combined account in response to the plug address request, and sending the plug address to the combined account; And receiving and storing the multimedia data stream sent by the combined account through the push address.
11. 11. The method of claim 10, wherein the sending the multimedia data stream of the target synopsis account to the moderator account in response to the pull request of the moderator account for the target synopsis account comprises: Receiving a pull flow verification request for a target combined account, which is sent by the host account; responding to the pulling flow verification request to verify the pulling flow authority of the host account, acquiring a pulling flow address corresponding to the target compliance account after verification is passed, and generating an aging token; Sending verification feedback to the host account, wherein the verification feedback comprises the pull stream address and the aging token; receiving a pulling request sent by the host account based on the pulling address, wherein the pulling request comprises the aging token; And after the aging token passes the verification, acquiring the multimedia data stream of the target combined account, and transmitting the multimedia data stream of the target combined account to the host account.
12. 12. The method according to any one of claims 9 to 11, further comprising: receiving a virtual space plug flow request sent by the host account; And responding to the virtual space plug-in request, and feeding back a virtual space plug-in address to the host account, wherein the virtual space plug-in address is used for the host account to issue a virtual space multimedia data stream to the audience account, the multimedia data stream is obtained by packaging an aggregate video stream and an aggregate audio stream, the aggregate video stream is obtained by rendering a video stream in the multimedia data stream corresponding to each target combined account based on a picture layout corresponding to a combining mode, and the aggregate audio stream is obtained by aggregating an audio stream in the multimedia data stream corresponding to each target combined account and a local audio stream of the host account.
13. 13. The method according to any one of claims 9 to 11, further comprising: Receiving the director account transmitted director control stream data, wherein the director control stream data comprises an aggregate video stream and an audio stream of a director account in the director virtual space, and the audio stream of the director account comprises speaking data of the director account and background music played by the director account; and sending the combined control stream data to each combined account.
14. 14. The method according to any one of claims 9 to 11, further comprising: and sending the combined virtual space data to each combined account, wherein the combined virtual space data comprises at least one of comment data, list data and virtual resource data, and the combined virtual space data is used for displaying in a virtual space interface of the combined account.
15. 15. The method according to any one of claims 9 to 11, further comprising: After the combined account joins the combined virtual space, a first pulling address switching instruction is sent to an audience account associated with the virtual space of the combined account, wherein the first pulling address switching instruction comprises a first pulling address corresponding to the combined virtual space, and the first pulling address switching instruction is used for indicating the audience account to switch a pulling address from a second pulling address to the first pulling address, and the second pulling address is the pulling address of the virtual space of the combined account; And after the combined account exits the combined virtual space, sending a second pulling address switching instruction to an audience account associated with the virtual space of the combined account, wherein the second pulling address switching instruction comprises the second pulling address, and the second pulling address switching instruction is used for indicating the audience account to switch the pulling address from a first pulling address to the second pulling address.
16. 16. An interactive device based on a continuous screen, which is applied to a first terminal, comprises: a creation unit configured to perform a creation operation for a synopsis virtual space in response to a moderator account, the synopsis virtual space including at least one synopsis account therein; A streaming unit configured to determine a target compliance account from the at least one compliance account, and pull multimedia data streams of each target compliance account from a server; And the stream pushing unit is configured to collect local audio streams in real time, merge the local audio streams with the multimedia data streams of each target combined account according to a combined mode to obtain virtual space multimedia data streams, and play the virtual space multimedia data streams in the combined virtual space.
17. 17. An interactive device based on a continuous screen, which is applied to a second terminal, comprises: A reference unit configured to perform reference participation operation for reference virtual space in response to reference participation operation, and to add user account as reference account into the reference virtual space; And the pushing unit is configured to collect multimedia data streams including local audio streams and video streams in real time, and continuously push the collected multimedia data streams to a server side so that a host account of the concurrent virtual space pulls the multimedia data streams from the server side and merges the multimedia data streams into a virtual space multimedia data stream, and then play the virtual space multimedia data stream in the concurrent virtual space.
18. 18. An interactive device based on linking screen, which is characterized in that it is applied to a server, comprising: a creation unit configured to perform creation of a synopsis virtual space including a moderator account and at least one synopsis account in response to a creation operation for the synopsis virtual space; a first receiving unit configured to perform receiving and storing multimedia data streams pushed by each of the group accounts; The first sending unit is configured to perform a streaming request for a target combined account in response to the host account, send a multimedia data stream of the target combined account to the host account, and play the virtual space multimedia data stream in the combined virtual space after the multimedia data of the target combined account are combined into the virtual space multimedia data stream at the host account side.
19. 19. An electronic device, comprising: A processor; a memory for storing the processor-executable instructions; Wherein the processor is configured to execute the instructions to implement the screen-connected based interaction method of any one of claims 1 to 4 or 5 to 8 or 9 to 15.
20. 20. A computer readable storage medium, characterized in that instructions in the computer readable storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the screen-based interaction method of any of claims 1 to 4 or 5 to 8 or 9 to 15.

Description

Interaction method and device based on screen connection, electronic equipment and storage medium Technical Field The disclosure relates to the technical field of computers, and in particular relates to an interaction method, device, electronic equipment and storage medium based on a screen. Background The multicast service breaks through the content limitation of single live broadcast by virtue of the characteristics of multi-user cooperation and real-time interaction, and improves the immersion and interaction willingness of users through a multi-element interaction playing method, thereby becoming one of the main modes of interaction among online users. Current multicast services rely mainly on offline operation technology modes. The hardware end needs to be equipped with professional camera shooting, lamplight, sound mixing equipment and scene building systems, the human end needs to build a main broadcasting team and a scene back supporting team such as field control, broadcasting guiding and operation, and the process end realizes content production and live broadcast execution through off-line centralized recruitment, training and field coordination. Therefore, the offline operation technology mode cannot perform real-time adjustment of the multicast service, and requires offline adaptive adjustment of at least one of a hardware end, a manpower end and a process end no matter which layer is adjusted, so that the operation flexibility is poor. Disclosure of Invention The disclosure provides an interaction method and device based on a screen connection and electronic equipment, so as to at least solve the problem of poor flexibility of an operation mode of a multicast service in the related technology. The technical scheme of the present disclosure is as follows: according to a first aspect of embodiments of the present disclosure, there is provided a screen-connection-based interaction method, applied to a first terminal, including: Creating a combined virtual space in response to a creation operation of a host account for the combined virtual space, wherein the combined virtual space comprises at least one combined account; determining a target combined account from the at least one combined account, and pulling a multimedia data stream of each target combined account from a server; And collecting local audio streams in real time, converging the local audio streams and the multimedia data streams of each target combined account according to a combined mode to obtain virtual space multimedia data streams, and playing the virtual space multimedia data streams in the combined virtual space. In one embodiment, the target multimedia data stream includes a video stream and an audio stream, and the merging the local audio stream and the multimedia data streams of each target merge account according to the merge mode to obtain a virtual space multimedia data stream includes: acquiring the video stream and the audio stream from the multimedia data stream aiming at the multimedia data stream of any target combined account; According to the picture layout corresponding to the combination mode, rendering video streams corresponding to the target combination accounts to obtain aggregated video streams, extracting speaking audio data from audio streams corresponding to the target combination accounts according to an audio control strategy corresponding to the combination mode, and aggregating the speaking audio data and the local audio streams to obtain aggregated audio streams; And packaging the aggregate video stream and the aggregate audio stream to obtain a virtual space multimedia data stream. In one embodiment, the method further comprises: Packaging the aggregate video stream and the local audio stream to obtain the data of a director control stream, wherein the local audio stream comprises speaking data of the account of the director and background music data played by the account of the director; and sending a control instruction to the server, wherein the control instruction comprises the combined control stream data, and the control instruction is used for indicating the server to send the combined control stream data to each combined account. In one embodiment, the pulling the multimedia data stream of each target group account from the server includes: For any target compliance account, sending a pull stream verification request for the target compliance account to a server, wherein the pull stream verification request is used for indicating the server to verify the pull stream authority of the host account; Receiving verification feedback sent by the server, wherein the verification feedback comprises a pull stream address and an aging token corresponding to the target compliance account, and the pull stream address is associated with the pull stream address of the target compliance account; And pulling the multimedia data stream of the target combined account from the server based on the pulling add