Search

CN-122027763-A - Virtual reality picture sharing method and device

CN122027763ACN 122027763 ACN122027763 ACN 122027763ACN-122027763-A

Abstract

The application discloses a virtual reality picture sharing method and device, relates to the field of data processing, and is used for solving the problem that an existing conference system is incompatible with VR equipment. The method comprises the steps of entering a sharing conference room of a user based on a webpage conference function integrated by a VR application, enabling the VR application to be any application in VR equipment, creating a virtual camera based on a Unity engine after receiving a picture sharing instruction, obtaining a sharing picture appointed by the user based on the virtual camera, sending the sharing picture to a server based on webpage real-time communication WebRTC, and enabling a virtual participant participating in the sharing conference room to share the sharing picture to the sharing conference room through a non-interface browser in the server. Through the scheme, the VR equipment can be compatible in the existing conference system, and the picture sharing is realized.

Inventors

  • LIU WENBO
  • YAO HUIYA

Assignees

  • 深圳前海微众银行股份有限公司

Dates

Publication Date
20260512
Application Date
20260210

Claims (15)

  1. 1. A virtual reality picture sharing method, applied to a virtual reality VR device, comprising: based on a webpage conference function integrated by VR application, entering a shared conference room of a user, wherein the VR application is any application in the VR equipment; after receiving the picture sharing instruction, creating a virtual camera based on the Unity engine; Acquiring the sharing picture designated by the user based on the virtual camera; The method comprises the steps of sending a sharing picture to a server based on web page real-time communication WebRTC, wherein a virtual participant participating in the sharing conference room is arranged in the server, and the virtual participant shares the sharing picture to the sharing conference room through a non-interface browser in the server.
  2. 2. The method of claim 1, wherein the web-based real-time communication WebRTC sends the shared picture to a server, comprising: Sending a first connection request to the server based on web page real-time communication WebRTC, wherein the first connection request is JavaScript object notation text; And after receiving the confirmation message of the server, sending the sharing picture to the server.
  3. 3. The method of claim 2, wherein the sending the shared picture to the service comprises: And after each interactive connection establishment candidate address is generated, the sharing picture is sent to the server through the interactive connection establishment candidate address.
  4. 4. The method of claim 1, wherein before the web-based real-time communication WebRTC sends the shared picture to a server, further comprising: And determining the coding parameters of the WebRTC based on the hardware performance of the VR device, wherein the coding parameters comprise at least one of video frame rate, code rate and audio track switch.
  5. 5. The method of any one of claims 1 to 4, further comprising: if a labeling request of a user is received, creating a transparent canvas on the sharing picture; obtaining the mark of the user on the canvas; converting the marked coordinates from three-dimensional coordinates to two-dimensional coordinates based on a perspective projection algorithm; Determining an offset of the two-dimensional coordinates relative to the center of the canvas; Determining a texture array corresponding to the annotation based on the offset; and sending the texture array corresponding to the label to the server.
  6. 6. The method of claim 5, wherein the sending the texture array corresponding to the callout to the server comprises: Based on a user datagram protocol UDP, the texture array corresponding to the label is sent to different participants in the shared conference room through the server.
  7. 7. The virtual reality picture sharing method is characterized by being applied to a server and comprising the following steps of: Calling an interface-free browser positioned in the server to create a browser instance for the VR application based on triggering of a user on a webpage conference function integrated with the virtual reality VR application; Receiving a sharing picture from the VR device based on web page real-time communication WebRTC; And joining a shared conference room with the identity of the virtual participant, and sharing the shared picture to the shared conference room through the browser instance, wherein the shared conference room is a conference participated by the user.
  8. 8. The method of claim 7, wherein joining the shared conference room with the identity of the virtual participant comprises: determining a unique identification of a virtual participant as a shared source characterizing the user; in audio silence mode, the automated joining of the shared conference room.
  9. 9. The method of claim 7, wherein the sharing the shared screen to the shared conference room through the browser instance comprises: Based on the selective forwarding function, the shared picture is sent to different participants in the shared conference room at different code rates through the browser instance.
  10. 10. The method of claim 7, wherein the web-based real-time communication WebRTC receives a shared picture from the VR device, comprising: receiving a first connection request of the virtual reality equipment based on web page real-time communication WebRTC, wherein the first connection request is JavaScript object representation text, and the first connection request comprises authentication parameters; And authenticating the validity of the VR equipment based on the authentication parameter in the first connection request, and receiving a sharing picture from the VR equipment after transmitting confirmation information to the VR equipment after passing authentication.
  11. 11. The method of claim 10, wherein the receiving the shared picture from the VR device comprises: And after each interactive connection establishment candidate address is generated, receiving a sharing picture from the VR equipment through the interactive connection establishment candidate address.
  12. 12. The method of claim 10, wherein the sharing the shared screen to the shared conference room through the browser instance comprises: The sharing picture is sent to the browser instance based on a first mapping table, wherein the first mapping table comprises mapping relations between VR equipment and the browser instance; after the authentication is passed and the acknowledgement information is sent to the VR equipment, the method further comprises the following steps: And if the heartbeat time stamp of the VR equipment is larger than the set threshold value, canceling the connection with the VR equipment based on a second mapping table, wherein the second mapping table comprises the mapping relation between the VR equipment and the heartbeat time stamp.
  13. 13. The method of claim 7, wherein the web-based real-time communication WebRTC receives a shared picture from the VR device, comprising: Adjusting the size of a receiving buffer zone of the browser based on feedback information of a real-time transmission control protocol; and the browser instance packages the video stream corresponding to the shared picture as a local video input source and transmits the video stream to the shared conference room.
  14. 14. A virtual reality picture sharing apparatus comprising a processor coupled to a memory, the memory for storing a computer program or instructions, the processor for executing the computer program or instructions to implement the method of any of claims 1 to 13.
  15. 15. A computer program product, characterized in that it, when run on a computer, causes the computer to perform the method according to any one of claims 1 to 13.

Description

Virtual reality picture sharing method and device Technical Field The present application relates to the field of data processing, and in particular, to a method and apparatus for sharing virtual reality frames. Background In the prior art, when conference picture sharing is performed, a three-dimensional voice SDK or an avatar software SDK can be provided by manufacturers such as Pico, quest and the like based on a Virtual Reality (VR) platform native conference software development kit (software development kit, SDK), so that VR application invokes platform side capability to realize conference functions such as voice call, avatar on-screen and the like. However, the meeting system and the account system of the scheme are highly dependent on the ecology of a specific VR platform, and only the intercommunication between users on the same platform is usually supported, so that the meeting system and the account system of an enterprise (such as Jitsi, zoom, teams and the like) are difficult to carry out compatible access or unified management, and the applicability of the meeting system and the account system in an enterprise-level collaborative office scene is limited. Furthermore, VR devices in the prior art often have difficulty achieving stable and controllable picture sharing when conducting teleconferencing. Therefore, how to be compatible with VR devices in existing conference systems to achieve stable and reliable picture sharing is a problem that needs to be solved at present. Disclosure of Invention The application provides a virtual reality picture sharing method and device, which are used for being compatible with VR equipment in the existing conference system so as to realize stable and reliable picture sharing. In a first aspect, the present application provides a virtual reality picture sharing method applied to a virtual reality VR device, the method including entering a sharing conference room of a user based on a VR application integrated web conference function; the VR application is any application in the VR device, after receiving a picture sharing instruction, a virtual camera is created based on a Unity engine, a sharing picture appointed by a user is obtained based on the virtual camera, the sharing picture is sent to a server based on web page real-time communication WebRTC, a virtual participant participating in a sharing conference room is arranged in the server, and the virtual participant shares the sharing picture to the sharing conference room through a non-interface browser in the server. Through the scheme, the sharing picture at the VR equipment side is introduced into the conference room in which the user actually participates in a mode of being identifiable by the webpage conference system by means of 'VR end directional capturing+server end virtual participant injection', so that availability and compatibility of VR picture sharing are realized. Specifically, firstly, a web conference function integrated based on VR application enters a shared conference room, so that a VR end can directly join a conference participated by a user and multiplexing room management and a conference mechanism of an existing conference platform, secondly, a virtual camera is created based on Unity and a shared picture appointed by the user is obtained, a shared source is converted into a target picture for directional capture from 'VR full view or whole screen capture', thereby improving controllability of shared content, reducing information leakage risk, finally, a virtual participant participated in the shared conference room is arranged on a server side, the shared picture is shared to the shared conference room through a non-interface browser in the server, so that the server side can complete conference participation and shared picture release in a meeting system approved participant form, reduce transformation requirements on conference system bodies, improve compatibility and deployment flexibility with different conference platform forms, and realize stable and reliable picture sharing by VR equipment in the existing conference system. Optionally, the method for sending the sharing picture to the server based on the web page real-time communication WebRTC includes the steps of sending a first connection request to the server based on the web page real-time communication WebRTC, wherein the first connection request is JavaScript object representation French text, and sending the sharing picture to the server after receiving a confirmation message of the server. Thus, by replacing the native extensible message and presence protocol binary protocol with the lightweight JavaScript object notation French text protocol, the handshake complexity and handshake time can be reduced. Optionally, sending the shared frame to the service includes sending the shared frame to the server via the interactive connection establishment candidate address after each interactive connection establishment cand