US-12627718-B2 - Virtual interaction session to facilitate augmented reality based communication between multiple users

US12627718B2US 12627718 B2US12627718 B2US 12627718B2US-12627718-B2

Abstract

Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, method, and user interface to facilitate augmented reality based communication between multiple users over a network. A first user of a first device is enabled to view a real-world environment that is visible to a second user via a second device by causing display, at the first device, of a live camera feed generated at the second device. The live camera feed comprises images of the real-world environment that is visible to the second user. Input data indicative of a selection by the first user of a virtual content item to apply to the real-world environment that is visible to the second user is received. The first device and second device present media objects overlaid on the real-world environment based on the input data.

Inventors

Brian Anthony SMITH
Yu Jiang Tham
Rajan Vaish
Hemant Surale

Assignees

SNAP INC.

Dates

Publication Date: 20260512
Application Date: 20240611

Claims (20)

1 . A method comprising: initiating a virtual interaction session with a first device of a first user based on session configuration data received from the first device, the session configuration data including one or more configuration parameters of the virtual interaction session specified by the first user; establishing, prior to expiration of a session duration for the virtual interaction session, a first real-time communication link between the first user and a second user by causing display, at a second device of the second user, of a live camera feed generated at the first device, the live camera feed comprising images of a real-world environment that is visible to the first user via the first device; receiving, from the second device, input data indicative of a selection by the second user of a virtual content item to apply to the real-world environment that is visible to the first user, the virtual content item comprising one or more media objects; based on the second input data, causing display, by the first device and second device, of the one or more media objects overlaid on the real-world environment; terminating the first real-time communication link between the first user and the second user; upon termination of the first real-time communication link between the first user and the second user, providing an interactive element to the second device to trigger transmission of a request for a second real-time communication link between the first user and the second user; and establishing the second real-time communication link between the first user and the second user in response to receiving the request prior to expiration of the session duration.
2 . The method of claim 1 , wherein causing display, by the second device, of the one or more media objects comprises causing display of the live camera feed augmented to include the one or more media objects overlaid on the real-world environment.
3 . The method of claim 1 , wherein: the first device is a wearable device worn by the first user; the wearable device comprises a transparent display; and causing the first device to present the one or more media objects overlaid on real-world environment comprises causing the transparent display to present the one or more media objects.
4 . The method of claim 1 , further comprising: causing display, by the first device, of a session configuration interface, the session configuration interface comprising one or more interface elements to specify the one or more configuration parameters.
5 . The method of claim 1 , further comprising receiving, from the first device of a first user, the session configuration data for the virtual interaction session with the first user, the session configuration data including the one or more configuration parameters of the virtual interaction session specified by the first user, the one or more configuration parameters indicating that the second user is permitted by the first user to join the virtual interaction session with the first user.
6 . The method of claim 1 , wherein: the one or more configuration parameters include a micro-chat duration, the method further comprise detecting expiration of the micro-chat duration; and the terminating of the first real-time communication link is based on detecting expiration of the micro-chat duration.
7 . The method of claim 1 , further comprising: receiving, from the second device, a request to join the virtual interaction session; wherein establishing the first real-time communication link between the first user and the second user is in response to receiving the request and based on the session configuration data.
8 . The method of claim 1 , further comprising: based on the display of the live camera feed generated by the first device being displayed by the second device of the second user, causing display, at the first device, of an indicator that the second user is viewing the live camera feed generated by the camera of the first device.
9 . The method of claim 1 , further comprising: enabling an audio communication link between the first and second user while the live camera feed is displayed by the second device.
10 . The method of claim 1 , further comprising: causing display, at the second device, of a set of selectable virtual content items to apply to the real-world environment that is visible to the first user.
11 . A system comprising: a memory that stores instructions; and one or more processors configured by the instructions to perform operations comprising: initiating a virtual interaction session with a first device of a first user based on session configuration data received from the first device, the session configuration data including one or more configuration parameters of the virtual interaction session specified by the first user; establishing, prior to expiration of a session duration for the virtual interaction session, a first real-time communication link between the first user and a second user by causing display, at a second device of the second user, of a live camera feed generated at the first device, the live camera feed comprising images of a real-world environment that is visible to the first user via the first device; receiving, from the second device, input data indicative of a selection by the second user of a virtual content item to apply to the real-world environment that is visible to the first user, the virtual content item comprising one or more media objects; based on the second input data, causing display, by the first device and second device, of the one or more media objects overlaid on the real-world environment; terminating the first real-time communication link between the first user and the second user; upon termination of the first real-time communication link between the first user and the second user, providing an interactive element to the second device to trigger transmission of a request for a second real-time communication link between the first user and the second user; and establishing the second real-time communication link between the first user and the second user in response to receiving the request prior to expiration of the session duration.
12 . The system of claim 11 , wherein causing display, by the second device, of the one or more media objects comprises causing display of the live camera feed augmented to include the one or more media objects overlaid on the real-world environment.
13 . The system of claim 11 , wherein: the first device is a wearable device worn by the first user; the wearable device comprises a transparent display; and causing the first device to present the one or more media objects overlaid on real-world environment comprises causing the transparent display to present the one or more media objects.
14 . The system of claim 11 , wherein the operations further comprise: causing display, by the first device, of a session configuration interface, the session configuration interface comprising one or more interface elements to specify the one or more configuration parameters.
15 . The system of claim 11 , wherein the operations further comprise receiving, from the first device of a first user, the session configuration data for the virtual interaction session with the first user, the session configuration data including the one or more configuration parameters of the virtual interaction session specified by the first user, the one or more configuration parameters indicating that the second user is permitted by the first user to join the virtual interaction session with the first user.
16 . The system of claim 11 , wherein: the one or more configuration parameters include a micro-chat duration, the operations further comprise detecting expiration of the micro-chat duration; and the terminating of the first real-time communication link is based on detecting expiration of the micro-chat duration.
17 . The system of claim 11 , wherein the operations further comprise: receiving, from the second device, a request to join the virtual interaction session; wherein establishing the first real-time communication link between the first user and the second user is in response to receiving the request and based on the session configuration data.
18 . The system of claim 11 , wherein the operations further comprise: based on the display of the live camera feed generated by the first device being displayed by the second device of the second user, causing display, at the first device, of an indicator that the second user is viewing the live camera feed generated by the camera of the first device.
19 . The system of claim 11 , wherein the operations further comprise: enabling an audio communication link between the first and second user while the live camera feed is displayed by the second device.
20 . A machine-readable medium storing instructions that, when executed by a computer system, cause the computer system to perform operations comprising: initiating a virtual interaction session with a first device of a first user based on session configuration data received from the first device, the session configuration data including one or more configuration parameters of the virtual interaction session specified by the first user; establishing, prior to expiration of a session duration for the virtual interaction session, a first real-time communication link between the first user and a second user by causing display, at a second device of the second user, of a live camera feed generated at the first device, the live camera feed comprising images of a real-world environment that is visible to the first user via the first device; receiving, from the second device, input data indicative of a selection by the second user of a virtual content item to apply to the real-world environment that is visible to the first user, the virtual content item comprising one or more media objects; based on the second input data, causing display, by the first device and second device, of the one or more media objects overlaid on the real-world environment; terminating the first real-time communication link between the first user and the second user; upon termination of the first real-time communication link between the first user and the second user, providing an interactive element to the second device to trigger transmission of a request for a second real-time communication link between the first user and the second user; and establishing the second real-time communication link between the first user and the second user in response to receiving the request prior to expiration of the session duration.

Description

CLAIM OF PRIORITY This application is a continuation of U.S. patent application Ser. No. 17/207,080, filed Mar. 19, 2021, which claims the benefit of priority to U.S. Provisional Application Ser. No. 62/994,641, filed on Mar. 25, 2020, each of which are incorporated herein by reference in their entireties. TECHNICAL FIELD The present disclosure generally relates to mobile and wearable computing technology. In particular, example embodiments of the present disclosure address systems, methods, and user interfaces to facilitate augmented reality based communication between multiple users over a network. BACKGROUND Many wearable and mobile devices such as “smart” glasses include an embedded camera. Virtual rendering systems implemented using these types of devices can be used to create engaging and entertaining augmented reality experiences, in which three-dimensional (3D) graphic content appears to be present in the real world. BRIEF DESCRIPTION OF THE DRAWINGS To easily identify the discussion of any particular element or act, the most significant digit or digits in a reference number refer to the figure number in which that element or act is first introduced. FIG. 1 is a diagrammatic representation of a networked environment in which the present disclosure may be deployed, in accordance with some example embodiments. FIG. 2 is a diagrammatic representation of a communication system, in accordance with some example embodiments. FIG. 3 is a diagram illustrating a wearable device for use in a virtual interaction session, according to some example embodiments. FIG. 4 is a block diagram illustrating aspects of the wearable device, according to some example embodiments. FIGS. 5A-5H are conceptual diagrams illustrating a flow of an example virtual interaction session between a first and second user, according to some example embodiments. FIGS. 6A and 6B are interaction diagrams illustrating example interactions between components of the communication system in performing a method for facilitating a virtual interaction session with a first user, according to example embodiments. FIGS. 7-9 are flowcharts illustrating operations of the communication system in performing a method for facilitating a virtual interaction session with a first user, according to example embodiments. FIG. 10 is a block diagram illustrating a representative software architecture, which may be used in conjunction with various hardware architectures herein described, according to example embodiments. FIG. 11 is a block diagram illustrating components of a machine able to read instructions from a machine-readable medium (e.g., a machine-readable storage medium) and perform any one or more of the methodologies discussed herein, according to example embodiments. DETAILED DESCRIPTION The description that follows includes systems, methods, techniques, instruction sequences, and computing machine program products that embody illustrative embodiments of the disclosure. In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide an understanding of various embodiments of the inventive subject matter. It will be evident, however, to those skilled in the art, that embodiments of the inventive subject matter may be practiced without these specific details. In general, well-known instruction instances, protocols, structures, and techniques are not necessarily shown in detail. Aspects of the present disclosure include systems, methods, techniques, instruction sequences, and computing machine program products for facilitating virtual interaction sessions between users. In the context of a virtual interaction, one or more users may establish a real-time communication link with a host user. When a real-time communication link is established between a first user and a second user, a live camera feed that depicts a real-world environment at the location of the first user is presented to the second user. As an example, the first user may be wearing a wearable device that includes a camera and optical elements that include a display from which the real-world environment is visible to the first user. During an active virtual interaction session with the first user, a real-time communication link between the first and second user can be established and as part of the real-time communication link the second user can view the real-world environment of the first user via the live camera feed generated by the camera of the first user's wearable device and displayed by a device of the second user. A real-time communication link can be initiated by the first user (the host user) by inviting the second user to join the virtual interaction session and the real-time communication link is established based on the second user accepting the invitation. While real-time communication link is active, the first and second user can communicate verbally using an audio communication link between devices and also using augm