US-12620155-B2 - Representations of participants in real-time communication sessions

US12620155B2US 12620155 B2US12620155 B2US 12620155B2US-12620155-B2

Abstract

A computer system optionally modifies a representation of a participant based on activity associated with the participant. A computer system optionally displays a self-view representation of an avatar of a user of the computer system. A computer system optionally updates a view of an avatar in a real-time communication session. A computer system optionally displays a representation of a participant in a real-time communication session.

Inventors

Jesse Chand
Shih-Sang Chiu

Assignees

APPLE INC.

Dates

Publication Date: 20260505
Application Date: 20230905

Claims (20)

1 . A computer system configured to communicate with one or more display generation components, the computer system comprising: one or more processors; and memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for: displaying, via the one or more display generation components, representations of a plurality of participants in a real-time communication session in a spatial arrangement of tiles in a three-dimensional space, including concurrently displaying: a representation of a first participant of the plurality of participants in a first tile in the spatial arrangement of tiles; and a representation of a second participant of the plurality of participants in a second tile in the spatial arrangement of tiles, wherein the second participant is different from the first participant and the second tile is different from the first tile; while displaying the representations of the plurality of participants in the real-time communication session in the three-dimensional space, detecting activity associated with the first participant of the plurality of participants; in response to detecting the activity associated with the first participant of the plurality of participants, modifying the representation of the first participant based on the activity, including moving the first tile relative to the second tile in the spatial arrangement of tiles; while displaying the representations of the plurality of participants in the real-time communication session in the three-dimensional space, detecting movement, relative to the first participant, of a device being used by the first participant to participate in the real-time communication session; and in response to detecting the movement, relative to the first participant, of the device being used by the first participant to participate in the real-time communication session, moving the first tile forwards or backwards relative to the second tile in the spatial arrangement of tiles.
2 . The computer system of claim 1 , wherein modifying the representation of the first participant based on the activity includes moving the representation of the first participant in the first tile forward or backward relative to the representation of the second participant in the second tile.
3 . The computer system of claim 1 , wherein modifying the representation of the first participant based on the activity includes changing a size of the representation of the first participant relative to a size of the representation of the second participant.
4 . The computer system of claim 1 , wherein modifying the representation of the first participant based on the activity includes changing an angle of the representation of the first participant relative to an angle of the representation of the second participant.
5 . The computer system of claim 1 , wherein detecting the activity associated with the first participant includes detecting sounds from the first participant and/or movement of the representation of the first participant.
6 . The computer system of claim 1 , wherein the activity associated with the first participant is based on movement of the first participant relative to a point of reference corresponding to the real-time communication session.
7 . The computer system of claim 6 , wherein the movement of the first participant relative to the point of reference corresponding to the real-time communication session is a change in orientation of a face of the first participant relative to the point of reference corresponding to the real-time communication session and modifying the representation of the first participant includes tilting, based on the change in orientation, the representation of the first participant.
8 . The computer system of claim 1 , wherein the activity associated with the first participant is based on movement of a point of reference corresponding to the real-time communication session relative to the first participant.
9 . The computer system of claim 8 , wherein modifying the representation of the first participant based on the activity includes: in accordance with the activity associated with the first participant including the point of reference corresponding to the real-time communication session moving closer to the first participant, moving the representation of the first participant forward relative to the representation of the second participant; and in accordance with the activity associated with the first participant including the point of reference corresponding to the real-time communication session moving farther from the first participant, moving the representation of the first participant backward relative to the representation of the second participant.
10 . The computer system of claim 1 , wherein displaying the representations of the plurality of participants in the real-time communication session in the spatial arrangement of tiles in the three-dimensional space includes displaying representations of a first subset of the plurality of participants in a first region and displaying representations of a second subset of the plurality of participants, different from the first subset of the plurality of participants, in a second region that is different from the first region, and wherein the one or more programs further include instructions for: detecting activity associated with a respective participant of the plurality of participants; and in response to detecting the activity associated with the respective participant of the plurality of participants: in accordance with a determination that a representation of the respective participant is of the first subset of the plurality of participants that is displayed in the first region, modifying the representation of the respective participant based on the activity associated with the respective participant; and in accordance with a determination that the representation of the respective participant is of the second subset of the plurality of participants that is displayed in the second region, forgoing modifying the representation of the respective participant based on the activity associated with the respective participant.
11 . A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system that is in communication with one or more display generation components, the one or more programs including instructions for: displaying, via the one or more display generation components, representations of a plurality of participants in a real-time communication session in a spatial arrangement of tiles in a three-dimensional space, including concurrently displaying: a representation of a first participant of the plurality of participants in a first tile in the spatial arrangement of tiles; and a representation of a second participant of the plurality of participants in a second tile in the spatial arrangement of tiles, wherein the second participant is different from the first participant and the second tile is different from the first tile; while displaying the representations of the plurality of participants in the real-time communication session in the three-dimensional space, detecting activity associated with the first participant of the plurality of participants; in response to detecting the activity associated with the first participant of the plurality of participants, modifying the representation of the first participant based on the activity, including moving the first tile relative to the second tile in the spatial arrangement of tiles; while displaying the representations of the plurality of participants in the real-time communication session in the three-dimensional space, detecting movement, relative to the first participant, of a device being used by the first participant to participate in the real-time communication session; and in response to detecting the movement, relative to the first participant, of the device being used by the first participant to participate in the real-time communication session, moving the first tile forwards or backwards relative to the second tile in the spatial arrangement of tiles.
12 . The non-transitory computer-readable storage medium of claim 11 , wherein modifying the representation of the first participant based on the activity includes moving the representation of the first participant in the first tile forward or backward relative to the representation of the second participant in the second tile.
13 . The non-transitory computer-readable storage medium of claim 11 , wherein modifying the representation of the first participant based on the activity includes changing a size of the representation of the first participant relative to a size of the representation of the second participant.
14 . The non-transitory computer-readable storage medium of claim 11 , wherein modifying the representation of the first participant based on the activity includes changing an angle of the representation of the first participant relative to an angle of the representation of the second participant.
15 . The non-transitory computer-readable storage medium of claim 11 , wherein detecting the activity associated with the first participant includes detecting sounds from the first participant and/or movement of the representation of the first participant.
16 . The non-transitory computer-readable storage medium of claim 11 , wherein the activity associated with the first participant is based on movement of the first participant relative to a point of reference corresponding to the real-time communication session.
17 . The non-transitory computer-readable storage medium of claim 16 , wherein the movement of the first participant relative to the point of reference corresponding to the real-time communication session is a change in orientation of a face of the first participant relative to the point of reference corresponding to the real-time communication session and modifying the representation of the first participant includes tilting, based on the change in orientation, the representation of the first participant.
18 . The non-transitory computer-readable storage medium of claim 11 , wherein the activity associated with the first participant is based on movement of a point of reference corresponding to the real-time communication session relative to the first participant.
19 . The non-transitory computer-readable storage medium of claim 18 , wherein modifying the representation of the first participant based on the activity includes: in accordance with the activity associated with the first participant including the point of reference corresponding to the real-time communication session moving closer to the first participant, moving the representation of the first participant forward relative to the representation of the second participant; and in accordance with the activity associated with the first participant including the point of reference corresponding to the real-time communication session moving farther from the first participant, moving the representation of the first participant backward relative to the representation of the second participant.
20 . The non-transitory computer-readable storage medium of claim 11 , wherein displaying the representations of the plurality of participants in the real-time communication session in the spatial arrangement of tiles in the three-dimensional space includes displaying representations of a first subset of the plurality of participants in a first region and displaying representations of a second subset of the plurality of participants, different from the first subset of the plurality of participants, in a second region that is different from the first region, and wherein the one or more programs further include instructions for: detecting activity associated with a respective participant of the plurality of participants; and in response to detecting the activity associated with the respective participant of the plurality of participants: in accordance with a determination that a representation of the respective participant is of the first subset of the plurality of participants that is displayed in the first region, modifying the representation of the respective participant based on the activity associated with the respective participant; and in accordance with a determination that the representation of the respective participant is of the second subset of the plurality of participants that is displayed in the second region, forgoing modifying the representation of the respective participant based on the activity associated with the respective participant.

Description

CROSS REFERENCE TO RELATED APPLICATIONS This application claims priority to the following applications: U.S. Application No. 63/470,884, entitled “REPRESENTATIONS OF PARTICIPANTS IN REAL-TIME COMMUNICATION SESSIONS,” filed on Jun. 3, 2023, U.S. Application No. 63/439,473, entitled “REPRESENTATIONS OF PARTICIPANTS IN REAL-TIME COMMUNICATION SESSIONS,” filed on Jan. 17, 2023, and U.S. Application No. 63/409,526, entitled “REPRESENTATIONS OF PARTICIPANTS IN REAL-TIME COMMUNICATION SESSIONS,” filed on Sep. 23, 2022. The entire contents of each of these applications are hereby incorporated by reference in their entireties. TECHNICAL FIELD The present disclosure relates generally to computer systems that are in communication with one or more display generation components and, optionally, one or more input devices that provide computer-generated experiences, including, but not limited to, electronic devices that provide virtual reality and mixed reality experiences via a display. BACKGROUND The development of computer systems for extended reality has increased significantly in recent years. Extended reality environment refers to a wholly or partially simulated environment that people sense and/or interact with via a computer system. Input devices, such as cameras, controllers, joysticks, touch-sensitive surfaces, and touch-screen displays for computer systems and other electronic computing devices are used to interact with extended reality environments that include virtual elements. Example virtual elements include virtual objects, such as digital images, video, text, icons, and control elements such as buttons and other graphics. SUMMARY Some methods and interfaces for managing representations of participants in real-time communication sessions are cumbersome, inefficient, and limited. For example, systems that provide insufficient feedback for performing actions associated with representations of participants of real-time communication sessions, systems that require a series of inputs to achieve a desired outcome in the real-time communication session, and systems in which manipulation of representations of participants of real-time communication sessions are complex, tedious, and error-prone, create a significant cognitive burden on a user, and detract from the experience with the virtual/augmented reality environment. In addition, these methods take longer than necessary, thereby wasting energy of the computer system. This latter consideration is particularly important in battery-operated devices. Accordingly, there is a need for computer systems with improved methods and interfaces for managing representations of participants of real-time communication sessions to make interactions with the computer systems more efficient and intuitive for a user. Such methods and interfaces optionally complement or replace conventional methods for managing representations of participants in real-time communication sessions. Such methods and interfaces reduce the number, extent, and/or nature of the inputs from a user by helping the user to understand the connection between provided inputs and device responses to the inputs, thereby creating a more efficient human-machine interface. The above deficiencies and other problems associated with user interfaces for computer systems are reduced or eliminated by the disclosed systems. In some embodiments, the computer system is a desktop computer with an associated display. In some embodiments, the computer system is portable device (e.g., a notebook computer, tablet computer, or handheld device). In some embodiments, the computer system is a personal electronic device (e.g., a wearable electronic device, such as a watch, or a head-mounted device). In some embodiments, the computer system has a touchpad. In some embodiments, the computer system has one or more cameras. In some embodiments, the computer system has a touch-sensitive display (also known as a “touch screen” or “touch-screen display”). In some embodiments, the computer system has one or more eye-tracking components. In some embodiments, the computer system has one or more hand-tracking components. In some embodiments, the computer system has one or more output devices in addition to the display generation component, the output devices including one or more tactile output generators and/or one or more audio output devices. In some embodiments, the computer system has a graphical user interface (GUI), one or more processors, memory and one or more modules, programs or sets of instructions stored in the memory for performing multiple functions. In some embodiments, the user interacts with the GUI through a stylus and/or finger contacts and gestures on the touch-sensitive surface, movement of the user's eyes and hand in space relative to the GUI (and/or computer system) or the user's body as captured by cameras and other movement sensors, and/or voice inputs as captured by one or more audio input devices. In some embodiments,