EP-3959585-B1 - DEVICES, METHODS, AND GRAPHICAL USER INTERFACES FOR DISPLAYING OBJECTS IN 3D CONTEXTS

EP3959585B1EP 3959585 B1EP3959585 B1EP 3959585B1EP-3959585-B1

Inventors

SCAPEL, Nicolas, V.
YERKES, GIANCARLO
PAUL, Grant, R.
DELLINGER, RICHARD, R.
LUI, DAVID
YU, Xiao, Jin
FISHER, Kyle, E.
MASON, Mathew, A.
CHEN, HON-MING

Dates

Publication Date: 20260506
Application Date: 20200401

Claims (14)

A method, comprising: at an electronic device including a display generation component, one or more input devices, and one or more cameras: displaying, by the display generation component, a plurality of virtual objects; while displaying the plurality of virtual objects, detecting a first manipulation input directed to a first virtual object of the plurality of virtual objects, wherein the plurality of virtual objects additionally includes one or more virtual objects other than the first virtual object; and in response to detecting the first manipulation input directed to the first virtual object of the plurality of virtual objects: in accordance with a determination that the plurality of virtual objects are displayed in a user interface region that includes at least a portion of a field of view of the one or more cameras, changing a display property of the first virtual object based on the first manipulation input without changing a display property of the one or more virtual objects other than the first virtual object based on the first manipulation input; and in accordance with a determination that the plurality of virtual objects are displayed in a user interface region that does not include the field of view of the one or more cameras: changing the display property of the plurality of virtual objects including the first virtual object based on the first manipulation input.
The method of claim 1, wherein: while the plurality of virtual objects are displayed in the user interface region that does not include the field of view of the one or more cameras, the plurality of virtual objects are displayed with a first set of display characteristics.
The method of claim 2, wherein: the first set of display characteristics includes a height ordering characteristic; and displaying the plurality of virtual objects in accordance with the height ordering characteristic includes ordering the plurality of virtual objects in accordance with respective heights of the plurality of virtual objects.
The method of any of claims 2-3, wherein: the first set of display characteristics includes a spacing characteristic; and displaying the plurality of virtual objects in accordance with the spacing characteristic includes positioning the plurality of virtual objects with a uniform distance between respective boundaries of the plurality of virtual objects.
The method of any of claims 2-4, including: while displaying the plurality of virtual objects in the user interface region that includes at least a portion of the field of view of the one or more cameras, after changing the display property of the first virtual object based on the first manipulation input, detecting a request to display the plurality of virtual objects in the user interface region that does not include the field of view of the one or more cameras; and in response to detecting the request to display the plurality of virtual objects in the user interface region that does not include the field of view of the one or more cameras, displaying the user interface region that does not include the field of view of the one or more cameras, including displaying the plurality of virtual objects with the first set of display characteristics.
The method of claim 5, including: after changing the display property of the plurality of virtual objects based on the first manipulation input, while displaying the plurality of virtual objects in the user interface region that does not include the field of view of the one or more cameras, detecting a request to re-display the plurality of virtual objects in the user interface region that includes at least a portion of the field of view of the one or more cameras; and in response to detecting the request to re-display the plurality of virtual objects in the user interface region that includes at least a portion of the field of view of the one or more cameras, re-displaying the plurality of virtual objects in the user interface region that includes at least a portion of the field of view of the one or more cameras, including re-displaying the plurality of virtual objects with the display property of the plurality of virtual objects as changed based on the first manipulation input.
The method of any of claims 1-6, wherein: the first manipulation input is a rotation input.
The method of any of claims 1-7, including: prior to displaying the plurality of virtual objects, displaying a selection user interface; while the selection user interface is displayed, detecting selection of the plurality of virtual objects; while the plurality of virtual objects is selected, receiving a request to display the plurality of virtual objects; and in response to receiving the request to display the plurality of virtual objects, displaying the plurality of virtual objects in the user interface region that does not include the field of view of the one or more cameras.
The method of claim 1, wherein: the first virtual object of the plurality of virtual objects has a first defined plane type; a second virtual object of the plurality of virtual objects has a second defined plane type that is distinct from the first defined plane type; and while displaying the first virtual object and the second virtual object in the user interface region that includes at least a portion of the field of view of the one or more cameras: the first virtual object of the plurality of virtual objects is displayed at a fixed position relative to a first plane that corresponds to the first defined plane type; and the second virtual object of the plurality of virtual objects is displayed at a fixed position relative to a second plane that corresponds to the second defined plane type that is distinct from the first defined plane type.
The method of claim 9, including: while displaying the first virtual object and the second virtual object in the user interface region that includes at least a portion of the field of view of the one or more cameras, displaying the first virtual object at the fixed position relative to the first plane, and displaying the second virtual object at the fixed position relative to the second plane, detecting a rotation input; and in response to detecting the rotation input: in accordance with a determination that the rotation input is directed to the first virtual object, rotating the first virtual object about a first axis; and in accordance with a determination that the rotation input is directed to the second virtual object, rotating the second virtual object about a second axis that is distinct from the first axis.
A computer system, comprising: a display generation component; one or more input devices; one or more cameras; one or more processors; and memory storing one or more programs, wherein the one or more programs are configured to be executed by the one or more processors, the one or more programs including instructions for: displaying, by the display generation component, a plurality of virtual objects; while displaying the plurality of virtual objects, detecting a first manipulation input directed to a first virtual object of the plurality of virtual objects, wherein the plurality of virtual objects additionally includes one or more virtual objects other than the first virtual object; and in response to detecting the first manipulation input directed to the first virtual object of the plurality of virtual objects: in accordance with a determination that the plurality of virtual objects are displayed in a user interface region that includes at least a portion of a field of view of the one or more cameras, changing a display property of the first virtual object based on the first manipulation input without changing a display property of the one or more virtual objects other than the first virtual object based on the first manipulation input; and in accordance with a determination that the plurality of virtual objects are displayed in a user interface region that does not include the field of view of the one or more cameras: changing the display property of the plurality of virtual objects including the first virtual object based on the first manipulation input.
The computer system of claim 11, wherein the one or more programs include instructions for performing the method of any of claims 2-10.
A computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which, when executed by a computer system with a display generation component, one or more input devices, and one or more cameras, cause the computer system to: display, by the display generation component, a plurality of virtual objects; while displaying the plurality of virtual objects, detect a first manipulation input directed to a first virtual object of the plurality of virtual objects, wherein the plurality of virtual objects additionally includes one or more virtual objects other than the first virtual object; and in response to detecting the first manipulation input directed to the first virtual object of the plurality of virtual objects: in accordance with a determination that the plurality of virtual objects are displayed in a user interface region that includes at least a portion of a field of view of the one or more cameras, change a display property of the first virtual object based on the first manipulation input without changing a display property of the one or more virtual objects other than the first virtual object based on the first manipulation input; and in accordance with a determination that the plurality of virtual objects are displayed in a user interface region that does not include the field of view of the one or more cameras: change the display property of the plurality of virtual objects including the first virtual object based on the first manipulation input.
The computer readable storage medium of claim 13, wherein the one or more programs include instructions that, when executed by the computer system, cause the computer system to perform the method of any of claims 2-10.

Description

TECHNICAL FIELD This relates generally to electronic devices that display virtual objects, including but not limited to electronic devices that display virtual objects in a variety of contexts. BACKGROUND The development of computer systems for augmented reality has increased significantly in recent years. Example augmented reality environments include at least some virtual elements that replace or augment the physical world. Input devices, such as touch-sensitive surfaces, for computer systems and other electronic computing devices are used to interact with virtual/augmented reality environments. Example touch-sensitive surfaces include touchpads, touch-sensitive remote controls, and touch-screen displays. Such surfaces are used to manipulate user interfaces and objects therein on a display. Example user interface objects include digital images, video, text, icons, and control elements such as buttons and other graphics. But methods and interfaces for interacting with environments that include at least some virtual elements (e.g., applications, augmented reality environments, mixed reality environments, and virtual reality environments) are cumbersome, inefficient, and limited. For example, systems that provide insufficient feedback for performing actions associated with virtual objects, systems that require a series of inputs to generate virtual objects suitable for display in an augmented reality environment, and systems that require lack handling for manipulating sets of virtual objects are tedious, create a significant cognitive burden on a user, and detract from the experience with the virtual/augmented reality environment. In addition, these methods take longer than necessary, thereby wasting energy. This latter consideration is particularly important in battery-operated devices. Patent documents AU 2019 100 079 A4 and US 9 754 419 B2 describe examples of the related art. SUMMARY Accordingly, there is a need for computer systems with improved methods and interfaces for interacting with virtual objects. Such methods and interfaces optionally complement or replace conventional methods for interacting with virtual objects. Such methods and interfaces reduce the number, extent, and/or nature of the inputs from a user and produce a more efficient human-machine interface. For battery-operated devices, such methods and interfaces conserve power and increase the time between battery charges. The above deficiencies and other problems associated with interfaces for interacting with virtual objects (e.g., user interfaces for augmented reality (AR) and related non-AR interfaces) are reduced or eliminated by the disclosed computer systems. In some embodiments, the computer system includes a desktop computer. In some embodiments, the computer system is portable (e.g., a notebook computer, tablet computer, or handheld device). In some embodiments, the computer system includes a personal electronic device (e.g., a wearable electronic device, such as a watch). In some embodiments, the computer system has (and/or is in communication with) a touchpad. In some embodiments, the computer system has (and/or is in communication with) a touch-sensitive display (also known as a "touch screen" or "touch-screen display"). In some embodiments, the computer system has a graphical user interface (GUI), one or more processors, memory and one or more modules, programs or sets of instructions stored in the memory for performing multiple functions. In some embodiments, the user interacts with the GUI in part through stylus and/or finger contacts and gestures on the touch-sensitive surface. In some embodiments, the functions optionally include game playing, image editing, drawing, presenting, word processing, spreadsheet making, telephoning, video conferencing, e-mailing, instant messaging, workout support, digital photographing, digital videoing, web browsing, digital music playing, note taking, and/or digital video playing. Executable instructions for performing these functions are, optionally, included in a non-transitory computer readable storage medium or other computer program product configured for execution by one or more processors. The invention is defined by the independent claims. Further aspects of the invention are outlined in the dependent claims. When the term embodiment is used for describing unclaimed combinations of features, the term has to be understood as referring to examples useful for understanding the present invention. In accordance with claim 1, a method is performed at an electronic device including a display generation component, one or more input devices, and one or more cameras. The method includes displaying, by the display generation component, a plurality of virtual objects. The method further includes, while displaying the plurality of virtual objects, detecting a first manipulation input directed to a first virtual object of the plurality of virtual objects, wherein the plurality of virtual objects