CN-121979393-A - Apparatus, method and graphical user interface for displaying virtual keyboard

Abstract

The present disclosure relates to devices, methods, and graphical user interfaces for displaying virtual keyboards. The present disclosure relates generally to methods and user interfaces for locating virtual keyboards in a three-dimensional environment, displaying various types of virtual keyboards, switching between virtual keyboards, and/or displaying virtual keyboards based on a user's location.

Inventors

• E. Kryvoluchko
• HU CHUNJIA
• S. H. Cali
• W. Abdallah
• Request for anonymity
• L. C. Madeia
• T. Baker
• YU SHUXIN
• E. K. Van Halen
• WU DINGYUAN
• E. Kasucci
• H. Tejaden
• S.O. Lemme
• Z. C. Taylor

Assignees

• 苹果公司

Dates

Publication Date

20260505

Application Date

20240530

Priority Date

20240529

Claims (20)

1. 1. A method, the method comprising: at a computer system in communication with a display generation component and one or more input devices: Displaying, via the display generating component, a user interface, the user interface comprising: representation of a portion of a three-dimensional environment, and A representation of the corresponding application; receiving a request via the one or more input devices for displaying a keyboard user interface in the representation of the portion of the three-dimensional environment, and In response to receiving the request to display the keyboard user interface, displaying the keyboard user interface via the display generating component at a respective keyboard location in the representation of the portion of the three-dimensional environment, wherein: In accordance with a determination that a first set of keyboard placement criteria is satisfied, wherein the first set of keyboard placement criteria includes a first application criteria that is satisfied when the representation of the respective application has a first pose in the representation of the portion of the three-dimensional environment, the respective keyboard location being a first keyboard location in the representation of the portion of the three-dimensional environment, and In accordance with a determination that a second set of keyboard placement criteria is met, wherein the second set of keyboard placement criteria includes a second application criteria that is met when the representation of the respective application has a second pose in the representation of the portion of the three-dimensional environment, wherein the second pose is different from the first pose, the respective keyboard location being a second keyboard location in the representation of the portion of the three-dimensional environment that is different from the first keyboard location in the representation of the portion of the three-dimensional environment.
2. 2. The method according to claim 1, wherein: The first set of keyboard placement criteria is satisfied when the representation of the respective application having the first pose is at a first angle relative to a first reference line that is greater than a first angle threshold, and The first keyboard location is based on a location of a line intersecting a body part of a user of the computer system and the representation of the respective application.
3. 3. The method according to claim 2, wherein: Meeting the second set of keyboard placement criteria when the representation of the respective application having the second pose is at a second angle relative to the first reference line that is less than the first angle threshold, and The second keyboard location is based on a horizon of the three-dimensional environment.
4. 4. The method of any of claims 1-2, wherein the first keyboard positioning is based on positioning of a respective body part of a user of the computer system within the three-dimensional environment, comprising: in accordance with a determination that the respective body part of the user of the computer system has a first location of the respective body part within the three-dimensional environment, the first keyboard location is a third keyboard location, and In accordance with a determination that the respective body part of the user of the computer system has a second location of the respective body part within the three-dimensional environment that is different from the first location of the respective body part, the first keyboard location is a fourth keyboard location that is different from the third keyboard location.
5. 5. The method of claim 4, wherein the positioning of the respective body part of the user of the computer system within the three-dimensional environment is positioning relative to a floor of the three-dimensional environment, and wherein the positioning of the floor is determined by the computer system.
6. 6. The method according to claim 4, wherein: the third keyboard positioning based on a first angle relative to the respective line, and The fourth keyboard positioning is based on a second angle different from the first angle relative to the respective line.
7. 7. The method according to claim 6, wherein: the first location of the corresponding body part of the user of the computer system is within a first location range within the three-dimensional environment, and The second location of the respective body part of the user of the computer system is a second location range within the three-dimensional environment that is different from the first location range.
8. 8. The method of claim 6, wherein the first keyboard positioning is based on the positioning of the respective body part of the user of the computer system within the three-dimensional environment, comprising: in accordance with a determination that the respective body part of the user of the computer system has a third location of the respective body part that is different from the first location of the respective body part, the second location of the respective body part, within a third location range within the three-dimensional environment, the first keyboard location is a fifth keyboard location that is different from the third keyboard location and the fourth keyboard location.
9. 9. The method of any of claims 1-2, wherein the first keyboard location is based on a respective device location of the computer system within the three-dimensional environment, comprising: in accordance with a determination that the computer system has a first location within the three-dimensional environment, the first keyboard location is a sixth keyboard location, and In accordance with a determination that the computer system has a second location within the three-dimensional environment, the first keyboard location is a seventh keyboard location that is different from the sixth keyboard location.
10. 10. The method according to claim 9, wherein: the first location of the computer system within the three-dimensional environment is a location relative to a first body part of a user of the computer system, and The second location of the computer system within the three-dimensional environment is a location of the first body part relative to the user of the computer system.
11. 11. The method according to claim 9, wherein: The first positioning of the computer system within the three-dimensional environment is a positioning relative to a physical object, and The second location of the computer system within the three-dimensional environment is a location relative to a physical object.
12. 12. The method of any of claims 1-2, wherein displaying the keyboard user interface at the first keyboard location is in accordance with a determination that a set of intersection criteria is met, wherein the set of intersection criteria includes a criterion that is met when the first keyboard location does not cause the keyboard user interface to intersect at least a portion of a physical object in the representation of the portion of the three-dimensional environment; Wherein displaying the keyboard user interface at the respective keyboard location in the representation of the portion of the three-dimensional environment further comprises: In accordance with a determination that the first set of keyboard placement criteria is met and in accordance with a determination that the set of intersection criteria is not met, the respective keyboard location is in an eighth keyboard location in the representation of the portion of the three-dimensional environment that is different from the first keyboard location.
13. 13. The method of any of claims 1-2, wherein the keyboard user interface comprises: a first character key which when selected causes a first alphanumeric character to be entered, and A second character key that when selected causes a second alphanumeric character to be entered that is different from the first alphanumeric character.
14. 14. The method of any of claims 1-2, wherein the keyboard user interface includes a respective interface object for suggested content, the method further comprising: detecting input for adding respective content via a hardware keyboard while the keyboard user interface is displayed at the respective keyboard location, and In response to detecting the input for adding the respective content: in accordance with a determination that the respective content is a first content, the respective interface object for the first content includes a first suggested content, and In accordance with a determination that the respective content is a second content that is different from the first content, the respective interface object for suggested content includes a second suggested content that is different from the first suggested content.
15. 15. The method of any one of claims 1 to 2, the method further comprising: detecting a key selection input via the one or more input devices, wherein the key selection input is an input detected on the keyboard user interface or an input detected via a hardware keyboard, and In response to detecting the key selection input, content corresponding to the key selection input is displayed in a text entry field via the display generating component.
16. 16. The method of any of claims 1-2, wherein displaying the keyboard user interface at the first keyboard location comprises displaying the keyboard user interface with a first set of one or more location parameters, the method further comprising: Detecting, via the one or more input devices, a request to modify the first set of one or more positioning parameters of the keyboard user interface while the keyboard user interface is displayed at the first keyboard location; in response to detecting the request to modify the first set of one or more positioning parameters of the keyboard user interface, displaying the keyboard user interface with a second set of one or more positioning parameters via the display generating component; Detecting a set of one or more inputs via the one or more input devices after displaying the keyboard user interface with the second set of one or more positioning parameters; In response to detecting the set of one or more inputs, ceasing display of the keyboard user interface; Receiving, via the one or more input devices, a second request to display the keyboard user interface in the representation of the portion of the three-dimensional environment when the keyboard user interface is not displayed, and In response to receiving the second request to display the keyboard user interface, displaying the keyboard user interface via the display generating component with a third set of one or more positioning parameters, wherein the third set of one or more positioning parameters includes at least one positioning parameter included in the second set of one or more positioning parameters.
17. 17. The method according to claim 16, wherein: displaying the keyboard user interface using the second set of one or more positioning parameters includes displaying the keyboard user interface at a respective position relative to the computer system, and Displaying the keyboard user interface using the third set of one or more positioning parameters includes: In accordance with a determination that a set of one or more positioning criteria is met, displaying the keyboard user interface at the respective positioning relative to the computer system, wherein the set of one or more positioning criteria includes a criterion that is met when at least a portion of a body of a user of the computer system has moved less than a threshold amount after stopping the display of the keyboard user interface.
18. 18. The method of claim 17, wherein the request to modify the first set of one or more positioning parameters of the keyboard user interface is a request to modify positioning of the keyboard user interface of a first application, the method further comprising: receiving a third request via the one or more input devices to display the keyboard user interface in the representation of the portion of the three-dimensional environment when the keyboard user interface is not displayed, and In response to receiving the third request to display the keyboard user interface: Displaying the keyboard user interface via the display generating component with the third set of one or more positioning parameters in accordance with a determination that the third request for displaying the keyboard user interface corresponds to a request for displaying the keyboard user interface of the first application program, and In accordance with a determination that the third request for displaying the keyboard user interface corresponds to a request for displaying the keyboard user interface of a second application different from the first application, displaying the keyboard user interface via the display generating component with a fourth set of one or more positioning parameters different from the third set of one or more positioning parameters.
19. 19. The method according to claim 16, wherein: displaying the keyboard user interface using the third set of one or more positioning parameters includes: In accordance with a determination that a second body part has moved beyond a threshold amount after stopping display of the keyboard user interface, the keyboard user interface is displayed via the display generating component at a respective location offset from a default location by a first offset value, wherein the first offset value is based on the second set of one or more location parameters.
20. 20. A computer readable storage medium storing one or more programs configured for execution by one or more processors of a computer system in communication with a display generation component and one or more input devices, the one or more programs comprising instructions for performing the method of any of claims 1-19.

Description

Apparatus, method and graphical user interface for displaying virtual keyboard Statement of case division The application is a divisional application of China patent application with the application date 2024, the month 05 and 30, the name of which is 'equipment, method and graphic user interface for displaying a virtual keyboard', and the application number 202480050339.4. Cross Reference to Related Applications The present application claims priority from U.S. patent application Ser. No. 18/676,851, entitled "DEVICES, METHODS, AND GRAPHICAL USER INTERFACES FOR DISPLAYING A VIRTUAL KEYBOARD", filed on day 29 of 5 in 2024, U.S. provisional application Ser. No. 63/652,633, entitled "DEVICES, METHODS, AND GRAPHICAL USER INTERFACES FOR DISPLAYING A VIRTUAL KEYBOARD", filed on day 28 in 2024, U.S. provisional application Ser. No. 63/470,915, entitled "DEVICES, METHODS, AND GRAPHICAL USER INTERFACES FOR DISPLAYING A VIRTUAL KEYBOARD", filed on day 4 in 2023, and U.S. provisional application Ser. No. 63/470,094, entitled "DEVICES, METHODS, AND GRAPHICAL USER INTERFACES FOR DISPLAYING A VIRTUAL KEYBOARD", filed on day 31 in 2023. The contents of each of these patent applications are incorporated herein by reference in their entirety. Technical Field The present disclosure relates generally to computer systems providing a computer-generated experience in communication with a display generation component and optionally with one or more input devices, including, but not limited to, electronic devices providing virtual reality and mixed reality experiences via a display. Background In recent years, the development of computer systems for augmented reality has increased significantly. An example augmented reality environment includes at least some virtual elements that replace or augment the physical world. Input devices (such as cameras, controllers, joysticks, touch-sensitive surfaces, and touch screen displays) for computer systems and other electronic computing devices are used to interact with the virtual/augmented reality environment. Example virtual elements include virtual objects such as digital images, videos, text, icons, and control elements (such as buttons and other graphics). Disclosure of Invention Some methods and interfaces for providing a virtual keyboard that includes at least some virtual elements (e.g., applications, augmented reality environments, mixed reality environments, and virtual reality environments) are cumbersome, inefficient, and limited. For example, systems that do not provide proper placement of a keyboard, systems that provide insufficient feedback for performing actions associated with virtual objects, systems that require a series of inputs to achieve desired results in an augmented reality environment, and systems that are complex, cumbersome, and error-prone to virtual object manipulation can create a significant cognitive burden on the user and detract from the experience of the virtual/augmented reality environment. In addition, these methods take longer than necessary, wasting energy from the computer system. This latter consideration is particularly important in battery-powered devices. Accordingly, there is a need for a computer system with improved methods and interfaces to provide a virtual keyboard to a user that makes interactions with the computer system more efficient and intuitive for the user. Such methods and interfaces optionally complement or replace conventional methods for providing a virtual keyboard to a user. Such methods and interfaces reduce the number, extent, and/or nature of inputs from a user by helping the user understand the association between the inputs provided and the response of the device to those inputs, thereby forming a more efficient human-machine interface. The above-described drawbacks and other problems associated with user interfaces of computer systems are reduced or eliminated by the disclosed systems. In some embodiments, the computer system is a desktop computer with an associated display. In some embodiments, the computer system is a portable device (e.g., a notebook computer, tablet computer, or handheld device). In some embodiments, the computer system is a personal electronic device (e.g., a wearable electronic device such as a watch or a head-mounted device). In some embodiments, the computer system has a touch pad. In some embodiments, the computer system has one or more cameras. In some embodiments, the computer system has (e.g., includes or communicates with) a display generating component (e.g., a display device, such as a head-mounted device (HMD), a display, a projector, a touch-sensitive display (also referred to as a "touch screen" or "touch screen display"), or other device or component that presents visual content to a user, for example, on or in the display generating component itself, or that is generated from the display generating component and is otherwise visible). In some embodiments, the computer system has o