US-12619346-B1 - Displaying content based on detecting an associated context in a physical environment

Abstract

Some examples of the disclosure are directed to systems and methods for displaying content such as a user interface element of an application executing on the electronic device upon detecting a context (e.g., detecting a physical object and optionally the satisfaction of one or more criteria) for displaying the user interface element in a three-dimensional environment presented at an electronic device. Examples of the disclosure are directed to improving the user experience by automatically displaying a user interface element when certain conditions are satisfied, such as when the context of the three-dimensional environment is appropriate (e.g., the context for displaying the user interface element is detected).

Inventors

• Thomas G. Salter
• Jeffrey S. Norris
• Christopher I. Word
• Michael J. Rockwell

Assignees

• APPLE INC.

Dates

Publication Date

20260505

Application Date

20240926

Claims (20)

1. 1 . A method comprising: at an electronic device in communication with one or more displays and one or more input devices including one or more cameras: detecting, using the one or more input devices including the one or more cameras, one or more physical objects in a physical environment; detecting an enrolled association between a first context and a first user interface of a weather application, the first context including detection of a window; in accordance with the detection of the enrolled association between the first context and the first user interface of the weather application and in response to a determination that one or more criteria are satisfied, the one or more criteria including a criterion that is satisfied when the one or more physical objects include a detected window, determining that the first context is detected; in accordance with determining that the first context is detected, displaying, using the one or more displays, the first user interface of the weather application; and in accordance with determining that the first context is not detected, forgoing displaying the first user interface of the weather application.
2. 2 . The method of claim 1 , wherein the one or more criteria include one or more of: a criterion that is satisfied when one or more weather parameters associated with an outside environment corresponding to a location of the electronic device satisfy one or more thresholds; a criterion that is satisfied when the one or more physical objects include one or more physical objects in an outside environment corresponding to a location of the electronic device that are visible through the window; a criterion that is satisfied when a distance between the window and the electronic device is within a threshold distance; a criterion that is satisfied when a direction of gaze is directed at the window or directed at a physical object that is visible through the window; a criterion that is satisfied when a duration of gaze directed at the window for greater than a threshold duration; criterion that is satisfied when a shape of the window corresponds to a predetermined shape, or one or more dimensions of the window are greater than one or more threshold dimensions; a criterion that is satisfied when an orientation or viewport of the electronic device is directed at the window; a criterion that is satisfied when a location of the electronic device is within a predetermined region of the physical environment; a criterion that is satisfied when a time of detecting the first context corresponds to a predetermined time period; or a criterion that is satisfied when the window is a predetermined window or type of window.
3. 3 . The method of claim 1 , wherein the first user interface of the weather application is customized in accordance with the one or more criteria of the first context.
4. 4 . The method of claim 1 , further comprising: detecting movement of the electronic device; and in accordance with detecting the movement of the electronic device, updating a location, an orientation, a size or a shape of the first user interface of the weather application.
5. 5 . The method of claim 1 , further comprising: detecting one or more inputs directed to the first user interface of the weather application; and in response to detecting the one or more inputs directed to the first user interface of the weather application: displaying a second user interface of the weather application; and ceasing displaying the first user interface of the weather application.
6. 6 . The method of claim 1 , further comprising: detecting an enrolled association between a second context and a second user interface of the weather application, the second user interface different from the first user interface; in accordance with the detection of the enrolled association between the second context and the second user interface of the weather application, determining that the second context is detected; and in accordance with the determination that the second context is detected, concurrently displaying the first user interface of the weather application and the second user interface of the weather application.
7. 7 . The method of claim 1 , further comprising: detecting an enrolled association between a second context and a second user interface of a second application, the second user interface and the second application different from the first user interface and the weather application; in accordance with the detection of the enrolled association between the second context and the second user interface, determining that the second context is detected; and in accordance with the determination that the second context is detected, concurrently displaying, using the one or more displays, the first user interface of the weather application and the second user interface of the second application.
8. 8 . The method of claim 1 , further comprising: in accordance with a determination that the first context is no longer detected, ceasing displaying the first user interface of the weather application.
9. 9 . An electronic device comprising: one or more processors; one or more input devices including one or more cameras; memory; and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for: detecting, using the one or more input devices including the one or more cameras, one or more physical objects in a physical environment; detecting an enrolled association between a first context and a first user interface of a weather application, the first context including detection of a window; in accordance with the detection of the enrolled association between the first context and the first user interface of the weather application and in response to a determination that one or more criteria are satisfied, the one or more criteria including a criterion that is satisfied when the one or more physical objects include a detected window, determining that the first context is detected; in accordance with the determination that the first context is detected, displaying, using the one or more displays, the first user interface of the weather application; and in accordance with determining that the first context is not detected, forgoing displaying, using the one or more displays, the first user interface of the weather application.
10. 10 . The electronic device of claim 9 , wherein the one or more criteria include one or more of: a criterion that is satisfied when one or more weather parameters associated with an outside environment corresponding to a location of the electronic device satisfy one or more thresholds; a criterion that is satisfied when the one or more physical objects include one or more physical objects in an outside environment corresponding to a location of the electronic device that are visible through the window; a criterion that is satisfied when a distance between the window and the electronic device is within a threshold distance; a criterion that is satisfied when a direction of gaze is directed at the window or directed at a physical object that is visible through the window; a criterion that is satisfied when a duration of gaze directed at the window for greater than a threshold duration; criterion that is satisfied when a shape of the window corresponds to a predetermined shape, or one or more dimensions of the window are greater than one or more threshold dimensions; a criterion that is satisfied when an orientation or viewport of the electronic device is directed at the window; a criterion that is satisfied when a location of the electronic device is within a predetermined region of the physical environment; a criterion that is satisfied when a time of detecting the first context corresponds to a predetermined time period; or a criterion that is satisfied when the window is a predetermined window or type of window.
11. 11 . The electronic device of claim 9 , wherein the first user interface of the weather application is customized in accordance with the one or more criteria of the first context.
12. 12 . The electronic device of claim 9 , the one or more programs further including instructions for: detecting movement of the electronic device; and in accordance with detecting of movement of the electronic device, updating a location, an orientation, a size or a shape of the first user interface of the weather application.
13. 13 . The electronic device of claim 9 , the one or more programs further including instructions for: detecting one or more inputs directed to the first user interface of the weather application; and in response to detecting the one or more inputs directed to the first user interface of the weather application: displaying a second user interface of the weather application; and ceasing displaying the first user interface of the weather application.
14. 14 . The electronic device of claim 9 , the one or more programs further including instructions for: detecting an enrolled association between a second context and a second user interface of the weather application, the second user interface different from the first user interface; in accordance with the detection of the enrolled association between the second context and the second user interface of the weather application, determining that the second context is detected; and in accordance with the determination that the second context is detected, concurrently displaying the first user interface of the weather application and the second user interface of the weather application.
15. 15 . The electronic device of claim 9 , the one or more programs further including instructions for: detecting an enrolled association between a second context and a second user interface of a second application, the second user interface and the second application different from the first user interface and the weather application; in accordance with the detection of the enrolled association between the second context and the second user interface, determining that the second context is detected; and in accordance with the determination that the second context is detected, concurrently displaying, using the one or more displays, the first user interface of the weather application and the second user interface of the second application.
16. 16 . The electronic device of claim 9 , the one or more programs further including instructions for: in accordance with a determination that the first context is no longer detected, ceasing displaying the first user interface of the weather application.
17. 17 . A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device including one or more input devices including one or more cameras, cause the electronic device to: detect, using the one or more input devices including the one or more cameras, one or more physical objects in a physical environment; detect an enrolled association between a first context and a first user interface of a weather application, the first context including detection of a window; in accordance with the detection of the enrolled association between the first context and the first user interface of the weather application and in response to determining that one or more criteria are satisfied the one or more criteria including a criterion that is satisfied when the one or more physical objects include a detected window, determine that the first context is detected; in accordance with the determination that the first context is detected, display, using the one or more displays, the first user interface of the weather application; and in accordance with determining that the first context is not detected, forgo displaying, using the one or more displays, the first user interface of the weather application.
18. 18 . The non-transitory computer readable storage medium of claim 17 , wherein the one or more criteria include one or more of: a criterion that is satisfied when one or more weather parameters associated with an outside environment corresponding to a location of the electronic device satisfy one or more thresholds; a criterion that is satisfied when the one or more physical objects include one or more physical objects in an outside environment corresponding to a location of the electronic device that are visible through the window; a criterion that is satisfied when a distance between the window and the electronic device is within a threshold distance; a criterion that is satisfied when a direction of gaze is directed at the window or directed at a physical object that is visible through the window; a criterion that is satisfied when a duration of gaze directed at the window for greater than a threshold duration; criterion that is satisfied when a shape of the window corresponds to a predetermined shape, or one or more dimensions of the window are greater than one or more threshold dimensions; a criterion that is satisfied when an orientation or viewport of the electronic device is directed at the window; a criterion that is satisfied when a location of the electronic device is within a predetermined region of the physical environment; a criterion that is satisfied when a time of detecting the first context corresponds to a predetermined time period; or a criterion that is satisfied when the window is a predetermined window or type of window.
19. 19 . The non-transitory computer readable storage medium of claim 17 , wherein the first user interface of the weather application is customized in accordance with the one or more criteria of the first context.
20. 20 . The non-transitory computer readable storage medium of claim 17 , further storing instructions which, when executed by the one or more processors, further cause the electronic device to perform: detecting movement of the electronic device; and in accordance with detecting of movement of the electronic device, updating a location, an orientation, a size or a shape of the first user interface of the weather application.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Application No. 63/586,986, filed Sep. 29, 2023, the entire disclosure of which is herein incorporated by reference for all purposes. FIELD OF THE DISCLOSURE This relates generally to systems and methods of displaying content such as user interface elements of an application based on detecting an associated context in a physical environment. BACKGROUND OF THE DISCLOSURE Some computer graphical environments provide two-dimensional and/or three-dimensional environments where at least some objects displayed for a user's viewing are virtual and generated by a computer. In some examples, a physical environment (e.g., including one or more physical objects) is presented, optionally along with one or more virtual objects, in a three-dimensional environment. SUMMARY OF THE DISCLOSURE Some examples of the disclosure are directed to systems and methods for displaying content such as one or more user interface elements of an application executing on the electronic device based on detection of one or more contexts (e.g., detecting a physical object and optionally the satisfaction of one or more criteria) in a computer-generated environment. In some examples, the electronic device captures, via one or more cameras, a portion of one or more physical environments (e.g., indoor and outdoor environments) in the field of view of the one or more cameras of the electronic device, and presents, via the one or more displays, representations of the one or more physical objects within the one or more physical environment. In some examples, the electronic device detects user input, via one or more input devices, to enroll an association between a specific physical object (e.g., closet or window), optionally one or more criteria, and a user interface of an application (e.g., a weather application and/or reminder application). Upon enrolling the association, a context including detection of the specific physical object and the optional satisfaction of one or more criteria is established. In other examples, the electronic device detects user input directed at opting into a pre-enrolled association between a generic physical object (e.g., a generic window or window-like object that includes a transparent portion), optionally one or more criteria, and a user interface of an application. Upon opting into the pre-enrolled associations, a context including the generic physical object and the optional satisfaction of one or more criteria is established. In some examples, more than one context is associated with a physical object. In some examples, after any associations between contexts and user interface elements have been enrolled, and/or after any pre-enrolled associations between contexts and user interface elements have been accepted, the electronic device detects a physical object in the field of view of the one or more cameras, and determines whether any contexts associated with that physical object are detected. In response to detection of a context, the electronic device displays, via one or more displays, an associated user interface element of an application. The full descriptions of these examples are provided in the Drawings and the Detailed Description, and it is understood that this Summary does not limit the scope of the disclosure in any way. BRIEF DESCRIPTION OF THE DRAWINGS For improved understanding of the various examples described herein, reference should be made to the Detailed Description below along with the following drawings. Like reference numerals often refer to corresponding parts throughout the drawings. FIG. 1 illustrates an electronic device presenting an extended reality environment according to some examples of the disclosure. FIG. 2 illustrates a block diagram of an example architecture for an electronic device according to some examples of the disclosure. FIGS. 3A-3E illustrate an electronic device displaying one or more weather user interface elements when one or more contexts are detected according to some examples of the disclosure. FIG. 4A illustrates an example of an electronic device that has moved in the physical environment (relative to the example of FIG. 3E), has detected a different context as compared to FIG. 3E, and is displaying a user interface element of an associated weather application according to some examples of the disclosure. FIG. 4B illustrates an example of an electronic device that differs from the example depicted in FIG. 4A by displaying an additional and different user interface element of an additionally associated reminder application in response to detecting an additional context (as compared to FIG. 4A) according to some examples of the disclosure. FIGS. 5A-5B illustrate examples of an electronic device that differ from the examples depicted in FIGS. 3C-4B by displaying a different user interface element of an associated calendar application in response to detecting a different