US-12626473-B2 - Dynamic virtual objects

Abstract

Some implementations offset or reduce user fatigue and/or discomfort by making relatively small (e.g., potentially imperceptible) changes to relative positions of virtual objects (e.g., virtual screens) or entire scenes to cause or encourage the user to make small adjustments, e.g., changing their posture. By periodically making small changes over long periods of time, the changes may be imperceptible, inspire unconscious user posture changes or movements, and/or improve the user experience without burdening or distracting the user from their desired activity, e.g., watching a movie, working on a virtual UI interface, etc.

Inventors

• Jack R. Dashwood

Assignees

• APPLE INC.

Dates

Publication Date

20260512

Application Date

20240711

Claims (20)

1. 1 . A method, comprising: at a head mounted device (HMD) having a processor: determining a user muscle fatigue condition associated with maintaining head, neck, or eye state stability during presentation of views via the HMD, the views comprising virtual content positioned within an extended reality (XR) environment; in accordance with determining the user muscle fatigue condition, determining a sequence of modifications to alter the views, wherein the sequence of modifications comprises: view-to-view modifications that change a parameter less than a first threshold amount between consecutive views during the presentation of the views; and a cumulative modification of the parameter over the sequence of modifications that changes the parameter more than a second threshold amount; and presenting the views in accordance with the determined sequence of modifications.
2. 2 . The method of claim 1 , wherein the cumulative modification is configured to encourage a change in: a stable head-to-neck alignment; a stable head position; a stable head orientation; or a stable eye focus level.
3. 3 . The method of claim 1 , wherein the parameter is: a vertical position of a virtual screen upon which the virtual content is displayed within the XR environment; a size of a virtual screen upon which the virtual content is displayed within the XR environment; or a yaw or tilt of a virtual screen upon which the virtual content is displayed within the XR environment.
4. 4 . The method of claim 1 , wherein the parameter is: a virtual focal distance used by the HMD to display the views, or a position of a display panel in the HMD.
5. 5 . The method of claim 1 , wherein determining the user muscle fatigue condition comprises: determining that a head, eyes, or neck of a user have remained stable for more than a third threshold of time, wherein stability of the head, eyes, or neck is determined based on an inertial measurement unit (IMU) of the HMD; determining a type of the virtual content corresponds with head, eye, or neck stability; or determining a user movement pattern associated with discomfort or fatigue.
6. 6 . The method of claim 1 wherein the parameter adjusts background content separate from the virtual content, wherein the background content is included as passthrough video in the views.
7. 7 . The method of claim 1 , wherein the parameter is oscillated between two endpoint values over time.
8. 8 . The method of claim 1 , wherein the parameter is modified: to encourage an accommodation shift; based on detecting a user response to modification of the parameter; or based on a positional constraint based on anchoring of content within the XR environment.
9. 9 . A head mounted device (HMD) comprising: a non-transitory computer-readable storage medium; and one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the HMD to perform operations comprising: determining a user muscle fatigue condition associated with maintaining head, neck, or eye state stability during presentation of views via the HMD, the views comprising virtual content positioned within an extended reality (XR) environment; in accordance with determining the user muscle fatigue condition, determining a sequence of modifications to alter the views, wherein the sequence of modifications comprises: view-to-view modifications that change a parameter less than a first threshold amount between consecutive views during the presentation of the views; and a cumulative modification of the parameter over the sequence of modifications that changes the parameter more than a second threshold amount; and presenting the views in accordance with the determined sequence of modifications.
10. 10 . The HMD of claim 9 , wherein the cumulative modification is configured to encourage a change in: a stable head-to-neck alignment; a stable head position; a stable head orientation; or a stable eye focus level.
11. 11 . The HMD of claim 9 , wherein the parameter is: a vertical position of a virtual screen upon which the virtual content is displayed within the XR environment; a size of a virtual screen upon which the virtual content is displayed within the XR environment; or a yaw or tilt of a virtual screen upon which the virtual content is displayed within the XR environment.
12. 12 . The HMD of claim 9 , wherein the parameter is: a virtual focal distance used by the HMD to display the views, or a position of a display panel in the HMD.
13. 13 . The HMD of claim 9 , wherein determining the user muscle fatigue condition comprises: determining that a head, eyes, or neck of a user have remained stable for more than a third threshold of time, wherein stability of the head, eyes, or neck is determined based on an inertial measurement unit (IMU) of the HMD; determining a type of the virtual content corresponds with head, eye, or neck stability; or determining a user movement pattern associated with discomfort or fatigue.
14. 14 . The HMD of claim 9 , wherein the parameter adjusts background content separate from the virtual content, wherein the background content is included as passthrough video in the views.
15. 15 . The HMD of claim 9 , wherein the parameter is oscillated between two endpoint values over time.
16. 16 . The HMD of claim 9 , wherein the parameter is modified: to encourage an accommodation shift; based on detecting a user response to modification of the parameter; or based on a positional constraint based on anchoring of content within the XR environment.
17. 17 . A non-transitory computer-readable storage medium, storing program instructions executable via a processor to perform operations comprising: determining a user muscle fatigue condition associated with maintaining head, neck, or eye state stability during presentation of views via a head-mounted device (HMD), the views comprising virtual content positioned within an extended reality (XR) environment; in accordance with determining the user muscle fatigue condition, determining a sequence of modifications to alter the views, wherein the sequence of modifications comprises: view-to-view modifications that change a parameter less than a first threshold amount between consecutive views during the presentation of the views; and a cumulative modification of the parameter over the sequence of modifications that changes the parameter more than a second threshold amount; and presenting the views in accordance with the determined sequence of modifications.
18. 18 . The non-transitory computer-readable storage medium of claim 17 , wherein the cumulative modification is configured to encourage a change in: a stable head-to-neck alignment; a stable head position; a stable head orientation; or a stable eye focus level.
19. 19 . The non-transitory computer-readable storage medium of claim 17 , wherein the parameter is: a vertical position of a virtual screen upon which the virtual content is displayed within the XR environment; a size of a virtual screen upon which the virtual content is displayed within the XR environment; or a yaw or tilt of a virtual screen upon which the virtual content is displayed within the XR environment.
20. 20 . The non-transitory computer-readable storage medium of claim 17 , wherein the parameter is: a virtual focal distance used by the HMD to display the views, or a position of a display panel in the HMD.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This Application claims the benefit of U.S. Provisional Application Ser. No. 63/536,584 filed Sep. 5, 2023, which is incorporated herein in its entirety. TECHNICAL FIELD The present disclosure generally relates to electronic devices, and in particular, to systems, methods, and devices for reducing user fatigue and/or discomfort during use of electronic devices. BACKGROUND Users of electronic devices (e.g., head mounted devices (HMDs), wearable electronic devices, desktop computers, laptops, mobile devices, televisions, and other electronic devices) sometimes experience fatigue and associated discomfort when using those devices for relatively long periods of time. For example, a user holding their head, neck, and/or torso in a generally static position to view a movie on a virtual screen positioned within an extended reality (XR) environment may experience fatigue and associated discomfort over the course of the movie. SUMMARY As discussed above, users of electronic devices sometimes experience fatigue when using those devices for long periods of time during which there may be little or no head, neck, eye, torso, or other body movements. Some implementations offset or reduce user fatigue or the associated discomfort by making relatively small (e.g., potentially imperceptible) changes to relative positions of virtual objects (e.g., virtual screens) or entire scenes to cause or encourage the user to make small adjustments, e.g., changing their posture or moving their bodies over time. By periodically making small changes over long periods of time, the changes may be imperceptible, inspire unconscious user posture changes or movements, and/or improve the user experience without burdening or distracting the user from their desired activity, e.g., watching a movie, working on a virtual UI interface, etc. Some implementations identify a user fatigue condition associated with maintaining head, neck, or eye state stability during presentation of content in XR. Doing so may involve subtly changing the content over time (e.g., changing the height of the virtual screen or focal distance used to display the XR) in an imperceptible way to encourage the user to alter muscle usage to reduce the fatigue that might otherwise develop from holding the head, neck, or eyes in a stable/fixed state. Various implementations disclosed herein include devices, systems, and methods for reducing user fatigue and the associated discomfort during use of an electronic device. Such methods may determine a user muscle fatigue condition associated with maintaining head, neck, or eye state stability during presentation of views via the HMD. The views may comprise virtual content positioned within an extended reality (XR) environment. Determining the user muscle fatigue condition may involve determining that the user's head, eyes, and/or neck (or other body portions) have remained stable or are expected to continue in a stable state based on sensor data regarding the user or information about the content, e.g., the type of content being a movie, etc. In accordance with determining the user muscle fatigue condition, the methods may determine a sequence of modifications to alter the views. The sequence of modification may comprise: view-to-view modifications that change a parameter less than a first threshold amount between consecutive views during the presentation of the views; (e.g., changes small enough from frame to frame so that they are not consciously perceptible/objectionable). The sequence of modification may comprise: a cumulative modification of the parameter over the sequence of modifications that changes the parameter more than a second threshold amount (e.g., the changes being cumulatively significant enough to encourage the user to alter muscle usage to reduce fatigue and associated discomfort that might otherwise develop. The methods may present the views in accordance with the determined sequence of modifications. In accordance with some implementations, a non-transitory computer readable storage medium has stored therein instructions that are computer-executable to perform or cause performance of any of the methods described herein. In accordance with some implementations, a device includes one or more processors, a non-transitory memory, and one or more programs; the one or more programs are stored in the non-transitory memory and configured to be executed by the one or more processors and the one or more programs include instructions for performing or causing performance of any of the methods described herein. BRIEF DESCRIPTION OF THE DRAWINGS So that the present disclosure can be understood by those of ordinary skill in the art, a more detailed description may be had by reference to aspects of some illustrative implementations, some of which are shown in the accompanying drawings. FIG. 1 illustrates an example operating environment in accordance with some implementations. FIG. 2 illustr