US-12627787-B2 - Distance-based vertical misalignment correction

Abstract

A method for correcting vertical misalignment in a binocular display system comprises receiving a signal from a misalignment detection system comprising information related to vertical misalignment between a left eye display and a right eye display of the binocular display system. Image content displayed via the binocular display system is analyzed to determine a distance to a foreground virtual object in the image content at which a user is gazing. The method further comprises analyzing depth image data to determine a distance to a background object in a real-world environment. A vertical misalignment correction strategy is determined based at least upon the distance to the foreground virtual object and the distance to the background object. Based upon the vertical misalignment correction strategy, the binocular display system is controlled to correct the vertical misalignment.

Inventors

• Michaela PORUBANOVA
• Bjorn Nicolaas Servatius Vlaskamp
• Jacob ELLIOTT HADNETT-HUNTER

Assignees

• MICROSOFT TECHNOLOGY LICENSING, LLC

Dates

Publication Date

20260512

Application Date

20230403

Claims (19)

1. 1 . A head-mounted display device, comprising: a binocular display system comprising a left eye display and a right eye display, the binocular display system configured to display image content; a depth imaging system configured to obtain depth image data of a real-world environment; a misalignment detection system configured to monitor vertical display misalignment of the binocular display system; and a controller configured to receive a signal from the misalignment detection system comprising information related to vertical misalignment between the left eye display and the right eye display, analyze the image content to determine a distance from the head-mounted display device to a foreground virtual object in the image content at which a user is gazing, analyze the depth image data to determine a distance from the head-mounted display device to a background object in the real-world environment, determine a degree of coplanarity of the foreground virtual object and the background object based at least upon the distance from the head-mounted display device to the foreground virtual object and the distance from the head-mounted display device to the background object, determine a rate at which to correct the vertical misalignment based at least upon the degree of coplanarity, and control the binocular display system to correct the vertical misalignment at the rate determined.
2. 2 . The head-mounted display device of claim 1 , wherein the distance from the head-mounted display device to the foreground virtual object comprises a distance to a vergence plane.
3. 3 . The head-mounted display device of claim 1 , wherein the controller is configured to determine the distance from the head-mounted display device to the background object in a foveal area of the user.
4. 4 . The head-mounted display device of claim 1 , wherein the controller is configured to analyze the image content and analyze the depth image data after the image content is displayed.
5. 5 . The head-mounted display device of claim 1 , wherein the controller is configured to correct the vertical misalignment at a relatively slower rate in response to the foreground virtual object being relatively closer to the background object, and correct the vertical misalignment at a relatively faster rate in response to the foreground virtual object being relatively farther from the background object.
6. 6 . The head-mounted display device of claim 5 , wherein the relatively faster rate comprises a rate of 10 arcminutes per second or greater.
7. 7 . The head-mounted display device of claim 5 , wherein the relatively faster rate is instantaneous.
8. 8 . The head-mounted display device of claim 5 , wherein the relatively slower rate comprises 1-10 arcminutes per second.
9. 9 . The head-mounted display device of claim 5 , wherein the controller is configured to correct the vertical misalignment at the relatively slower rate in response to the background object being located in the user's peripheral vision.
10. 10 . The head-mounted display device of claim 1 , wherein the controller is further configured to map the image content to different pixel rows for display to correct the vertical misalignment.
11. 11 . The head-mounted display device of claim 1 , wherein the controller is further configured to, after an update interval, update the distance from the head-mounted display device to the foreground virtual object and the distance from the head-mounted display device to the background object.
12. 12 . A method for correcting vertical misalignment in a binocular display system, the method comprising: receiving a signal from a misalignment detection system comprising information related to vertical misalignment between a left eye display and a right eye display of the binocular display system; analyzing image content displayed via the binocular display system to determine a distance from the binocular display system to a foreground virtual object in the image content at which a user is gazing; analyzing depth image data to determine a distance from the binocular display system to a background object in a real-world environment; determining a degree of coplanarity of the foreground virtual object and the background object based at least upon the distance from the binocular display system to the foreground virtual object and the distance from the binocular display system to the background object; determining a rate at which to correct the vertical misalignment based at least upon the degree of coplanarity; and controlling the binocular display system to correct the vertical misalignment at the rate determined.
13. 13 . The method of claim 12 , wherein analyzing the depth image data comprises determining the distance from the binocular display system to the background object in a foveal area of the user.
14. 14 . The method of claim 12 , further comprising: correcting the vertical misalignment at a relatively faster rate in response to the foreground virtual object being relatively farther from the background object, and correcting the vertical misalignment at a relatively slower rate in response to the foreground virtual object being relatively closer to the background object.
15. 15 . The method of claim 14 , further comprising correcting the vertical misalignment at the relatively slower rate in response to the background object being located in the user's peripheral vision.
16. 16 . The method of claim 14 , wherein correcting the vertical misalignment at the relatively slower rate comprises correcting the vertical misalignment at a rate of 1-10 arcminutes per second.
17. 17 . A computing device, comprising: a processor configured to: receive a signal from a misalignment detection system comprising information related to vertical misalignment between a left eye display and a right eye display of a binocular display system; analyze image content displayed via the binocular display system to determine a distance from the computing device to a foreground virtual object in the image content at which a user is gazing; determine a distance from the computing device to a background object in a real-world environment; determine a degree of coplanarity of the foreground virtual object and the background object based at least upon the distance from the computing device to the foreground virtual object and the distance from the computing device to the background object, determine a rate at which to correct the vertical misalignment based at least upon the degree of coplanarity, and control the binocular display system to correct the vertical misalignment at the rate determined.
18. 18 . The computing device of claim 17 , wherein the processor is further configured to determine the distance from the computing device to the background object in a foveal area of the user.
19. 19 . The computing device of claim 17 , wherein the processor is further configured to: correct the vertical misalignment at a relatively slower rate in response to the foreground virtual object being relatively closer to the background object, and correct the vertical misalignment at a relatively faster rate in response to the foreground virtual object being relatively farther from the background object.

Description

BACKGROUND Mixed-reality (MR) systems, including virtual-reality (VR) and augmented-reality (AR) systems, can include separate left eye and right eye display components. The separate display components can display left eye and right eye images from different perspectives to provide a stereoscopic viewing experience for the user. Human vision systems fuse the different images, enabling the user to perceive the displayed content at a predetermined depth and/or location within a MR scene. SUMMARY This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that solve any or all disadvantages noted in any part of this disclosure. Examples are disclosed that relate to correcting vertical misalignment in a binocular display system. One example provides a method comprising receiving a signal from a misalignment detection system. The signal comprises information related to vertical misalignment between a left eye display and a right eye display of the binocular display system. Image content displayed via the binocular display system is analyzed to determine a distance to a foreground virtual object in the image content at which a user is gazing. The method further comprises analyzing depth image data to determine a distance to a background object in a real-world environment, and determining a vertical misalignment correction strategy based at least upon the distance to the foreground virtual object and the distance to the background object. Based upon the vertical misalignment correction strategy, the binocular display system is controlled to correct the vertical misalignment. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 shows an example of a real-world environment including a real-world background object and a head-mounted display (HMD) device that displays a foreground virtual object. FIG. 2 schematically illustrates an example of a left eye image and a right eye image that can be displayed by the HMD of FIG. 1. FIG. 3 schematically illustrates an example of vertical misalignment between the left eye image and the right eye image of FIG. 2 that causes the human vision system not to fuse the left eye and right eye images. FIG. 4 illustrates an example HMD. FIGS. 5A-5B show an example of the foreground virtual object of FIG. 1 located a first distance from the real-world background object, and a resulting non-fusion of vertically offset left eye and right eye images. FIGS. 6A-6B show an example of the foreground virtual object of FIG. 1 located a second distance from the real-world background object, and a resulting fusion of vertically offset left eye and right eye images. FIG. 7 schematically illustrates a foveal region within an example field of view for the HMD of FIG. 1. FIG. 8 illustrates an example mapping of rendered pixel data to display hardware prior to performing a vertical offset correction. FIG. 9 illustrates an example remapping of the rendered pixel data of FIG. 8 to the display hardware of FIG. 8 following a vertical offset correction. FIG. 10 shows another example of a real-world environment including a real-world background object that is in a peripheral view of a user gazing at a virtual object. FIG. 11 is a block diagram of an example method for correcting vertical misalignment between left eye and right eye displays in a binocular display system. FIG. 12 is a block diagram of an example computing system. DETAILED DESCRIPTION As introduced above, mixed-reality (MR) systems, including virtual-reality (VR) and augmented-reality (AR) systems, can include separate left eye display and right eye display components to display different left and right eye images for stereoscopic image presentation. FIG. 1 shows an example real-world environment 100 in which a user 102 is wearing a head-mounted display (HMD) device 104. As described in more detail below, the HMD 104 comprises a binocular display system 106. The binocular display system 106 comprises a left eye display 108L and a right eye display 108R. The binocular display system 106 can display image content 110, such as a foreground virtual object 112, against a real world background 113. FIG. 2 shows an example of a left eye image 109L for display via the left eye display 108L of FIG. 1 and a right eye image 109R for display via the right eye display 108R. As illustrated, the left eye image 109L and the right eye image 109R are not vertically offset, with reference to the illustrated dashed line. Human vision systems can fuse the left eye image 109L and the right eye image 109R, enabling the user to perceive an apparent three-dimensional image of the foreground virtual object 112 at a predetermined depth and/or lo