KR-102960626-B1 - Dynamic convergence adjustment of augmented reality headsets

KR102960626B1KR 102960626 B1KR102960626 B1KR 102960626B1KR-102960626-B1

Abstract

A system and method are disclosed for dynamically laterally moving each virtual object displayed by an augmented reality headset by a respective distance when each virtual object is displayed, in order to change the virtual depth from a first virtual depth to a second virtual depth. Each distance may be determined based on the lateral distance between a first convergence vector of the user's eye having each virtual object at the first virtual depth and a second convergence vector of the user's eye having each virtual object at the second virtual depth along the display, and may be based on the interpupillary distance. In this way, the display of the virtual object can be adjusted so that the user's line of sight converges to where the virtual object appears to be.

Inventors

린 유-젠
고어젠 패트릭 존
그라함 마틴 에번

Assignees

유니버셜 시티 스튜디오스 엘엘씨

Dates

Publication Date: 20260507
Application Date: 20191209
Priority Date: 20181219

Claims (20)

As an augmented reality system, Augmented reality headset and Includes processor and memory, but, The above augmented reality headset is, A left display configured to display the left image of a left virtual object to the user's left eye, and A right display configured to display the right image of a right virtual object to the user's right eye—wherein the left image and the right image appear as a first image to the user when viewing the left image and the right image, and the left virtual object and the right virtual object appear as the first virtual object of the first image to the user when viewing the left virtual object and the right virtual object—and, It includes a pupil tracking sensor configured to detect and provide an indication of the user's pupil position, and The above memory stores instructions, and When the above instruction is executed by the processor, the processor, Determining the interpupillary distance of the user based on the indication of the above pupil position, and Provides a first adjustment for displaying the first virtual object as the virtual depth from the first image to the second image is changed based on the above-mentioned inter-particle distance, wherein the first adjustment includes a left-side adjustment of the left virtual object displayed on the left display toward a first inner edge or a first outer edge of the left display, and a right-side adjustment of the right virtual object displayed on the right display toward a second inner edge or a second outer edge of the right display. A second adjustment is provided for displaying a second virtual object as the virtual depth from the first image to the second image is changed based on the above inter-papular distance, and The augmented reality headset is configured to display the first virtual object within the second image based on the first adjustment, and The augmented reality headset is made to display the second virtual object within the second image based on the second adjustment, and The first adjustment for the first virtual object is different from the second adjustment for the second virtual object, and The above virtual depth is changed from a first virtual depth to a second virtual depth, and the first adjustment is calculated for the first virtual object based on a first convergence vector at the first virtual depth and a second convergence vector at the second virtual depth, Augmented reality system.
In Article 1, The above left side adjustment and the above right side adjustment include distance, The processor is configured to cause the augmented reality headset to display the first virtual object by laterally moving the left virtual object and the right virtual object by the distance, Augmented reality system.
In Article 2, The processor is configured to cause the augmented reality headset to display the first virtual object by laterally moving the left virtual object toward the first outer edge of the left display by the distance when the second virtual depth is greater than the first virtual depth. Augmented reality system.
In Article 2, The processor is configured to cause the augmented reality headset to display the first virtual object by laterally moving the left virtual object toward the first inner edge of the left display by the distance when the second virtual depth is smaller than the first virtual depth. Augmented reality system.
In Article 2, The processor is configured to cause the augmented reality headset to display the first virtual object by laterally moving the right virtual object toward the second outer edge of the right display by the distance when the second virtual depth is greater than the first virtual depth. Augmented reality system.
In Article 2, The processor is configured to cause the augmented reality headset to display the first virtual object by laterally moving the right virtual object toward the second inner edge of the right display by the distance when the second virtual depth is smaller than the first virtual depth. Augmented reality system.
In Article 1, The processor is configured to cause the augmented reality headset to display the first virtual object by laterally shifting the first center of the left virtual object, laterally shifting the second center of the right virtual object, or both of these. Augmented reality system.
In Article 1, The above left display and the above right display include a transparent display, The above left display is configured to display the above left image and the above left virtual object overlapping with the real world environment, and The right display above is configured to display the right image and the right virtual object overlapping with the real world environment. Augmented reality system.
As a type of non-transient computer-readable medium comprising instructions for controlling the display of multiple virtual objects, When the above instruction is executed by the processor, the processor, Receiving an indication that each of the plurality of virtual objects will be displayed as moving from each first virtual depth to each second virtual depth, Determining the interpupillary distance, Based on the above inter-coordinate distance, the respective lateral distance between each first line of sight associated with each of the plurality of virtual objects at each first virtual depth and each second line of sight associated with the virtual object at each second virtual depth is dynamically determined, and An augmented reality headset is made to display each virtual object of the plurality of virtual objects moving from each first virtual depth to each second virtual depth based on each lateral distance by laterally adjusting each virtual object of the plurality of virtual objects along a display line passing through a plurality of displays of the augmented reality headset. The plurality of virtual objects includes a first virtual object and a second virtual object, each lateral distance to the first virtual object includes a first lateral distance, each lateral distance to the second virtual object includes a second lateral distance, and the first lateral distance is different from the second lateral distance. Type of non-transient computer-readable medium.
In Article 9, The instruction that causes the processor to dynamically determine each of the above-mentioned side distances causes the processor, Half of the above interpupillary distance is divided by the respective first virtual distance between the center point between the user's pupils and the respective reference point of each virtual object of the respective first virtual depth to determine each first quotient, and Determining each first lateral pupil distance by multiplying each first quotient by each first display distance between the user's pupil and the displays of the plurality of displays, Type of non-transient computer-readable medium.
In Article 10, The above reference point is the proximity center point of each of the above virtual objects, Type of non-transient computer-readable medium.
In Article 10, The instruction that causes the processor to dynamically determine each of the above-mentioned side distances causes the processor, Determining each second quotient by dividing half of the above interpupillary distance by the respective second virtual distance between the center point between the user's pupils and the reference point of each virtual object at each of the respective second virtual depths, and Determining each second lateral pupil distance by multiplying each of the above second quotients by each second display distance between the user's pupil and the plurality of displays, Type of non-transient computer-readable medium.
In Article 12, Each of the above lateral distances includes the difference between each of the above first lateral pupil distances and each of the above second lateral pupil distances, Type of non-transient computer-readable medium.
In Article 9, The above display line passes through the center of each of the plurality of displays of the augmented reality headset, Type of non-transient computer-readable medium.
As a method for adjusting the display of virtual objects, The step of receiving a first indication that the first displayed object will be displayed as moving from the first virtual depth of the first image to the second virtual depth of the second image, and The step of receiving a second indication that the second displayed object will be displayed as moving from the third virtual depth of the first image to the fourth virtual depth of the second image, and A step for determining the interpupillary distance, and A step of determining a first lateral distance between a first convergence vector associated with the first displayed object at the first virtual depth and a second convergence vector associated with the first displayed object at the second virtual depth based on the above inter-coordinate distance, and A step of determining a second lateral distance between a third convergence vector associated with the second displayed object at the third virtual depth and a fourth convergence vector associated with the second displayed object at the fourth virtual depth based on the above inter-coordinate distance, and A step of displaying the first displayed object by laterally adjusting it toward the inner edge of the display of the augmented reality headset by the first lateral distance or toward the outer edge of the display by the first lateral distance, thereby moving the first displayed object from the first virtual depth of the first image to the second virtual depth of the second image based on the first lateral distance; A method comprising the step of displaying the second displayed object by moving it from the third virtual depth of the first image to the fourth virtual depth of the second image by laterally adjusting the second displayed object toward the inner edge of the display by the second lateral distance or toward the outer edge of the display by the second lateral distance. method.
In Article 15, The first convergence vector passes through the user's eye and the reference point of the first displayed object at the first virtual depth, and the second convergence vector passes through the user's eye and the reference point of the first displayed object at the second virtual depth, method.
In Article 15, The above first side distance is measured at the display, method.
In Article 15, The first displayed object is partially displayed as a left display object on the left display of the augmented reality headset and partially displayed as a right display object on the right display of the augmented reality headset. method.
In Article 18, The step of displaying the first displayed object by moving it from the first virtual depth to the second virtual depth includes laterally moving the left display object toward the outer edge of the left display by the lateral distance when the second virtual depth is greater than the first virtual depth. method.
In Article 18, The step of displaying the first displayed object by moving it from the first virtual depth to the second virtual depth comprises, in response to determining that the second virtual depth is smaller than the first virtual depth, laterally moving the left display object toward the inner edge of the left display by the lateral distance. method.

Description

Dynamic convergence adjustment of augmented reality headsets This application claims priority to U.S. Provisional Application No. 62/777,545, titled “DYNAMIC CONVERGENCE ADJUSTMENT IN AUGMENTED REALITY HEADSETS”, filed on December 10, 2018, the entirety of which is incorporated herein by reference for all purposes. The present disclosure generally relates to any other suitable interactive computer-generated experience taking place within augmented reality, virtual reality, mixed reality, or a simulated environment. More specifically, certain embodiments of the present disclosure relate to operational features of a headset that provides an interactive computer-generated experience. For example, augmented reality systems are gaining increasing popularity. It is now recognized that existing technologies providing augmented reality systems, such as augmented reality headsets, can benefit from improvements in system components and functions. In particular, it is now acknowledged that some traditional systems and technologies providing augmented reality visual effects can cause sensory conflict. Therefore, there is a need to provide improved augmented reality systems configured to limit or prevent such sensory conflicts. This section is intended to introduce the reader to various aspects of the technology that may be related to the various aspects of the present technology described and/or claimed below. It is believed that this discussion will help provide the reader with background information to facilitate a better understanding of the various aspects of the present disclosure. Accordingly, it should be understood that these statements should be read in this context, not as an acknowledgment of prior art. Specific embodiments corresponding to the scope of the disclosed invention are summarized below. These embodiments are not intended to limit the scope of the disclosure, but rather are intended to provide a brief summary of the disclosed embodiments. In practice, the disclosure may include various forms that may be identical or different from the embodiments described below. In particular, in one embodiment, the augmented reality system includes an augmented reality headset having a left display that displays a left virtual image to the user's left eye. The augmented reality headset also includes a right display that displays a right virtual image to the user's right eye. When viewing the left virtual image and the right virtual image, the left virtual image and the right virtual image appear to the user as a single virtual image. The augmented reality headset further includes a pupil tracking sensor that detects and provides an indication of the user's pupil position. The augmented reality system also includes a convergence adjustment system having a pupil distance determination engine that determines the user's pupil distance based on receiving the indication of the pupil position. The convergence adjustment system also includes a display adjustment engine that provides an indication that the virtual object is changing its virtual depth based on the pupil distance and provides adjustments to the display of the virtual object in the virtual image. The convergence adjustment system further includes a processor that displays the virtual object based on adjustments from the display adjustment engine. In another embodiment, a non-transient computer-readable medium of a type is provided having instructions for coordinating the display of a virtual object, which, when executed by a processor, cause the processor to receive an indication that the virtual object will be displayed as moving from each first virtual depth to each second virtual depth. The instructions also cause the processor to determine a co-pathic distance and, based on the co-pathic distance, dynamically determine each lateral distance between each first line of sight associated with each virtual object at each first virtual depth and each second line of sight associated with the virtual object at each second virtual depth. The instructions also cause the processor to display each virtual object as moving from each first virtual depth to each second virtual depth based on each lateral distance. In another embodiment, a method for adjusting the display of a virtual object includes the step of receiving an indication that one or more displayed objects will be displayed as moving from a first virtual depth to a second virtual depth. The method also includes the step of determining a co-occurrence distance and, based on the co-occurrence distance, determining a lateral distance between a first convergence vector associated with an object displayed at the first virtual depth and a second convergence vector associated with an object displayed at the second virtual depth. The method further includes the step of displaying an object displayed as moving from a first virtual depth to a second virtual depth based on the lateral distance. These and other featu