US-12625551-B2 - Method and system for performing eye tracking in augmented reality devices

Abstract

A wearable device for projecting image light to an eye of a viewer and forming an image of virtual content in an augmented reality display is provided. The wearable device includes a projector and stack of waveguides optically connected to the projector. The wearable device also includes an eye tracking system comprising a plurality of illumination sources, an optical element having optical power, and a set of cameras. The optical element is disposed between the plurality of illumination sources and the set of cameras. In some embodiments, the augmented reality display includes an eyepiece operable to output virtual content from an output region and a plurality of illumination sources. At least some of the plurality of illumination sources overlap with the output region.

Inventors

• Giovanni Garcia
• Christian Melo
• Daniel Farmer
• Jason Allen Shultz
• Bach Nguyen
• Charles Robert Schabacker
• Michael Shoaee

Assignees

• MAGIC LEAP, INC.

Dates

Publication Date

20260512

Application Date

20241216

Claims (20)

1. 1 . An augmented reality display comprising: a projector operable to generate virtual content characterized by a rectangular display area; an eyepiece operable to output the virtual content from an output region; a plurality of illumination sources, wherein at least some of the plurality of illumination sources overlap with the output region and are positioned within the rectangular display area; and a set of cameras facing a user and positioned within a width and a height of the output region and the rectangular display area.
2. 2 . The augmented reality display of claim 1 wherein the eyepiece has a world side and an eye side opposing the world side and the plurality of illumination sources are disposed on the eye side of the eyepiece.
3. 3 . The augmented reality display of claim 2 wherein the set of cameras disposed on the eye side.
4. 4 . The augmented reality display of claim 3 wherein the set of cameras are positioned farther from the eyepiece than the plurality of illumination sources.
5. 5 . The augmented reality display of claim 3 further comprising a camera flex cable, wherein the set of cameras are disposed on the camera flex cable.
6. 6 . The augmented reality display of claim 1 wherein the output region is disposed in a plane orthogonal to an optical axis and a projection of each of the plurality of illumination sources parallel to the optical axis is located within the output region.
7. 7 . The augmented reality display of claim 6 wherein the plurality of illumination sources are positioned at three different radial distances from the optical axis.
8. 8 . The augmented reality display of claim 1 wherein the virtual content is output by a diffractive structure in the eyepiece defined by a boundary and the one or more of the plurality of illumination sources are located above the boundary of the diffractive structure in plan view.
9. 9 . The augmented reality display of claim 1 wherein the plurality of illumination sources are disposed in a plane and virtual content passes through the plane at positions surrounding each of the plurality of illumination sources.
10. 10 . The augmented reality display of claim 1 wherein the plurality of illumination sources comprise a set of infrared light emitting diodes.
11. 11 . The augmented reality display of claim 1 further comprising: an optical element having optical power, wherein the plurality of illumination sources are disposed between the optical element and the eyepiece; and wherein the optical element is disposed between the plurality of illumination sources and the set of cameras.
12. 12 . An eye tracking system including: an eyepiece operable to output virtual content over an output region; a plurality of illumination sources overlapping with the output region; an optical element having optical power; and a set of cameras facing a user and disposed within a width and a height of the output region, wherein the optical element is disposed between the plurality of illumination sources and the set of cameras.
13. 13 . The eye tracking system of claim 12 wherein the plurality of illumination sources comprises a set of infrared (IR) light emitting diodes (LEDs) disposed in a plane.
14. 14 . The eye tracking system of claim 12 wherein the plurality of illumination sources are disposed on a world side of the optical element.
15. 15 . The eye tracking system of claim 12 wherein the plurality of illumination sources are laminated between a substrate and a passivation coating.
16. 16 . The eye tracking system of claim 15 wherein the optical element comprises a planar surface and the substrate is bonded to the planar surface of the optical element.
17. 17 . The eye tracking system of claim 12 wherein the optical power is a negative optical power.
18. 18 . The eye tracking system of claim 12 wherein the set of cameras are adjacent to the user and virtual content passes through a plane in which the plurality of illumination sources are disposed and then through the optical element before viewing by the user.
19. 19 . The eye tracking system of claim 12 wherein light from the plurality of illumination sources is configured to propagate through the optical element before reaching the user.
20. 20 . The eye tracking system of claim 12 wherein each of the plurality of illumination sources are mounted with an emission surface facing the optical element, wherein electrical contacts for each of the plurality of illumination sources are disposed between each of the plurality of illumination sources and the optical element.

Description

CROSS-REFERENCES TO RELATED APPLICATIONS This application is a continuation of International Patent Application No. PCT/US2023/025584, filed Jun. 16, 2023, entitled “METHOD AND SYSTEM FOR PERFORMING EYE TRACKING IN AUGMENTED REALITY DEVICES,” which claims the benefit of and priority to U.S. Provisional Patent Application No. 63/353,333, filed on Jun. 17, 2022, entitled “METHOD AND SYSTEM FOR PERFORMING EYE TRACKING IN AUGMENTED REALITY DEVICES,” the disclosure of which is hereby incorporated by reference in its entirety for all purposes. BACKGROUND OF THE INVENTION Modern computing and display technologies have facilitated the development of systems for so called “virtual reality” or “augmented reality” experiences, wherein digitally reproduced images or portions thereof are presented to a viewer in a manner wherein they seem to be, or may be perceived as, real. A virtual reality, or “VR,” scenario typically involves presentation of digital or virtual image information without transparency to other actual real-world visual input; an augmented reality, or “AR,” scenario typically involves presentation of digital or virtual image information as an augmentation to visualization of the actual world around the viewer. Despite the progress made in these display technologies, there is a need in the art for improved methods and systems related to eye tracking in augmented reality systems. SUMMARY OF THE INVENTION The present invention relates generally to methods and systems related to augmented reality systems. More particularly, embodiments of the present invention provide methods and systems for performing eye tracking in which eye tracking illumination sources are positioned within the field of view of the user of the augmented reality system. The invention is applicable to a variety of applications in computer vision and image display systems. Various examples of the present disclosure are provided below. As used below, any reference to a series of examples is to be understood as a reference to each of those examples disjunctively (e.g., “Examples 1-4” is to be understood as “Examples 1, 2, 3, or 4”). Example 1 is an augmented reality display comprising: an eyepiece operable to output virtual content from an output region; and a plurality of illumination sources, wherein at least some of the plurality of illumination sources overlap with the output region. Example 2 is the augmented reality display of example 1, wherein the eyepiece has a world side and an eye side opposing the world side and the plurality of illumination sources are disposed on the eye side of the eyepiece. Example 3 is the augmented reality display of example(s) 1-2 further comprising a set of cameras disposed on the eye side. Example 4 is the augmented reality display of example(s) 1-3, wherein the set of cameras are positioned farther from the eyepiece than the plurality of illumination sources Example 5 is the augmented reality display of example(s) 1-4, wherein the set of cameras are disposed within a width and a height of the output region. Example 6 is the augmented reality display of example(s) 1-5, wherein the output region is disposed in a plane orthogonal to an optical axis and a projection of each of the plurality of illumination sources parallel to the optical axis is located within the output region. Example 7 is the augmented reality display of example(s) 1-6, wherein the plurality of illumination sources are positioned at three different radial distances from the optical axis. Example 8 is the augmented reality display of example(s) 1-7, wherein the virtual content is output by a diffractive structure in the eyepiece defined by a boundary and the one or more of the plurality of illumination sources are located above the boundary of the diffractive structure in plan view. Example 9 is the augmented reality display of example(s) 1-8, wherein the plurality of illumination sources are disposed in a plane and virtual content passes through the plane at positions surrounding each of the plurality of illumination sources. Example 10 is the augmented reality display of example(s) 1-9, wherein the positions surrounding each of the plurality of illumination sources are disposed at substantially 360° with respect to each of the plurality of illumination sources. Example 11 is the augmented reality display of example(s) 1-10, wherein the plurality of illumination sources comprise a set of infrared light emitting diodes. Example 12 is the augmented reality display of example(s) 1-11, further comprising an optical element having optical power, wherein the plurality of illumination sources are disposed between the optical element and the eyepiece. Example 13 is the augmented reality display of example(s) 1-12, further comprising a set of cameras, wherein the optical element is disposed between the plurality of illumination sources and the set of cameras. Example 14 is an eye tracking system including: a plurality of illumination sources; an