EP-4034935-B1 - VOLUMETRIC DISPLAY INCLUDING LIQUID CRYSTAL-BASED LENSES

EP4034935B1EP 4034935 B1EP4034935 B1EP 4034935B1EP-4034935-B1

Inventors

JAMALI, AFSOON
ZHAO, YANG
LAM, WAI SZE TIFFANY
LU, Lu
LANMAN, Douglas, Robert

Dates

Publication Date: 20260506
Application Date: 20200906

Claims (15)

A volumetric display comprising: a two-dimensional display (203); a varifocal optical system (205) configured to receive image light from the two-dimensional display and focus the image light; and at least one processor (560) configured to: control the two-dimensional display to cause a plurality of sub-frames associated with an image frame to be displayed by the display, wherein each subframe of the plurality of sub-frames includes a corresponding portion of image data associated with the image frame; control the varifocal optical system to a corresponding focal state for each respective sub-frame; and coordinate a focal state of the varifocal optical system and the two-dimensional display by controlling the varifocal optical system to achieve a selected optical state associated with a sub-frame prior to the image light associated with the sub-frame propagating through the varifocal optical system.
The volumetric display of claim 1, wherein the varifocal optical system comprises a plurality of optical stages, and wherein each optical stage of the plurality of optical stages comprises a focusing optical element.
The volumetric display of claim 2, wherein the focusing optical element comprises a polarization sensitive focusing optical element, and wherein at least one optical stage of the plurality of optical stages further comprises a switchable wave retarder.
The volumetric display of claim 3, wherein the switchable wave retarder comprises a switchable half waveplate.
The volumetric display of claim 3 or claim 4, wherein the switchable wave retarder comprises at least one ferroelectric liquid crystal cell.
The volumetric display of any of claims 3 to 5, wherein the polarization sensitive focusing optical element comprises at least one of a Pancharatnam-Berry phase, PBP, lens, a polarization sensitive hologram, PSH, lens, a metamaterial, or a liquid crystal optical phase array.
The volumetric display of any preceding claim, wherein the at least one processor is further configured to generate the plurality of sub-frames by binning pixels of the image frame into bins based on depth values associated with the pixels.
The volumetric display of any preceding claim, wherein the plurality of subframes together recreate the image frame.
A system comprising: a head mounted display comprising: a volumetric display according to any preceding claim; and a housing; wherein the two-dimensional display mechanically is coupled to the housing, and wherein the varifocal optical system is mechanically coupled to the housing.
The system of claim 9, further comprising a console communicatively coupled to the head mounted display, wherein the console is configured to generate the plurality of sub-frames by binning pixels of the image frame into bins based on depth values associated with the pixels.
A method comprising: causing, by one or more processors, a plurality of sub-frames associated with an image frame to be displayed by a two-dimensional display, wherein each sub-frame of the plurality of sub-frames includes a corresponding portion of image data associated with the image frame, and wherein the plurality of sub-frames together include all of the image data associated with the image frame; controlling, by the one or more processors, a varifocal optical system to a corresponding focal state for each respective sub-frame, wherein the varifocal optical system is configured to receive image light from the two-dimensional display and focus the image light; and coordinating, by the one or more processors, a focal state of the varifocal optical system and the two-dimensional display by controlling the varifocal optical system to achieve a selected optical state associated with a sub-frame prior to the image light associated with the sub-frame propagating through the varifocal optical system.
The method of claim 11, wherein the varifocal optical system comprises a plurality of optical stages, and wherein each optical stage of the plurality of optical stages comprises a focusing optical element.
The method of claim 12, wherein the focusing optical element comprises a polarization sensitive focusing optical element, wherein at least one optical stage of the plurality of optical stages further comprises a switchable wave retarder, and wherein controlling the varifocal optical system comprises controlling a state of the switchable wave retarder.
The method of claim 13, wherein the polarization sensitive focusing optical element comprises at least one of a Pancharatnam-Berry phase, PBP, lens, a polarization sensitive hologram, PSH, lens, a metamaterial, or a liquid crystal optical phase array; and preferably wherein the polarization sensitive focusing optical element comprises an active polarization sensitive focusing optical element comprises controlling a state of the active polarization sensitive focusing optical element.
The method of any one of claims 11 to 14, further comprising generating, by the one or more processors, the plurality of sub-frames by binning pixels of the image frame into bins based on depth values associated with the pixels; and/or preferably further comprising generating, by a console, the plurality of sub-frames by binning pixels of the image frame into bins based on depth values associated with the pixels.

Description

TECHNICAL FIELD This invention relates to a volumetric display, and to a corresponding system and method. The disclosure generally relates to artificial reality systems, such as mixed reality and/or virtual reality systems. BACKGROUND Artificial reality systems have applications in many fields such as computer gaming, health and safety, industry, and education. As a few examples, artificial reality systems are being incorporated into mobile devices, gaming consoles, personal computers, movie theaters, and theme parks. In general, artificial reality is a form of reality that has been adjusted in some manner before presentation to a user, which may include, e.g., a virtual reality, an augmented reality, a mixed reality, a hybrid reality, or some combination and/or derivatives thereof. Typical artificial reality systems include one or more devices for rendering and displaying content to users. As one example, an artificial reality system may incorporate a head-mounted display (HMD) worn by a user and configured to output artificial reality content to the user. The artificial reality content may entirely consist of content that is generated by the system or may include generated content combined with real-world content (e.g., pass through views or captured real-world video and/or images of a user's physical environment). During operation, the user typically interacts with the artificial reality system to select content, launch applications, configure the system and, in general, experience artificial reality environments. US 2017/301313 A1 describes a near-eye display system that includes a display assembly comprising at least one display panel and a display driver to drive the display assembly to display a sequence of frames in a low-persistence mode based on a frame clock signal having a first frequency. The near-eye display system further includes a pair of liquid membrane lenses facing the display assembly, and a lens driver having an output coupled to an input of each of liquid membrane lenses of the pair, the lens driver to generate at the output a periodic, continuously variable driving signal having a second frequency, wherein the first frequency is an integer multiple of the second frequency. As the liquid membrane lenses are synchronized with the low-persistence display of the sequence of frames, each displayed frame is perceived through the liquid membrane lenses at a different nearly constant focal depth, and thus creating a perception to the user of multiple focal planes in the displayed imagery. TAO ZHAN ET AL: "High-resolution additive light field near-eye display by switchable Pancharatnam-Berry phase lenses", OPTICS EXPRESS, vol. 26, no. 4, 15 February 2018, page 4863, describes that conventional head-mounted displays present different images to each eye, and thereby create three-dimensional (3D) sensation lor viewers. This method can only control the stimulus to vergence but not accommodation, which is located at the apparent location of the physical displays. The disrupted coupling between vergence and accommodation could cause considerable visual discomfort. SUMMARY In general, the disclosure describes artificial reality systems and, more specifically, an artificial reality system that includes a volumetric display. As used herein, a volumetric display is a display that forms a visual representation of an object or a scene in apparent three-dimensions, rather than a two-dimensional planar image. The volumetric display described herein may include a two-dimensional planar display and a varifocal optical system. One or more processors may be configured to process an image frame to generate a plurality of sub-frames. Each sub-frame includes only a portion of the image data from the image frame. The portion of the image data for each sub-frame corresponds to a depth or depth range within the image frame. Taken together, the plurality of sub-frames collectively represent all image date in the frame, but each sub-frame only includes some of the image data. To produce the volumetric image for display, the one or more processors is configured to coordinate the output of the sub-frames by the display and the focal distance of the varifocal optical system such that the focal distance of the varifocal optical system correlates with the depth associated with the displayed sub-frame. To display an entire image frame, the one or more processors controls the display to output the sub-frames in a sequence and controls the focal distance of the varifocal optical system as each of the sub-frames are displayed. This technique takes advantage of persistence of vision, which allows the user's visual system to effectively combine the portions of the image data included in the subframes to recreate the image frame. According to an aspect of the invention, there is provided a volumetric display according to claim 1. In some embodiments, the varifocal optical system comprises a plurality of optical stages, and wherei