US-12626620-B2 - Method for projecting image content onto the retina of a user, image processing device for carrying out the method and an optical system for a virtual retinal scan display

US12626620B2US 12626620 B2US12626620 B2US 12626620B2US-12626620-B2

Abstract

A method for projecting image content onto the retina of a user using an optical system. Image data are captured using the image processing device. A first subset of the image data is blanked, thereby generating an active second subset of image data. The first and second subsets make up the total amount of the image data. The projector unit is controlled using the image data such that there is always only one eye box, generated on a common imaging path, with the same active image data arranged in the region of a first pupil region of the user. The first pupil region surrounds the pupil center. A portion of the eye box is arranged with a portion of the active image data within a second pupil region of the user. The second pupil region surrounds the pupil center. The second pupil region is arranged within the first pupil region.

Inventors

Hendrik Specht
Andreas Petersen
Christian Nitschke
Mazyar Sabbar
Nikolai Suchkov
Tadiyos Alemayehu

Assignees

ROBERT BOSCH GMBH

Dates

Publication Date: 20260512
Application Date: 20220708
Priority Date: 20210908

Claims (20)

1 . A method for projecting image content onto a retina of a user using an optical system, the image system including: an image source which is configured to supply image content in the form of image data, an image processing device for the image data, a projector unit with a temporally modulatable light source configured to generate at least one light beam and with a controllable deflection devicefor the at least one light beam for scanning projection of the image content, a redirection unit onto which the image content can be projected and which is configured to direct the projected image content onto an eye of the user, an optical segmentation element, which is arranged between the projector unit and the redirection unit and is configured to project the image content via at least two different imaging paths onto at least one projection region of the redirection unit, wherein at least some of the imaging paths are individually controllable, and an optical replication component, which is arranged in the at least one projection region of the redirection unit and is configured to direct the projected image content in a replicated and spatially offset manner onto the eye of the user, so that a plurality of mutually spatially offset eye boxes with the image content is generated, wherein the method comprises the following steps: capturing the image data using the image processing device; blanking at least one first subset of the image data thereby generating an active second subset of the image data, wherein the first subset of the image data and the active second subset of the image data make up a total amount of the image data; and controlling the projector unit using the blanked first subset of the image data and active second subset of of the image data such that there is always only one eye box, generated on a common imaging path, with the same active image data arranged in a region of a first pupil region of the user, wherein the first pupil region surrounds a pupil center, and at least a portion of the eye box is arranged with at least a portion of the active image data within a second pupil region of the user, wherein the second pupil region surrounds the pupil center, wherein the second pupil region is arranged within the first pupil region.
2 . The method according to claim 1 , further comprising: blanking at least one third subset of the image data, thereby generating an active fourth subset of the image data, wherein the third subset of the image data and the active fourth subset of the image data make up the total amount of the image data, wherein the active second subset of the image data and the active fourth subset of the image differ at least partially from one another.
3 . The method according to claim 2 , wherein the first subset of the image data, the active second subset of the image data, the third subset of the image data, and the active fourth subset of the image data enable a projection of at least a portion of the image content via at least four different imaging paths using the optical segmentation element onto at least one projection region of the redirection unit.
4 . The method according to claim 3 , further comprising: blanking at least one fifth subset of the image data, thereby generating an active sixth subset of the image data, wherein the fifth subset of the image data and the active sixth subset of the image data make up the total amount of the image data; and blanking at least one seventh subset of the image data, thereby generating an active eighth subset of the image data, wherein the seventh subset of the image data and the active eighth subset of the image data make up the total amount of the image data, wherein the active second subset of the image data, the active fourth subset of the image data, the active sixth subset of the image data, and the active eighth subset of the image data differ at least partially from one another.
5 . The method according to claim 4 , wherein all active subsets of the image data includig the active second subset of the image data, the active fourth subset of the image data, the active sixth subset of the image data, and the active eighth subset of the image data, together make up at least the complete image content.
6 . The method according to claim 5 , wherein all active subsets of the image data together make up more than the complete image content.
7 . The method according to claim 6 , wherein the active subsets of the image data at least partially overlap.
8 . The method according to claim 7 , further comprising: adjusting a brightness of the active subsets of the image data in an overlap region of the active subsets of the image data such that a distribution of a brightness of the projected image content is substantially uniform.
9 . The method according to claim 1 , wherein the first pupil region has a largest assumed pupil diameter of the user.
10 . The method according to claim 1 , wherein the second pupil region has a smallest assumed pupil diameter of the user.
11 . The method according to claim 1 , wherein the blanked first subset of the image data is selected such that the generated active second subset of the image data includes at least the image data with image content situated closest to the pupil center in the plurality of mutually spatially offset eye boxes, on an eye pupil plane.
12 . The method according to claim 4 , wherein, depending on a position of the pupil center of the user relative to the generated eye boxes, at least two of the blanked first, third, fifth, and seventh subsets of the image data are substantially the same size.
13 . The method according to claim 12 , wherein, depending on the pupil position of the user, the second active subset, fourth active subset, sixth active subset, and eighth active subset each correspond to a quarter of a total amount of the image data.
14 . The method according to claim 1 , wherein, using the optical segmentation element and the optical replication component, the plurality of eye boxes is generated such that the eye boxes are arranged in a grid.
15 . The method according to claim 14 , wherein the eye boxes are arranged in a rectangle shape or a parallelogram shape or a diamond shape in which two internal angles have an angle size of 60°.
16 . The method according to claim 1 , further comprising: predistorting the image data using the image processing device such that a distortion of the image content via the at least two imaging paths is at least partially compensated for.
17 . The method according to claim 16 , wherein the predistortion and the blanking of the image data are carried out simultaneously.
18 . The method according to claim 1 , wherein a position of the pupil center of the user relative to the generated eye boxes is detected using an eye tracker.
19 . An image processing device, configured to: capture image data; blank at least one first subset of the image data, thereby generating an active second subset of the image data, wherein the first subset of the image data and the active second subset of the image data make up the total amount of the image data; and control a projector unit using the blanked first subset of the image data and the active second subset of the image data such that there is always only one eye box generated on a common imaging path, with the same active image data arranged in a region of a first pupil region of the user, wherein the first pupil region surrounds a pupil center, and at least a portion of the eye box is arranged with at least a portion of the active image data within a second pupil region of the user, wherein the second pupil region surrounds the pupil center, wherein the second pupil region is arranged within the first pupil region.
20 . An optical system for a virtual retinal scan display, comprising: an image source, which supplies image content in the form of image data, an image processing device for the image data; a projector unit with a temporally modulatable light source configured to generate at least one light beam and with a controllable deflection device for the at least one light beam for scanning projection of the image content; a redirection unit onto which the image content can be projected and which is configured to direct the projected image content onto an eye of a user; an optical segmentation element, which is arranged between the projector unit and the redirection unit and is configured to project the image content via at least two different imaging paths onto at least one projection region of the redirection unit, wherein at least some of the imaging paths are individually controllable; and an optical replication component, which is arranged in the at least one projection region of the redirection unit and is configured to direct the projected image content in a replicated and spatially offset manner onto the eye of the user, so that a plurality of mutually spatially offset eye boxes with the image content is generated; wherein the image processing unit is configured to: capture the image data, blank at least one first subset of the image data, thereby generating an active second subset of the image data, wherein the first subset of the image data and the active second subset of the image data make up the total amount of the image data, and control the projector unit using the blanked first subset of the image data and the active second subset of the image data such that there is always only one eye box generated on a common imaging path, with the same active image data arranged in a region of a first pupil region of the user, wherein the first pupil region surrounds a pupil center, and at least a portion of the eye box is arranged with at least a portion of the active image data within a second pupil region of the user, wherein the second pupil region surrounds the pupil center, wherein the second pupil region is arranged within the first pupil region.

Description

FIELD The present invention relates to a method for projecting image content onto the retina of a user with the aid of an optical system. In addition, the present invention relates to an image processing device for carrying out the method and to an optical system for a virtual retinal scan display having the image processing device. BACKGROUND INFORMATION A method for projecting image content onto the retina of a user with the aid of an optical system is described in U.S. Patent Application Publication No. US 2016/0377865 A1. The distance between the eye pupil and the redirection unit integrated in the spectacle lens can be changed, for example, by shifting the spectacles along the nose, as a result of which the eye pupil no longer lies in the exit pupil plane of the optical system. Because the bundles of beams arising due to the redirection of the light beams onto the eye by means of the redirection unit are considerably fanned out outside the exit pupil plane, double images can occur for the user. Proceeding from this problem, it is an object of the present invention to develop a method for projecting image content onto the retina of a user, in which double images are avoided largely independently of the distance between the eye pupil and the redirection unit. SUMMARY In order to achieve the object, a method for projecting image content onto the retina of a user with the aid of an optical system is provided according to the present invention. In addition, an image processing device according and an optical system for a virtual retinal scan display are provided according to the present invention. According to an example embodiment of the present invention, the optical system with the aid of which the method for projecting image content onto the retina of the user is carried out has at least one image source, which supplies image content in the form of image data. The image data are in particular in the form of color image data, e.g., RGB image data. In particular, the image data can be in the form of non-moving or moving images, e.g. videos. In addition, the optical system has a projector unit with a temporally modulatable light source for generating at least one light beam and with a controllable deflection device for the at least one light beam for scanning projection of the image content. The projector unit is in particular designed to emit the image content from the image data in the form of scanned and/or rasterized light beams. The deflection device is in particular designed as a MEMS mirror (micromirror actuator), at least for controlled deflection of the at least one light beam of the light source of the projector unit. In addition, the optical system has a redirection unit onto which the image content can be projected and which is designed to direct the projected image content onto an eye of a user. The redirection unit comprises in particular an arrangement of optical elements, for example diffractive, reflective, refractive and/or holographic optical elements. Preferably, however, the redirection unit always comprises at least one holographic optical element. The redirection unit is in particular provided to redirect only a portion of the intensity of the projected image content onto the user's eye. At least a further portion of the intensity of the projected image content passes through the redirection unit. The redirection unit appears substantially transparent to a user at least from a perpendicular viewing direction. In particular, the redirection unit forms a projection region. Furthermore, the optical system comprises an optical segmentation element that is arranged between the projector unit and the redirection unit and is designed to project the image content via at least two different imaging paths onto at least one projection region of the redirection unit. Here, at least some imaging paths are individually controllable. In particular, the complete image content is selectively projected via one of the at least two different imaging paths onto the at least one projection region of the redirection unit at a first point in time. In particular, a first portion of the image data is projected via a first of the at least two different imaging paths at the first point in time, and a second portion of the image data is projected via a second of the at least two different imaging paths at a second point in time, following the first point in time, onto the at least one projection region of the redirection unit. In particular, a first portion of the image data is projected via the first of the at least two different imaging paths, and the second portion of the image data is projected via the second of the at least two different imaging paths onto the at least one projection region of the redirection unit simultaneously at the first point in time. The optical segmentation element can in particular be designed as a spatially segmented optical element, which is provided in particular to carry out a s