Search

EP-4160299-B1 - IMAGE DISPLAY CONTROL METHOD, IMAGE DISPLAY CONTROL APPARATUS, AND HEAD-MOUNTED DISPLAY DEVICE

EP4160299B1EP 4160299 B1EP4160299 B1EP 4160299B1EP-4160299-B1

Inventors

  • XIAO, BING

Dates

Publication Date
20260513
Application Date
20210722

Claims (15)

  1. An image display control method (600) for a head-mounted display device, the head-mounted display device comprising an optical imaging apparatus (210), an image source apparatus (230), a displacement apparatus (220), and an image display control apparatus (240), the image source apparatus (230) being optically aligned with the optical imaging apparatus (210) and having a plurality of source images, each source image having a different virtual scene display depth, the method (600) being implemented by the image display control apparatus (240), the image display control method (600) comprising: determining (610) a displacement sequence of the displacement apparatus (220) and a source image display sequence of the image source apparatus (230) based on a virtual scene display depth range, each displacement in the displacement sequence corresponding to one source image in the source image display sequence and to one virtual scene display depth in the virtual scene display depth range, wherein the virtual scene display depth range comprises a plurality of virtual scene display depths; determining, based on each virtual scene display depth, a corresponding displacement and a source image with the virtual scene display depth; controlling (620) the displacement apparatus (220) to move, according to the displacement sequence, the image source apparatus (230) to corresponding target positions; and after the image source apparatus (230) reaches each of the target positions, controlling (630), according to the source image display sequence, the image source apparatus (230) to perform source image projection, so as to display a clear virtual image of a source image, corresponding to the target position, in the source image display sequence.
  2. The image display control method (600) according to claim 1, wherein after the image source apparatus (230) reaches each of the target positions, controlling, according to the source image display sequence, the image source apparatus (230) to perform source image projection comprises: after the image source apparatus (230) reaches each of the target positions, controlling the image source apparatus (230) to perform image blurring on a first source image set, the first source image set being obtained by removing a source image, corresponding to the target position, in the source image display sequence from the plurality of source images; and controlling the image source apparatus (230) to project the plurality of blurred source images to display a clear virtual image of the source image, corresponding to the target position, in the source image display sequence.
  3. The image display control method (600) according to claim 2, wherein the first source image set is obtained by removing some or all of previously projected source images, and the source image, corresponding to the target position, in the source image display sequence from the plurality of source images.
  4. The image display control method (600) according to one of claims 1 to 3, wherein each source image of the plurality of source images having a virtual scene display depth outside the virtual scene display depth range is in an image blurred state at each time of source image projection.
  5. The image display control method (600) according to one of claims 1 to 4, wherein controlling the displacement apparatus (220) to move, according to the displacement sequence, the image source apparatus (230) to corresponding target positions comprises: controlling the displacement apparatus (220) to sequentially or cyclically move, according to the displacement sequence, the image source apparatus (230) to corresponding target positions.
  6. The image display control method according to claim 5, wherein if a cyclic movement period of the image source apparatus (230) is greater than a duration of vision of human eyes, a specified source image of the plurality of source images is in an image blurred state at each time of source image projection.
  7. The image display control method (600) according to claim 1, wherein the virtual scene display depth range is generated based on a human eye gaze depth sequence of a user of the head-mounted display device.
  8. The image display control method (600) according to claim 7, wherein the human eye gaze depth sequence is acquired by a depth acquisition apparatus.
  9. The image display control method (600) according to one of claims 1 to 8, wherein after the image source apparatus (230) reaches each of the target positions, controlling, according to the source image display sequence, the image source apparatus (230) to perform source image projection comprises: for a certain target position of the target positions, after the image source apparatus (230) reaches the target position, controlling the image source apparatus (230) to select a source image corresponding to the target position from the source image display sequence; and controlling the image source apparatus (230) to project the selected source image to display a corresponding clear virtual image.
  10. The image display control method (600) according to claim 1, wherein the virtual scene display depth range is determined according to a human eye gaze depth range of a user of the head-mounted display device.
  11. The image display control method (600) according to claim 1, wherein the virtual scene display depth range is determined according to a movable range of the image source apparatus.
  12. The image display control method (600) according to one of claims 1 to 11, further comprising: rapidly moving the displacement apparatus (220) so that a time interval of the displacement apparatus moving from any target position to another target position is less than or equal to a duration of vision of human eyes; and after the displacement apparatus (220) is moved each time, controlling the image source apparatus (230) to project a corresponding source image so that a time interval of updated display of virtual images with different depth information less than or equal to the duration of vision of human eyes.
  13. The image display control method according to claim 2, wherein controlling the image source apparatus (230) to perform image blurring on a first source image set comprises: performing low-pass filtering smoothing on image sources in the first image source set to obtain out-of-focus source images with an out-of-focus display effect; and determining the obtained out-of-focus image sources as blurred source images.
  14. A head-mounted display device, comprising: an optical imaging apparatus (210); an image source apparatus (230); a displacement apparatus (220); and an image display control apparatus (240) configured to perform the image display control method of any one of claims 1 to 13, wherein the image source apparatus (230) is optically aligned with the optical imaging apparatus.
  15. A machine-readable storage medium storing executable instructions, which, when executed on an electronic device (3000), cause at least one processor (3010) to: based on a virtual scene display depth range, determine a displacement sequence of a displacement apparatus (220) and a source image display sequence of an image source apparatus (230), each displacement in the displacement sequence corresponding to a source image in the source image display sequence and to a virtual scene display depth in the virtual scene display depth range; control the displacement apparatus (220) to move, according to the displacement sequence, the image source apparatus (230) to corresponding target positions; and after the image source apparatus (230) reaches each of the target positions, control, according to the source image display sequence, the image source apparatus (230) to perform source image projection, so as to display a clear virtual image of a source image, corresponding to the target position, in the source image display sequence, and to perform the image display control method of any one of claims 1 to 13.

Description

TECHNICAL FIELD Embodiments of the present description relate to the field of computer vision, and specifically relate to an image display control method, an image display control apparatus, and a head-mounted display device. BACKGROUND Virtual reality technology is a computer simulation technology that can be used to create and experience a virtual world. It uses data in real life and combines it with various output devices by electronic signals generated by computer technology, to transform it into phenomena that can be perceived by people, thereby simulating a world in reality by computer technology such that users are immersed in the environment. Virtual reality technology is recognized by more and more people. Users can experience the most real perceptions in a virtual reality world, in which a simulated environment is indistinguishable from the real world, such that people feel like being personally on the scene. In addition, virtual reality technology can truly achieve human-computer interaction, such that people can operate at will and get the most real feedback from the environment in the process of operation. The existential, multi-perceptual, interactive and other characteristics of virtual reality technology have made virtual reality technology popular with many people. In virtual reality display, a wearer of a head-mounted display device expects that when he is viewing virtual scene images, depth positions of the virtual images can be dynamically adjusted according to depth changes of virtual scene depth planes being viewed, to achieve dynamic depth display of the virtual images. Currently, there is still no better solution to the above problem in the art. CN 110 727 111 A discloses a head-mounted display system which comprises a projection display device and an eye-tracking device. The eye-tracking device is used for identifying a fixation point and a fixation depth of a human eye. The projection display device is configured for adjusting a virtual image distance according to the fixation point and the fixation depth information, so that the virtual imaging distance is matched with a human eye fixation depth detected by the eye-tracking device. WO 2018/196968 A1 relates to a device and a method for generating a 3D light field, in particular, for a Head Mount Display device. In particular, the device and the method can be used to produce different depth layers of the 3D light field within a frame duration. WO 2016/105521 A1 discloses methods and systems which provide three dimensional gradient and dynamic light fields for display in 3D technologies, in particular 3D augmented reality (AR) devices, by coupling visual accommodation and visual convergence to the same plane at any depth of an object of interest in real time. CN 103 487 939 B discloses an adjustable head-mount display optical system which comprises an image source module, a beam shaping system, an optical combiner and at least one set of fine adjusting mechanism. The fine adjusting mechanism is used for adjusting the position relation of the image source module, the beam shaping system and the optical combiner, and/or adjusting the position relation of optical elements in the beam shaping system, so that an adjustment of the distance between a virtual image and the human eyes is achieved. SUMMARY It is the object of the present invention to improve depth perception in virtual reality. This object is solved by the subject matter of the independent claims. Embodiments are defined in the dependent claims. In view of the above problem, embodiments of the present description provide an image display control method for a head-mounted display device, an image display control apparatus, and a head-mounted display device. Using this method and apparatus, a displacement sequence of a displacement apparatus and a source image display sequence of an image source apparatus are determined based on a virtual scene display depth range, and the image source apparatus is cyclically moved to target positions according to the displacement sequence, and a corresponding source image is projected when the image source apparatus reaches each target position, thereby achieving dynamic display of virtual images on virtual scene depth planes. According to an aspect of embodiments of the present description, there is provided an image display control method for a head-mounted display device. The head-mounted display device includes an optical imaging apparatus, an image source apparatus, a displacement apparatus and an image display control apparatus. The image source apparatus is optically aligned with the optical imaging apparatus and have a plurality of source images. Each source image has a different virtual scene display depth. The method is performed by the image display control apparatus. The image display control method includes: determining a displacement sequence of the displacement apparatus and a source image display sequence of the image source ap