US-12620063-B2 - Image generation device, autonomous cleaner and image generation method

Abstract

An image generation device, for generating an output image with first absolute depth information, comprising: at least one image sensor, configured to generate at least one first image with a first resolution and at least one second image with a second resolution, wherein the first image does not have depth information and the second image has second absolute depth information, wherein the first resolution is higher than the second resolution; and a processing circuit, configured to use a depth estimation engine to generate at least one third image with a third resolution according to the first image, wherein the third image has relative depth information; wherein the processing circuit further uses a fusion engine to fuse the second image and the third image to generate the output image.

Inventors

• Guo-Zhen Wang

Assignees

• PIXART IMAGING INC.

Dates

Publication Date

20260505

Application Date

20231213

Claims (20)

1. 1 . An image generation device, for generating an output image with first absolute depth information, comprising: at least one image sensor, configured to generate at least one first image with a first resolution and at least one second image with a second resolution, wherein the first image does not have depth information and the second image has second absolute depth information, wherein the first resolution is higher than the second resolution; and a processing circuit, configured to use a depth estimation engine to generate at least one third image with a third resolution according to the first image, wherein the third image has relative depth information, wherein the depth estimation engine uses deep-learning based monocular depth estimation; wherein the processing circuit further uses a fusion engine to fuse the second image and the third image to generate the output image.
2. 2 . The image generation device of claim 1 , wherein the third resolution is higher than the second resolution.
3. 3 . The image generation device of claim 1 , wherein an output resolution of the output image is higher than the second resolution.
4. 4 . The image generation device of claim 3 , wherein the output resolution is equal to the third resolution.
5. 5 . The image generation device of claim 1 , wherein a first frame rate of the first image is higher than a second frame rate of the second image.
6. 6 . The image generation device of claim 1 , wherein a third frame rate of the third image is higher than a second frame rate of the second image.
7. 7 . The image generation device of claim 6 , wherein the third frame rate is lower than a first frame rate of the first image.
8. 8 . The image generation device of claim 1 , wherein an output frame rate of the output image is lower than a third frame rate of the third image, wherein the output frame rate is lower than or equal to a second frame rate of the second image.
9. 9 . An autonomous cleaner, comprising: an image generation device, for generating an output image with first absolute depth information, comprising: at least one image sensor, configured to generate at least one first image with a first resolution and at least one second image with a second resolution, wherein the first image does not have depth information and the second image has second absolute depth information, wherein the first resolution is higher than the second resolution; and a processing circuit, configured to use a depth estimation engine to generate at least one third image with a third resolution according to the first image, wherein the third image has relative depth information, wherein the depth estimation engine uses deep-learning based monocular depth estimation; wherein the processing circuit further uses a fusion engine to fuse the second image and the third image to generate the output image; and a control circuit, configured to control operations of the autonomous cleaner according to the first absolute depth information.
10. 10 . The autonomous cleaner of claim 9 , wherein the third resolution is higher than the second resolution.
11. 11 . The autonomous cleaner of claim 9 , wherein an output resolution of the output image is higher than the second resolution.
12. 12 . The autonomous cleaner of claim 11 , wherein the output resolution is equal to the third resolution.
13. 13 . The autonomous cleaner of claim 9 , wherein a first frame rate of the first image is higher than a second frame rate of the second image.
14. 14 . The autonomous cleaner of claim 9 , wherein a third frame rate of the third image is higher than a second frame rate of the second image.
15. 15 . The autonomous cleaner of claim 14 , wherein the third frame rate is lower than a first frame rate of the first image.
16. 16 . The autonomous cleaner of claim 9 , wherein an output frame rate of the output image is lower than a third frame rate of the third image, wherein the output frame rate is lower than or equal to a second frame rate of the second image.
17. 17 . An image generation method, for generating an output image with first absolute depth information, comprising: generating at least one first image with a first resolution and at least one second image with a second resolution, wherein the first image does not have depth information and the second image has second absolute depth information, wherein the first resolution is higher than the second resolution; using a depth estimation engine to generate at least one third image with a third resolution according to the first image, wherein the third image has relative depth information, wherein the depth estimation engine uses deep-learning based monocular depth estimation; and using a fusion engine to fuse the second image and the third image to generate the output image.
18. 18 . The image generation method of claim 17 , wherein the third resolution is higher than the second resolution.
19. 19 . The image generation method of claim 17 , wherein an output resolution of the output image is higher than the second resolution.
20. 20 . The image generation method of claim 19 , wherein the output resolution is equal to the third resolution.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image generation device, an autonomous cleaner and an image generation method, and particularly relates to an image generation device, an autonomous cleaner and an image generation method which can reduce the computation effort and cost of images with absolute depth information. 2. Description of the Prior Art As technology advances, an autonomous cleaner (e.g., a robot cleaner) may have a 3D depth sensor which generates images with absolute depth information. The autonomous cleaner may use the images to determine a location thereof and surrounding environments. However, the 3D depth sensor is always expensive thus increases a cost of the autonomous cleaner. Additionally, the computation of the 3D depth sensor is complicated, thus a frame rate or a resolution of the images with absolute depth information may be low, which may cause wrong distance determination while the autonomous cleaner is moving fast. SUMMARY OF THE INVENTION One objective of the present invention is to provide an image generation device which can reduce the computation effort and cost of images with absolute depth information. Another objective of the present invention is to provide an autonomous cleaner which can reduce the computation effort and cost of images with absolute depth information. Still another objective of the present invention is to provide an image generation method which can reduce the computation effort and cost of images with absolute depth information. One embodiment of the present invention discloses an image generation device, for generating an output image with first absolute depth information, comprising: at least one image sensor, configured to generate at least one first image with a first resolution and at least one second image with a second resolution, wherein the first image does not have depth information and the second image has second absolute depth information, wherein the first resolution is higher than the second resolution; and a processing circuit, configured to use a depth estimation engine to generate at least one third image with a third resolution according to the first image, wherein the third image has relative depth information; wherein the processing circuit further uses a fusion engine to fuse the second image and the third image to generate the output image. Another embodiment of the present invention discloses an autonomous cleaner, comprising an image generation device and a control circuit. The image generation device, which is for generating an output image with first absolute depth information, comprises: at least one image sensor, configured to generate at least one first image with a first resolution and at least one second image with a second resolution, wherein the first image does not have depth information and the second image has second absolute depth information, wherein the first resolution is higher than the second resolution; and a processing circuit, configured to use a depth estimation engine to generate at least one third image with a third resolution according to the first image, wherein the third image has relative depth information; wherein the processing circuit further uses a fusion engine to fuse the second image and the third image to generate the output image. The control circuit controls operations of the autonomous cleaner according to the first absolute depth information. Still another embodiment of the present invention discloses an image generation method, for generating an output image with first absolute depth information, comprising: generating at least one first image with a first resolution and at least one second image with a second resolution, wherein the first image does not have depth information and the second image has second absolute depth information, wherein the first resolution is higher than the second resolution; using a depth estimation engine to generate at least one third image with a third resolution according to the first image, wherein the third image has relative depth information; and using a fusion engine to fuse the second image and the third image to generate the output image. In view above-mentioned embodiments, the distance determination can keep accurate while the computation effort and cost of images with absolute depth information are reduced. These and other objectives of the present invention will no doubt become obvious to those of ordinary skill in the art after reading the following detailed description of the preferred embodiment that is illustrated in the various figures and drawings. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram illustrating an image generation device according to one embodiment of the present invention. FIG. 2 is a schematic diagram illustrating detail operations of the image generation device shown in FIG. 1, according to one embodiment of the present invention. FIG. 3 is a schematic diagram illu