US-20260129292-A1 - Hybrid Auto-Focus System with Robust Macro Object Priority Focusing

Abstract

An example method includes displaying a zoomed preview of a scene captured by a camera system. The method includes determining a phase-detect auto-focus (PDAF) depth estimate and a time-of-flight (ToF) depth estimate for the scene. The method includes, based on a comparison of the PDAF and ToF depth estimates, determining whether a foreground object in the zoomed preview is in-focus for a ToF based AF mode. The method includes, based on a determination that the foreground object in the zoomed preview is in-focus for the ToF based AF mode, bypassing a PDAF mode and activating the ToF based AF mode to focus on the foreground object. The method includes displaying, based on the ToF based AF mode, the focused foreground object as part of the zoomed preview of the scene.

Inventors

• Mark Gamadia
• Minchieh Wang
• Jae Soo Kim
• Yang Yang
• Ying Chen Lou

Assignees

• GOOGLE LLC

Dates

Publication Date

20260507

Application Date

20231002

Claims (20)

1. 1 . A computer-implemented method, comprising: displaying, by a display screen of a camera system, a zoomed preview of a scene captured by the camera system; determining a phase-detect autofocus (PDAF) depth estimate and a time-of-flight (ToF) depth estimate for the scene; based on a comparison of the PDAF depth estimate and the ToF depth estimate, determining whether a foreground object in the zoomed preview is in-focus for a ToF based AF mode of the camera system; based on a determination that the foreground object in the zoomed preview is in-focus for the ToF based AF mode, bypassing a PDAF mode and activating the ToF based AF mode to focus on the foreground object, wherein the PDAF mode comprises focusing of the camera system based on the PDAF depth estimate, and wherein the ToF based AF mode comprises focusing of the camera system based on the ToF depth estimate; and displaying, by the display screen and based on the ToF based AF mode, the focused foreground object as part of the zoomed preview of the scene.
2. 2 . The method of claim 1 , further comprising: based on a second comparison of a second PDAF depth estimate and a second ToF depth estimate, determining that a second foreground object in a second zoomed preview of the scene is not in-focus for the ToF based AF mode; and based on the determination that the second foreground object in the second zoomed preview of the scene is not in-focus for the ToF based AF mode, maintaining the PDAF mode and not activating the ToF based AF mode, and wherein the displaying comprises displaying the second zoomed preview based on the PDAF mode.
3. 3 . The method of claim 1 , wherein the comparison of the PDAF depth estimate and the ToF depth estimate comprises: determining whether a delta depth estimate based on a difference between the PDAF depth estimate and the ToF depth estimate exceeds a depth threshold, and wherein the determination that the foreground object in the zoomed preview is in-focus for the ToF based AF mode is based on a determination that the delta depth estimate exceeds the depth threshold.
4. 4 . The method of claim 1 , wherein the receiving of the PDAF depth estimate further comprises: determining whether the PDAF depth estimate exceeds a PDAF confidence threshold, and wherein the bypassing of the PDAF mode is based on the determination whether the PDAF depth estimate exceeds the PDAF confidence threshold.
5. 5 . The method of claim 4 , further comprising: receiving a second PDAF depth estimate based on a second zoomed preview of the scene; determining that the second PDAF depth estimate does not exceed the PDAF confidence threshold; and maintaining the PDAF mode and not activating the ToF based AF mode, and wherein the displaying comprises displaying the second zoomed preview based on the PDAF mode.
6. 6 . The method of claim 1 , wherein the receiving of the ToF depth estimate further comprises: determining whether the ToF depth estimate exceeds a ToF confidence threshold, and wherein the focusing of the camera system in the ToF based AF mode is based on the determination whether the ToF depth estimate exceeds the ToF confidence threshold.
7. 7 . The method of claim 6 , further comprising: determining that the ToF depth estimate exceeds the ToF confidence threshold, and wherein the focusing of the camera system in the ToF based AF mode is based on a distance-to-position mapping for the foreground object based on the ToF depth estimate.
8. 8 . The method of claim 6 , further comprising: determining that the ToF depth estimate does not exceed the ToF confidence threshold, and wherein the focusing of the camera system in the ToF based AF mode is based on a multi-grid contrast detection autofocus (CDAF) search based on the ToF depth estimate.
9. 9 . The method of claim 8 , wherein the multi-grid CDAF search is based on one or more of a spatial grid, one or more directions, or one or more spatial frequencies.
10. 10 . The method of claim 1 , further comprising: receiving a second ToF depth estimate based on a second zoomed preview of the scene; determining that the second ToF depth estimate does not exceed a ToF confidence threshold; and maintaining the PDAF mode and not activating the ToF based AF mode, and wherein the displaying comprises displaying the second zoomed preview based on the PDAF mode.
11. 11 . The method of claim 1 , further comprising: determining that the PDAF depth estimate exceeds a PDAF confidence threshold; determining that the ToF depth estimate exceeds a ToF confidence threshold; determining, based on the zoomed preview of the scene, whether a brightness intensity of a background exceeds a brightness threshold, and wherein the bypassing of the PDAF mode is based on the determination whether the brightness intensity of the background exceeds the brightness threshold.
12. 12 . The method of claim 11 , further comprising: determining that the brightness intensity of the background exceeds the brightness threshold; and bypassing the PDAF mode and activating the ToF based AF mode.
13. 13 . The method of claim 11 , further comprising: determining that a second brightness intensity of a second background in a second zoomed preview does not exceed the brightness threshold; and maintaining the PDAF mode and not activating the ToF based AF mode, and wherein the displaying comprises displaying the second zoomed preview based on the PDAF mode.
14. 14 . The method of claim 1 , further comprising: receiving, by a user interface of the display screen, an indication to disable the ToF based AF mode; and responsive to the indication, maintaining the PDAF mode and not activating the ToF based AF mode, and wherein the displaying comprises displaying a second zoomed preview based on the PDAF mode.
15. 15 . The method of claim 1 , further comprising: displaying, by the display screen, an initial preview of the scene being captured by another camera system; detecting a zoom operation that causes a transition from the second camera system to the camera system; and providing, by a user interface of the display screen, a selectable virtual object to receive an indication whether to enable or disable the ToF based AF mode.
16. 16 . The method of claim 15 , wherein the camera system is configured to provide an ultrawide field of view (FOV), and wherein the other camera system is configured to provide a wide FOV.
17. 17 . The method of claim 1 , wherein the focusing of the camera system comprises adjusting at least one lens of the camera system.
18. 18 . The method of claim 1 , wherein the focusing of the camera system comprises determining an exposure time for the camera system based on a motion-blur tolerance of the camera system.
19. 19 . The method of claim 1 , wherein the camera system is a component of a mobile device.
20. 20 . A computing device, comprising: a display screen; a camera system configured to operate at a focal length less than a threshold focal length; one or more processors; and data storage, wherein the data storage has stored thereon computer-executable instructions that, when executed by the one or more processors, cause the mobile device to carry out functions comprising: displaying, by a display screen of the camera system, a zoomed preview of a scene captured by the camera system; determining a phase-detect autofocus (PDAF) depth estimate and a time-of-flight (ToF) depth estimate for the scene; based on a comparison of the PDAF depth estimate and the ToF depth estimate, determining whether a foreground object in the zoomed preview is in-focus for a ToF based AF mode of the camera system; based on a determination that the foreground object in the zoomed preview is in-focus for the ToF based AF mode, bypassing a PDAF mode and activating the ToF based AF mode to focus on the foreground object, wherein the PDAF mode comprises focusing of the camera system based on the PDAF depth estimate, and wherein the ToF based AF mode comprises focusing of the camera system based on the ToF depth estimate; and displaying, by the display screen and based on the ToF based AF mode, the focused foreground object as part of the zoomed preview of the scene.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS/INCORPORATION BY REFERENCE This application claims priority to U.S. Provisional Patent Application No. 63/378,652, filed on Oct. 6, 2022, which is hereby incorporated by reference in its entirety. BACKGROUND Many modern computing devices, including mobile phones, personal computers, and tablets, include image capture devices. Some image capture devices are configured with multi-camera systems. The camera systems are configured to use their respective specifications to collaboratively meet different image capturing requirements. A smart phone can integrate multiple types of cameras with a variety of focal lengths to take care of objects in different distances and scenes in different fields of view (FOVs). SUMMARY The present disclosure generally relates to a transition between multiple cameras. In one aspect, an image capture device may include multiple cameras. Transitioning from a normal mode to a macro mode may result in a perceptible lack of focus of target close-up objects. As described herein, a priority hybrid auto-focus strategy is described, thereby reducing the perceptible lack of focus subsequent to a camera switch (e.g., to an ultra-wide camera). In a first aspect, a computer-implemented method is provided. The method includes displaying, by a display screen of a camera system, a zoomed preview of a scene captured by the camera system. The method includes determining a phase-detect autofocus (PDAF) depth estimate and a time-of-flight (ToF) depth estimate for the scene. The method includes, based on a comparison of the PDAF depth estimate and the ToF depth estimate, determining whether a foreground object in the zoomed preview is in-focus for a ToF based autofocus (AF) mode of the camera system. The method includes, based on a determination that the foreground object in the zoomed preview is in-focus for the ToF based AF mode, bypassing a PDAF mode and activating the ToF based AF mode to focus on the foreground object, wherein the PDAF mode comprises focusing of the camera system based on the PDAF depth estimate, and wherein the ToF based AF mode comprises focusing of the camera system based on the ToF depth estimate. The method includes displaying, by the display screen and based on the ToF based AF mode, the focused foreground object as part of the zoomed preview of the scene. In a second aspect, a computing device is provided. The computing device includes a display screen, a camera system configured to operate at a focal length less than a threshold focal length, one or more processors, and data storage, wherein the data storage has stored thereon computer-executable instructions that, when executed by the one or more processors, cause the mobile device to carry out functions. The operations include displaying, by a display screen of the camera system, a zoomed preview of a scene captured by the camera system; receiving, based on the zoomed preview of the scene, a phase-detect autofocus (PDAF) depth estimate and a time-of-flight (ToF) depth estimate for the scene; based on a comparison of the PDAF depth estimate and the ToF depth estimate, determining whether a foreground object in the zoomed preview is in-focus for a ToF based AF mode of the camera system; based on a determination that the foreground object in the zoomed preview is in-focus for the ToF based AF mode, bypassing a PDAF mode and activating the ToF based AF mode to focus on the foreground object, wherein the PDAF mode comprises focusing of the camera system based on the PDAF depth estimate, and wherein the ToF based AF mode comprises focusing of the camera system based on the ToF depth estimate; and displaying, by the display screen and based on the ToF based AF mode, the focused foreground object as part of the zoomed preview of the scene. In a third aspect, an article of manufacture is provided. The article of manufacture may include a non-transitory computer-readable medium comprising program instructions executable by one or more processors to cause the one or more processors to perform operations. The operations include displaying, by a display screen of a camera system, a zoomed preview of a scene captured by the camera system; determining a phase-detect autofocus (PDAF) depth estimate and a time-of-flight (ToF) depth estimate for the scene; based on a comparison of the PDAF depth estimate and the ToF depth estimate, determining whether a foreground object in the zoomed preview is in-focus for a ToF based AF mode of the camera system; based on a determination that the foreground object in the zoomed preview is in-focus for the ToF based AF mode, bypassing a PDAF mode and activating the ToF based AF mode to focus on the foreground object, wherein the PDAF mode comprises focusing of the camera system based on the PDAF depth estimate, and wherein the ToF based AF mode comprises focusing of the camera system based on the ToF depth estimate; and displaying, by the display screen and based