US-20260129298-A1 - Field of View Correction Techniques for Shutterless Camera Systems

Abstract

Example embodiments relate to field of view correction techniques for shutterless camera systems. A mobile device displaying an initial preview of a scene being captured by an image capturing device of the computing device may determine a zoom operation configured to cause the imaging capturing device to focus on a target. The imaging capturing device is configured to change focal length when performing the zoom operation. While the image capturing device performs the zoom operation, the computing device may then map focal lengths used by the imaging capturing device to a virtual focal length such that a field of view of the scene remains consistent across image frames displayed by the display screen between the initial preview of the scene and the zoomed preview of the scene that focuses on the target and display the zoomed preview of the scene that focuses on the target.

Inventors

• Hua Cheng
• Youyou Wang
• Chucai Yi
• Fuhao Shi
• Chia-Kai Liang

Assignees

• GOOGLE LLC

Dates

Publication Date

20260507

Application Date

20221004

Claims (20)

1. 1 . A computer-implemented method, comprising: displaying, by a display screen of a computing device, an initial preview of a scene being captured by an image capturing device of the computing device, wherein the image capturing device is operating at an initial focal length when capturing the initial preview of the scene; determining, by the computing device, a zoom operation configured to cause the imaging capturing device to focus on a target, wherein the imaging capturing device is configured to change focal length when performing the zoom operation; while the image capturing device performs the zoom operation, mapping focal lengths used by the imaging capturing device to a virtual focal length such that a field of view of the scene remains consistent across image frames displayed by the display screen between the initial preview of the scene and a zoomed preview of the scene that focuses on the target; and displaying, by the display screen, the zoomed preview of the scene that focuses on the target.
2. 2 . The method of claim 1 , further comprising: obtaining frame-based data for each image frame while the image capturing device performs the zoom operation; and determining geometric data for the image capturing device based on the frame-based data for each image frame.
3. 3 . The method of claim 2 , wherein frame-based data comprises voice coil motor (VCM) data.
4. 4 . The method of claim 3 , wherein mapping the focal lengths used by the image capturing device to the virtual focal length comprises: determining a real focal length used by the image capturing device for an image frame based on the VCM data corresponding to the image frame; and applying a warping transform that maps the real focal length determined for the image frame to the virtual focal length.
5. 5 . The method of claim 4 , wherein determining the real focal length comprises: determining a set of real focal lengths corresponding to scanlines in the image frame, wherein a real focal length for a scanline in the image frame is determined based on an average focal length of an exposure interval for the scanline.
6. 6 . The method of claim 5 , wherein applying the warping transform comprises: applying the warping transform to map each real focal length from the set of real focal lengths to the virtual focal length.
7. 7 . The method of claim 4 , wherein determining the real focal length comprises: determining a set of real focal lengths corresponding to scanlines in the image frame, wherein a real focal length for a scanline in the image frame is determined based on a given focal length at a middle of an exposure interval for the scanline.
8. 8 . The method of claim 7 , wherein applying the warping transform comprises: applying the warping transform to map each real focal length from the set of real focal lengths to the virtual focal length.
9. 9 . The method of claim 1 , wherein determining the zoom operation configured to cause the image capturing device to focus on the target comprises: causing the image capturing device to perform an auto focus (AF) technique.
10. 10 . The method of claim 1 , further comprising: obtaining frame-based data representing intrinsic parameters corresponding to the image capturing device, wherein frame-based data includes timestamps; and based on the frame-based data, interpolating a focal length representation per mesh row.
11. 11 . The method of claim 10 , wherein interpolating the focal length representation per mesh row comprises: determining the focal length representation based on an average focal length in an exposure interval.
12. 12 . The method of claim 10 , wherein interpolating the focal length representation per mesh row comprises: determining the focal length representation based on a middle focal length in an exposure interval.
13. 13 . The method of claim 10 , wherein mapping focal lengths used by the image capturing device to the virtual focal length comprises: generating a backward mesh warp based on the focal length representation per mesh row; and applying the backward mesh warp for a given image frame.
14. 14 . The method of claim 1 , further comprising: detecting the target in the scene based on one or more visual features in one or more image frames being captured by the image capturing device, wherein the one or more image frames are subsequent to the initial preview of the scene; and wherein determining the zoom operation comprises: determining the zoom operation responsive to detecting the target.
15. 15 . The method of claim 1 , further comprising: determining the virtual focal length based on the initial focal length.
16. 16 . The method of claim 1 , further comprising: obtaining a calibration model for the image capturing device; and determining the virtual focal length based on the calibration model for the image capturing device.
17. 17 . The method of claim 16 , wherein mapping focal lengths used by the image capturing device to the virtual focal length comprises: computing a scaling ratio between a given focal length for an image frame and the virtual focal length; and applying the scaling ratio.
18. 18 . A mobile device comprising: a display screen; an image capturing device; one or more processors; and data storage, wherein the data storage has stored thereon computer-executable instructions that, when executed by the one or more processors, cause the mobile device to carry out functions comprising: displaying, by the display screen, an initial preview of a scene being captured by the image capturing device, wherein the image capturing device is operating at an initial focal length when capturing the initial preview of the scene; determining a zoom operation configured to cause the imaging capturing device to focus on a target, wherein the imaging capturing device is configured to change focal length when performing the zoom operation; while the image capturing device performs the zoom operation, mapping focal lengths used by the imaging capturing device to a virtual focal length such that a field of view of the scene remains consistent across image frames displayed by the display screen between the initial preview of the scene and a zoomed preview of the scene that focuses on the target; and displaying, by the display screen, the zoomed preview of the scene that focuses on the target.
19. 19 . The mobile device of claim 18 , wherein the image capturing device is a shutterless camera system.
20. 20 . A non-transitory computer-readable medium comprising program instructions executable by one or more processors to cause the one or more processors to perform operations comprising: displaying, by a display screen, an initial preview of a scene being captured by an image capturing device, wherein the image capturing device is operating at an initial focal length when capturing the initial preview of the scene; determining a zoom operation configured to cause the imaging capturing device to focus on a target, wherein the imaging capturing device is configured to change focal length when performing the zoom operation; while the image capturing device performs the zoom operation, mapping focal lengths used by the imaging capturing device to a virtual focal length such that a field of view of the scene remains consistent across image frames displayed by the display screen between the initial preview of the scene and a zoomed preview of the scene that focuses on the target; and displaying, by the display screen, the zoomed preview of the scene that focuses on the target.

Description

BACKGROUND Many modern computing devices, such as mobile phones, personal computers, and tablets, include image capture devices (e.g., still and/or video cameras). The image capture devices can capture images that can depict a variety of scenes, including scenes that involve people, animals, landscapes, and/or objects. Some image capture devices are configured with telephoto capabilities. SUMMARY Example embodiments presented herein relate to field of view (FOV) correction techniques for shutterless camera systems. To reduce undesirable viewing artifacts that can arise during auto-focus sweeps, a mobile device or another type of computing device may use camera parameter interpolation to apply FOV correction techniques that keep the field of view consistent across image frames being displayed by the device. When the camera used by the mobile device is shutterless with rows (or columns) of image frames readout in sequence, the mobile device may analyze real focal length and optical center on a per-row basis (or per-column basis) when applying FOV correction techniques to accommodate the different exposure intervals associated with the sequence readout. Accordingly, in a first example embodiment, a computer-implemented method is provided. The method involves displaying, by a display screen of a computing device, an initial preview of a scene being captured by an image capturing device of the computing device, wherein the image capturing device is operating at an initial focal length when capturing the initial preview of the scene. The method also involves determining, by the computing device, a zoom operation configured to cause the imaging capturing device to focus on a target, wherein the imaging capturing device is configured to change focal length when performing the zoom operation. The method further involves, while the image capturing device performs the zoom operation, mapping focal lengths used by the imaging capturing device to a virtual focal length such that a field of view of the scene remains consistent across image frames displayed by the display screen between the initial preview of the scene and a zoomed preview of the scene that focuses on the target, and displaying, by the display screen of the computing device, the zoomed preview of the scene that focuses on the target. In a second example embodiment, a mobile device is provided. The mobile device includes a display screen, an image capturing device, one or more processors, and data storage. The data storage has stored thereon computer-executable instructions, that, when executed by the one or more processors, cause the mobile device to carry out operations. The operations involve displaying, by the display screen, an initial preview of a scene being captured by an image capturing device of the computing device, wherein the image capturing device is operating at an initial focal length when capturing the initial preview of the scene. The operations also involve determining a zoom operation configured to cause the image capturing device to focus on a target, wherein the image capturing device is configured to change focal length when performing the zoom operation. The operations further involve, while the image capturing device performs the zoom operation, mapping focal lengths used by the image capturing device to a virtual focal length such that a field of view of the scene remains consistent across image frames displayed by the display screen between the initial preview of the scene and a zoomed preview of the scene that focuses on the target. The operations also involve displaying, by the display screen, the zoomed preview of the scene that focuses on the target. In a third example embodiment, a non-transitory computer-readable medium comprising program instructions executable by one or more processors to cause the one or more processors to perform operations. The operations involve displaying, by the display screen, an initial preview of a scene being captured by an image capturing device of the computing device, wherein the image capturing device is operating at an initial focal length when capturing the initial preview of the scene. The operations also involve determining a zoom operation configured to cause the image capturing device to focus on a target, wherein the image capturing device is configured to change focal length when performing the zoom operation. The operations further involve, while the image capturing device performs the zoom operation, mapping focal lengths used by the image capturing device to a virtual focal length such that a field of view of the scene remains consistent across image frames displayed by the display screen between the initial preview of the scene and a zoomed preview of the scene that focuses on the target. The operations also involve displaying, by the display screen, the zoomed preview of the scene that focuses on the target. In a fourth example embodiment, a system may include various means for