CN-116648717-B - Camera sensor changes in multi-sensor devices

Abstract

By controlled timing of image sensor switching according to a predefined image sensor configuration, interruption of the continuity of image frames output from the image capturing device due to switching from one image sensor of the device to another image sensor may be reduced or eliminated. The operation of the multi-sensor image device according to the predefined image sensor configuration may include appropriately selecting a source of image adjustment during the zoom level transition. The predefined image sensor configuration may define conversion parameters for a particular zoom range of the image capturing device.

Inventors

• LIU SHIZHONG
• REN JIANFENG
• LIU WEILIANG
• N. Yu Xi
• J.HUANG
• S. M. Panilajevan Katesh
• K. Kolatapili
• W.Zou

Assignees

• 高通股份有限公司

Dates

Publication Date

20260508

Application Date

20211122

Priority Date

20201223

Claims (20)

1. 1. A method for image processing, comprising: receiving a first request for a transition from a first zoom level to a second zoom level; receiving a second request for a change from a first sensor to a second sensor during image capture from a multi-sensor capture device, wherein the second request is received based on the first request; determining a current zoom level corresponding to the second request to change from the first sensor to the second sensor, and Upon receiving the second request to change from the first sensor to the second sensor, generating output image data from the multi-sensor capture device by: Adjusting an image from the first sensor based on the current zoom level being determined to be within a first defined range, wherein adjusting the image comprises geometrically deforming the image from the first sensor to align with a field of view associated with the second sensor, and Adjusting an image from the second sensor based on the current zoom level being determined to be within a second defined range, wherein the image from the second sensor is adjusted to align with a field of view associated with the first sensor.
2. 2. The method of claim 1, wherein the adjusting the image from the first sensor is based on a mixing weight parameter, and wherein the method further comprises changing to the second sensor after the mixing weight parameter reaches a threshold.
3. 3. The method of claim 1, wherein generating the output image data from the multi-sensor capture device by adjusting the image from the first sensor is performed when a margin on the first sensor at the second zoom level is above a threshold amount.
4. 4. The method of claim 1, wherein the second request to change from the first sensor to the second sensor is based on an illumination condition during the image capture.
5. 5. The method of claim 1, wherein the second request to change from the first sensor to the second sensor is based on movement detected during the image capture.
6. 6. The method of claim 1, wherein the second request to change from the first sensor to the second sensor is based on user input for tracking an object during the image capture.
7. 7. The method of claim 1, wherein the method further comprises: Receiving a request for a change from the second sensor to a third sensor during the image capture from the multi-sensor capture device; determining a second current zoom level corresponding to the received request to change from the second sensor to the third sensor; determining whether the second current zoom level is within a third defined range, and The output image data is generated from the multi-sensor capture device by adjusting an image from one of the second sensor or the third sensor based on whether the second current zoom level is determined to be within the third defined range.
8. 8. The method of claim 1, wherein the first sensor is configured to obtain an image through a first lens having a first optical axis, and wherein the second sensor is configured to obtain an image through a second lens having a second optical axis different from the first optical axis.
9. 9. The method of claim 8, wherein the first lens has a first magnification, and wherein the second lens has a second magnification different from the first magnification.
10. 10. The method of claim 1, wherein the second request to change from a first sensor to a second sensor corresponds to a change during a transition from a first zoom level to a second zoom level.
11. 11. The method of claim 1, wherein the second request is received while the multi-sensor capture device transitions from the current zoom level to the second zoom level.
12. 12. An apparatus for image processing, comprising: processor, and A memory coupled to the processor and storing instructions that, when executed by the processor, cause the device to perform operations comprising: receiving a first request for a transition from a first zoom level to a second zoom level; receiving a second request for a change from a first sensor to a second sensor during image capture from a multi-sensor capture device, wherein the second request is received based on the first request; determining a current zoom level corresponding to the second request to change from the first sensor to the second sensor, and Upon receiving the second request to change from the first sensor to the second sensor, generating output image data from the multi-sensor capture device by: Adjusting an image from the first sensor based on the current zoom level being determined to be within a first defined range, wherein adjusting the image comprises geometrically deforming the image from the first sensor to align with a field of view associated with the second sensor, and Adjusting an image from the second sensor based on the current zoom level being determined to be within a second defined range, wherein the image from the second sensor is adjusted to align with a field of view associated with the first sensor.
13. 13. The apparatus of claim 12, wherein the adjusting the image from the first sensor is based on a mixing weight parameter, and wherein execution of the instructions causes the apparatus to perform operations further comprising changing to the second sensor after the mixing weight parameter reaches a threshold.
14. 14. The device of claim 12, wherein generating the output image data from the multi-sensor capture device by adjusting the image from the first sensor is performed when a margin on the first sensor at the second zoom level is above a threshold amount.
15. 15. The device of claim 12, wherein the second request to change from the first sensor to the second sensor is based on an illumination condition during the image capture.
16. 16. The device of claim 12, wherein the second request to change from the first sensor to the second sensor is based on movement detected during the image capture.
17. 17. The device of claim 12, wherein the second request to change from the first sensor to the second sensor is based on user input for tracking an object during the image capture.
18. 18. The device of claim 12, wherein execution of the instructions causes the device to perform operations further comprising: Receiving a request for a change from the second sensor to a third sensor during the image capture from the multi-sensor capture device; Determining a second current zoom level corresponding to the received request to change from the second sensor to the third sensor; determining whether the second current zoom level is within a third defined range, and The output image data is generated from the multi-sensor capture device by adjusting an image from one of the second sensor or the third sensor based on whether the second current zoom level is determined to be within the third defined range.
19. 19. The device of claim 12, wherein the first sensor is configured to obtain an image through a first lens having a first optical axis, and wherein the second sensor is configured to obtain an image through a second lens having a second optical axis different from the first optical axis.
20. 20. The device of claim 19, wherein the first lens has a first magnification, and wherein the second lens has a second magnification different from the first magnification.

Description

Camera sensor changes in multi-sensor devices Cross Reference to Related Applications The present application claims the benefit of U.S. patent application Ser. No.17/133,221, entitled "CAMERA SENSOR CHANGES IN MULTI-SENSOR DEVICE", filed on even 23, 12, 2020, which is expressly incorporated herein by reference in its entirety. Technical Field Aspects of the present disclosure relate generally to image processing. Some features of the present disclosure may enable and provide improvements in the processing of outputs from a multi-sensor image capture device by an image signal processor. Background Image capture devices, i.e., devices capable of capturing one or more digital images (whether a still image photograph or an image sequence of video), may be incorporated into a variety of devices. For example, the image capturing device may include a stand alone digital camera or digital video camera, a camera-equipped wireless communication device handset, such as a mobile phone, cellular phone or satellite radio phone, personal Digital Assistant (PDA), panel or tablet, gaming device, computer device (e.g., webcam, video surveillance camera) or other device having digital imaging or video capabilities. Some image capture devices include multiple image sensors that capture image data through one or more lenses, which may be referred to as multi-sensor image capture devices. The plurality of image sensors may be configured with different shots to provide multiple fields of view of the scene and/or different zoom levels of the scene. Example lens types include wide angle lenses, ultra wide angle lenses, tele lenses, telescope lenses, periscopic zoom lenses, fisheye lenses, macro lenses, fixed focus lenses, or various combinations thereof. In one example, the dual camera configuration may include a wide angle lens and a tele lens. However, the use of multiple image sensors increases the complexity of image processing in the device, as users are often not interested in multiple images of a scene, but rather in capturing and displaying a single image. Multiple frames captured from multiple image sensors may be processed to generate a single image for a user. Furthermore, due to the different physical characteristics of the image sensors, the frames obtained from each image sensor may flow together in such a way that the transition from one image sensor to another is perceptible to the human eye, such as a scene shift that occurs in the resulting video or preview display. For example, zooming in or out on a device may involve switching from one image sensor to another, which may result in a significant change in the field of view upon sensor switching, which is noticeable to a user viewing a preview image on the display or viewing video recorded during zooming in or out. Such artifacts in the output of a multi-sensor image capture device are undesirable. The disadvantages noted herein are merely representative and are included to highlight problems that have been identified and attempted to be solved by the inventors for existing devices. Aspects of the devices described below may address some or all of the shortcomings as well as other shortcomings known in the art. Aspects of the improved apparatus described below may exhibit other advantages in addition to those described above, and may be used in applications other than those described above. Disclosure of Invention By properly selecting the source and/or combination thereof for image adjustment according to a predefined image sensor configuration, the interruption of the continuity of image frames output from the image capturing device due to switching from one image sensor of the device to another image sensor may be reduced or eliminated. The predefined image sensor configuration may define conversion parameters for a particular zoom range of the image capturing device. For example, the predefined image sensor configuration may define a first zoom range and a second zoom range for the image capture device. The first zoom range may specify a zoom range in which image sensor switching to the target sensor is performed by deforming the source sensor to align the image with the target sensor. Then, when the target sensor is better able to match the field of view of the source sensor, sensor switching occurs at a subsequent zoom level after a duration to reduce significant offset in the output of the multi-sensor image device. The second zoom range may specify a zoom range in which the image sensor is outside the desired range, wherein the image capturing device immediately switches from the source sensor to the target sensor. After the immediate switch, the output of the target sensor is adjusted to align with the output of the source sensor from the previous frame to reduce the apparent offset in the output of the multi-sensor image device. In both examples defined ranges, the image signal processor determines whether to adjust the output from the source sensor to