US-12627896-B2 - Image processing for exposure bracketing

US12627896B2US 12627896 B2US12627896 B2US 12627896B2US-12627896-B2

Abstract

This disclosure provides systems, methods, and devices for image signal processing that support high dynamic range (HDR) image processing on image frames with temporally-aligned centers to reduce artifacts resulting from fusing image frames with different temporal centers. In some aspects, a method of image processing includes capturing three or more image frames having at least two different exposure lengths. The three or more image frames are processed to obtain two image frames with temporally-aligned centers, and those two image frames are processed in HDR fusion logic to obtain an output HDR image frame. Other aspects and features are also claimed and described.

Inventors

Meng-Lin Wu
Venkata Ravi Kiran Dayana
Sandesh Ghimire
Kai Liu
Ching-Fu Chen

Assignees

QUALCOMM INCORPORATED

Dates

Publication Date: 20260512
Application Date: 20230906

Claims (20)

1 . A method, comprising: receiving, by at least one processor, image data comprising first image data of a first exposure duration, second image data of a second exposure duration, and third image data of a third exposure duration, wherein: the first exposure duration is longer than the second exposure duration, and the third exposure duration is longer than the second exposure duration; determining, by the at least one processor, first summed image data by summing corresponding pixel intensity values of the first image data, the second image data, and the third image data; determining, by the at least one processor, second summed image data by summing corresponding pixel intensity values of the second image data and the third image data; and determining, by the at least one processor, a first output image frame by combining the first summed image data, the second summed image data, and the second image data.
2 . The method of claim 1 , wherein the first exposure duration is equal to the third exposure duration.
3 . The method of claim 1 , wherein a second temporal center of the second image data is equally spaced, in time, between a first temporal center of the first image data and a third temporal center of the third image data.
4 . The method of claim 1 , wherein the steps of determining the first summed image data, determining the second summed image data, and determining the first output image frame do not include spatial image alignment processing.
5 . The method of claim 1 , wherein the combining comprises executing a high dynamic range (HDR) fusion algorithm.
6 . The method of claim 1 , further comprising: configuring, by the at least one processor, an image sensor to capture the first image data at the first exposure duration, the second image data at the second exposure duration, and the third image data at the third exposure duration, wherein: the first exposure duration is longer than the second exposure duration, and the third exposure duration is longer than the second exposure duration.
7 . The method of claim 1 , further comprising: receiving, by the at least one processor, a fourth image data of a fourth exposure duration and a fifth image data of a fifth exposure duration, wherein: the fourth exposure duration is equal to the second exposure duration, the fifth exposure duration is equal to the third exposure duration, and a temporal center of the fourth image data is equally spaced in time between a temporal center of the third image data and a temporal center of the fifth image data; determining, by the at least one processor, a third summed image data by summing corresponding pixel intensity values of the third image data, the fourth image data, and the fifth image data; and determining, by the at least one processor, a second output image frame by combining the third summed image data and the fourth image data.
8 . The method of claim 1 , further comprising: receiving, by the at least one processor, a fourth image data, a fifth image data, and a sixth image data; determining, by the at least one processor, a third summed image data by summing corresponding pixel intensity values of the fourth image data, the fifth image data, and the sixth image data; determining, by the at least one processor, a second output image frame by combining the third summed image data with the fifth image data; and determining, by the at least one processor, a third output image frame by combining the first output image frame with the second output image frame.
9 . The method of claim 1 , further comprising: receiving, by the at least one processor, a fourth image data of a fourth exposure duration and a fifth image data of a fifth exposure duration, wherein: the fourth exposure duration is longer than the second exposure duration, the fifth exposure duration is longer than the second exposure duration, and a temporal center of the second image data is equally spaced in time between a first temporal center of the fourth image data and a second temporal center of the fifth image data; and determining, by the at least one processor, a third summed image data by summing the corresponding pixel intensity values of the first summed image data, the fourth image data, and the fifth image data.
10 . The method of claim 1 , wherein a temporal center of the second exposure duration is aligned with an activation of a flash.
11 . An apparatus, comprising: a memory storing processor-readable code; and at least one processor coupled to the memory, the at least one processor configured to execute the processor-readable code to cause the at least one processor to perform operations including: receiving, by the at least one processor, image data comprising first image data of a first exposure duration, second image data of a second exposure duration, and third image data of a third exposure duration, wherein: the first exposure duration is longer than the second exposure duration, and the third exposure duration is longer than the second exposure duration; determining, by the at least one processor, first summed image data by summing corresponding pixel intensity values of the first image data, the second image data, and the third image data; determining, by the at least one processor, second summed image data by summing corresponding pixel intensity values of the second image data and the third image data; and determining, by the at least one processor, a first output image frame by combining the first summed image data, the second summed image data, and the second image data.
12 . The apparatus of claim 11 , wherein the first exposure duration is equal to the third exposure duration.
13 . The apparatus of claim 11 , wherein a temporal center of the second image data is equally spaced, in time, between a first temporal center of the first image data and a second temporal center of the third image data.
14 . The apparatus of claim 11 , wherein the steps of determining the first summed image data, determining the second summed image data, and determining the first output image frame do not include spatial image alignment processing.
15 . The apparatus of claim 11 , wherein the combining comprises executing a high dynamic range (HDR) fusion algorithm.
16 . The apparatus of claim 11 , wherein the at least one processor is further configured to perform operations comprising: configuring, by the at least one processor, an image sensor to capture the first image data at the first exposure duration, the second image data at the second exposure duration, and the third image data at the third exposure duration, wherein: the first exposure duration is longer than the second exposure duration, and the third exposure duration is longer than the second exposure duration.
17 . The apparatus of claim 11 , wherein the at least one processor is further configured to perform operations comprising: receiving, by the at least one processor, a fourth image data of a fourth exposure duration and a fifth image data of a fifth exposure duration, wherein: the fourth exposure duration is equal to the second exposure duration, the fifth exposure duration is equal to the third exposure duration, and a temporal center of the fourth image data is equally spaced in time between a first temporal center of the third image data and a temporal second center of the fifth image data; determining, by the at least one processor, a third summed image data by summing corresponding pixel intensity values of the third image data, the fourth image data, and the fifth image data; and determining, by the at least one processor, a second output image frame by combining the third summed image data and the fourth image data.
18 . The apparatus of claim 11 , wherein the at least one processor is further configured to perform operations comprising: receiving, by the at least one processor, a fourth image data, a fifth image data, and a sixth image data; determining, by the at least one processor, a third summed image data by summing corresponding pixel intensity values of the fourth image data, the fifth image data, and the sixth image data; determining, by the at least one processor, a second output image frame by combining the third summed image data with the fifth image data; and determining, by the at least one processor, a third output image frame by combining the first output image frame with the second output image frame.
19 . The apparatus of claim 11 , wherein the at least one processor is further configured to perform operations comprising: receiving, by the at least one processor, a fourth image data of a fourth exposure duration and a fifth image data of a fifth exposure duration, wherein: the fourth exposure duration is longer than the second exposure duration, the fifth exposure duration is longer than the second exposure duration, and a temporal center of the second image data is equally spaced in time between a first temporal center of the fourth image data and a second temporal center of the fifth image data; and determining, by the at least one processor, a third summed image data by summing the corresponding pixel intensity values of the first summed image data, the fourth image data, and the fifth image data.
20 . The apparatus of claim 11 , wherein a temporal center of the second exposure duration is aligned with an activation of a flash.

Description

TECHNICAL FIELD Aspects of the present disclosure relate generally to image processing, and more particularly, to image processing of exposure-bracketed image frames. Some features may enable and provide improved image processing in low-light multi-frame imaging, high-motion photography, or other applications of exposure bracketing, including improved dynamic range in images while reducing motion blur occurring from motion during image capture. INTRODUCTION Image capture devices are devices that can capture one or more digital images, whether still images for photos or sequences of images for videos. Capture devices can be incorporated into a wide variety of devices. By way of example, image capture devices may comprise stand-alone digital cameras or digital video camcorders, camera-equipped wireless communication device handsets, such as mobile telephones, cellular or satellite radio telephones, personal digital assistants (PDAs), panels or tablets, gaming devices, computing devices such as webcams, video surveillance cameras, or other devices with digital imaging or video capabilities. Dynamic range may be important to image quality when capturing a representation of a scene with a wide color gamut using an image capture device. Conventional image sensors have a limited dynamic range, which may be smaller than the dynamic range of human eyes. Dynamic range may refer to the light range between bright portions of an image and dark portions of an image. A conventional image sensor may increase an exposure time to improve detail in dark portions of an image at the expense of saturating bright portions of an image. Alternatively, a conventional image sensor may decrease an exposure time to improve detail in bright portions of an image at the expense of losing detail in dark portions of the image. BRIEF SUMMARY OF SOME EXAMPLES The following summarizes some aspects of the present disclosure to provide a basic understanding of the discussed technology. This summary is not an extensive overview of all contemplated features of the disclosure and is intended neither to identify key or critical elements of all aspects of the disclosure nor to delineate the scope of any or all aspects of the disclosure. Its sole purpose is to present some concepts of one or more aspects of the disclosure in summary form as a prelude to the more detailed description that is presented later. High dynamic range (HDR) photography improves photography using these conventional image sensors by combining multiple recorded representations of a scene (e.g., image frames) from the image sensor. HDR photography may combine image frames of the same scene captured with different exposure lengths. The different exposure lengths capture different details of the scene. A long exposure image frame captures details in low-light portions of a scene. A short exposure image frame captures details in high-light portions of a scene. Combining a long exposure image frame and a short exposure image frame conventionally involves aligning the image frames before fusing the image frames to obtain a single output image frame having higher dynamic range than either the long or short image frames individually. The alignment allows the fusion operation to fuse corresponding portions of the image frames. Alignment reduces artefacts resulting from the fact that the different image frames are captured at different times, during which the image capture device may have moved or an object in the scene may have moved. For example, global motion may cause motion blur in the long exposure frame, which reduces alignment accuracy. As another example, local motion may result in non-uniform image warping. The longer the duration of the long exposure frame capture the more difficult alignment becomes because the longer exposure results in a further temporal distance between the long image frame and the short image frame. In some aspects of this disclosure, HDR image capture is performed by processing temporally-aligned image frames. Arranging the capture of image data to temporally align the image frames reduces or eliminates alignment performed on image frames when fusing the image frames to determine an output HDR image frame. The temporal alignment of image frames results in obtaining spatially aligned image frames. The temporally-aligned image frames may be obtained in a manner that assumes approximately linear motion within an exposure time, such that the temporal midpoint of exposure time coincides with the spatial midpoint of any scene motion. The temporally-aligned image frames may also be obtained in a manner that assumes exposure is a linear operation, such that all pixel values are assumed to be in a linear domain and pixel values are proportional to exposure time. Based on the assumptions, the capture of image frames using one of many different HDR techniques may be temporally arranged such that the resulting image frames are spatially aligned and may be processed wi