US-20260129304-A1 - IMAGING DEVICE AND CONTROL METHOD THEREOF

US20260129304A1US 20260129304 A1US20260129304 A1US 20260129304A1US-20260129304-A1

Abstract

An imaging device and a control method thereof are provided. The control method is configured to control a first-frame timing of a first imaging sensor. The method includes: obtaining an estimated value of an operation latency time, and obtaining a candidate target timing; determining, based on a current timing, the estimated value of the operation latency time, and the candidate target timing, an additional delay time; executing an additional delay operation with a duration equal to the additional delay time; and triggering a first-frame acquisition operation of the first imaging sensor. The operation latency time is a total duration from triggering the first-frame acquisition operation to the first imaging sensor starting an exposure phase. The additional delay time is configured to enable a timing of a preset synchronization event of a first frame of the first imaging sensor to synchronize with the candidate target timing.

Inventors

Chi-cheng Ju
Yan-Shao LIU
Yu-Chang Huang
Kai Wei
Sheng-Jie LI

Assignees

MEDIATEK INC.

Dates

Publication Date: 20260507
Application Date: 20251031

Claims (20)

1 . A method for controlling a first-frame timing of a first imaging sensor in an imaging device, comprising: obtaining an estimated value of an operation latency time, and obtaining a candidate target timing; determining, based on a current timing, the estimated value of the operation latency time, and the candidate target timing, an additional delay time; executing an additional delay operation with a duration equal to the additional delay time; and triggering a first-frame acquisition operation of the first imaging sensor; wherein the operation latency time is a total duration from triggering the first-frame acquisition operation of the first imaging sensor to the first imaging sensor starting an exposure phase of the first-frame acquisition operation; the additional delay time is configured to enable a timing of a preset synchronization event of a first frame of the first imaging sensor to synchronize with the candidate target timing.
2 . The method as claimed in claim 1 , wherein the determining, based on the current timing, the estimated value of the operation latency time, and the candidate target timing, the additional delay time comprises: calculating the additional delay time as a result obtained by subtracting a sum of the current timing, the estimated value of the operation latency time, and a first time interval of the first imaging sensor from the candidate target timing; wherein the first time interval is a duration from when the first imaging sensor starts the exposure phase to the preset synchronization event in the first frame.
3 . The method as claimed in claim 2 , wherein before obtaining the estimated value of the operation latency time, and obtaining the candidate target timing, the method further comprises: performing, in response to an instruction to launch the first imaging sensor, a preparation operation for the launch of the first imaging sensor; the triggering the first-frame acquisition operation of the first imaging sensor comprises: launching the first imaging sensor.
4 . The method as claimed in claim 3 , wherein the imaging device further comprises a processor electrically connected to the first imaging sensor; the performing the preparation operation for the launch of the first imaging sensor comprises at least one selected from the group consisting of: determining, by the processor, configuration parameters for the first imaging sensor; performing a power-on procedure for the first imaging sensor; and/or transiting the first imaging sensor from a standby mode to an active mode; and/or the launch the first imaging sensor comprises: writing, by the processor, a stream-on instruction to the first imaging sensor; and the first imaging sensor performing an internal processing before the exposure phase; wherein the operation latency time is an operation time for launching the first imaging sensor.
5 . The method as claimed in claim 3 , wherein the processor is an image signal processor or an application processor; and/or the writing, by the processor, the stream-on instruction to the first imaging sensor comprises: the processor writing, based on an I2C protocol, the stream-on instruction to the first imaging sensor.
6 . The method as claimed in claim 3 , wherein the obtaining the estimated value of the operation latency time comprises: obtaining, through an integrator, the estimated value of the operation latency time; wherein the integrator is configured to collect actual values of the operation latency time of a plurality of first frames of the first imaging sensor, and to perform statistical processing on the actual values to generate the estimated value; each integrator corresponds to only one operation mode of one first imaging sensor.
7 . The method as claimed in claim 3 , wherein the imaging device further comprises a second imaging sensor; the obtaining the candidate target timing comprises: obtaining an estimated value of a timing of a preset synchronization event of a future frame of the second imaging sensor; wherein, in a case where both the future frame of the second imaging sensor and the first frame of the first imaging sensor are in a single-exposure operation mode, the preset synchronization event of a frame is start of exposure data readout of the frame; and/or, in a case where both the future frame of the second imaging sensor and the first frame of the first imaging sensor are in a multiple-exposure operation mode, the preset synchronization event of a frame is start of exposure data readout of the last exposure of the frame.
8 . The method as claimed in claim 7 , wherein the obtaining the estimated value of the timing of the preset synchronization event of the future frame of the second imaging sensor comprises: obtaining a plurality of estimated timings corresponding to the preset synchronization events of a plurality of consecutive future frames of the second imaging sensor; the calculating the additional delay time as a result obtained by subtracting a sum of the current timing, the estimated value of the operation latency time, and the first time interval of the first imaging sensor from the candidate target timing comprises: selecting one from the plurality of estimated timings in an ascending order, subtracting a sum of the current timing, the estimated value of the operation latency time, and the first time interval of the first imaging sensor, and taking a first positive result obtained as the additional delay time.
9 . The method as claimed in claim 7 , wherein in a case where a preparation operation for launching the second imaging sensor is performed during performing the preparation operation for the launching of the first imaging sensor, after performing, in response to the instruction to launch the first imaging sensor, the preparation operation for the launch of the first imaging sensor, the method further comprises: waiting for a completion of the preparation operation for launching the second imaging sensor.
10 . The method as claimed in claim 3 , wherein the imaging device further comprises a second imaging sensor; in a case where the second imaging sensor is to be streamed off or is being streamed off during the preparation operation for the launch of the first imaging sensor, the candidate target timing is: after the second imaging sensor has been streamed off, the processor successfully releases associated resources allocated for the second imaging sensor.
11 . The method as claimed in claim 2 , wherein the triggering the first-frame acquisition operation of the first imaging sensor comprises: performing a fast mode transit operation of the first imaging sensor from a current mode to a target mode.
12 . The method as claimed in claim 11 , wherein before executing the additional delay operation with the duration equal to the additional delay time, the method further comprises: in response to an instruction to perform the fast mode transit for the first imaging sensor, extending, based on an originally configured exposure duration, an exposure duration of the current frame of the first imaging sensor, and performing an exposure data readout process of the current frame.
13 . The method as claimed in claim 11 , wherein the imaging device further comprises a processor electrically connected to the first imaging sensor; the performing the fast mode transit operation of the first imaging sensor from the current mode to the target mode comprises: writing, by the processor to the first imaging sensor, a fast mode transit instruction to transit to the target mode; and performing, by the first imaging sensor, an internal processing before an exposure phase of the first frame in the target mode state.
14 . The method as claimed in claim 13 , wherein the processor is an image signal processor or an application processor; and/or the writing, by the processor to the first imaging sensor, the fast mode transit instruction to transit to the target mode comprises: writing, by the processor to the first imaging sensor via an I2C protocol, the fast mode transit instruction to transit to the target mode.
15 . The method as claimed in claim 11 , wherein the obtaining the estimated value of the operation latency time comprises: obtaining the estimated value of the operation latency time through an integrator; wherein the integrator is configured to collect an actual value of the operation latency time for transiting from the current mode to the target mode when the first imaging sensor performs the additional delay operation, and to perform statistical processing on the actual value to generate the estimated value; each integrator corresponds to only one operation mode transiting scenario of one first imaging sensor.
16 . The method as claimed in claim 11 , wherein the imaging device further comprises a second imaging sensor; the obtaining the candidate target timing comprises: obtaining an estimated value of a timing of a preset synchronization event of a future frame of the second imaging sensor; wherein, in a case where both the future frame of the second imaging sensor and the first frame of the first imaging sensor in the target mode are in a single-exposure operation mode, the preset synchronization event of a frame is a start of exposure data readout of the frame; and/or, in a case where both the future frame of the second imaging sensor and the first frame of the first imaging sensor in the target mode are in a multiple-exposure operation mode, the preset synchronization event of a frame is a start of exposure data readout of the last exposure of the frame.
17 . The method as claimed in claim 1 , wherein a first frame signal output by the first imaging sensor through a Mobile Industry Processor Interface is aligned with the candidate target timing; and/or a length distribution of the additional delay times for multiple first frames of the first imaging sensor is non-uniform.
18 . An imaging device, comprising a first imaging sensor and a processor electrically connected to each other, wherein the processor is configured to execute a first-frame timing control method for the first imaging sensor, the control method comprises: obtaining an estimated value of an operation latency time, and obtaining a candidate target timing; determining, based on a current timing, the estimated value of the operation latency time, and the candidate target timing, an additional delay time; executing an additional delay operation with a duration equal to the additional delay time; and triggering a first-frame acquisition operation of the first imaging sensor; wherein the operation latency time is a total duration from triggering the first-frame acquisition operation of the first imaging sensor to the first imaging sensor starting an exposure phase of the first-frame acquisition operation; the additional delay time is configured to enable a timing of a preset synchronization event of a first frame of the first imaging sensor to synchronize with the candidate target timing.
19 . The imaging device as claimed in claim 18 , wherein the determining, based on the current timing, the estimated value of the operation latency time, and the candidate target timing, the additional delay time comprises: calculating the additional delay time as a result obtained by subtracting a sum of the current timing, the estimated value of the operation latency time, and a first time interval of the first imaging sensor from the candidate target timing; wherein the first time interval is a duration from when the first imaging sensor starts the exposure phase to the preset synchronization event in the first frame.
20 . The imaging device as claimed in claim 19 , wherein the triggering the first-frame acquisition operation of the first imaging sensor comprises: launching the first imaging sensor; or performing a fast mode transit operation of the first imaging sensor from a current mode to a target mode.

Description

CROSS-REFERENCE TO RELATED APPLICATION This application is a continuation application of U.S. Provisional Patent Application No. 63/716,276 filed on Nov. 5, 2024, which is incorporated herein by reference in its entirety. TECHNICAL FIELD The present disclosure relates to the technical field of imaging technologies, in particularly relates to an imaging device and a control method thereof. BACKGROUND After an imaging sensor is streamed on or switches its operation mode, it often needs to synchronize image frames with specific events, such as image frames from other imaging sensors, time windows of external events, etc. The first frame generated by the imaging sensor after streamed on or switching the operation mode is referred to as the “first frame”. However, before an imaging sensor starts readout of the exposure data, it must go through multiple stages including software configuration, hardware communication, internal sensor processing, and exposure. The involved software and hardware factors are complex, and there is a lack of a unified timing control mechanism. As a result, frame synchronization solutions may usually have to wait for several frames after the first frame to achieve the frame synchronization. Additionally, imaging sensors from different manufacturers or with different configurations may require separate development of timing synchronization programs, which further leads to issues such as poor versatility and high development costs of existing solutions. SUMMARY According to a first aspect of the present disclosure, a method for controlling a first-frame timing of a first imaging sensor in an imaging device may be provided. The method may include: obtaining an estimated value of an operation latency time, and obtaining a candidate target timing; determining, based on a current timing, the estimated value of the operation latency time, and the candidate target timing, an additional delay time; executing an additional delay operation with a duration equal to the additional delay time; and triggering a first-frame acquisition operation of the first imaging sensor. The operation latency time may be a total duration from triggering the first-frame acquisition operation of the first imaging sensor to the first imaging sensor starting an exposure phase of the first-frame acquisition operation. The additional delay time may be configured to enable a timing of a preset synchronization event of a first frame of the first imaging sensor to synchronize with the candidate target timing. According to a second aspect of the present disclosure, an imaging device may be provided. The imaging device may include a first imaging sensor and a processor that are electrically connected to each other. The processor may be configured to perform a first-frame timing control method for the first image sensor. The method may include: obtaining an estimated value of an operation latency time, and obtaining a candidate target timing; determining, based on a current timing, the estimated value of the operation latency time, and the candidate target timing, an additional delay time; executing an additional delay operation with a duration equal to the additional delay time; and triggering a first-frame acquisition operation of the first imaging sensor. The operation latency time may be a total duration from triggering the first-frame acquisition operation of the first imaging sensor to the first imaging sensor starting an exposure phase of the first-frame acquisition operation. The additional delay time may be configured to enable a timing of a preset synchronization event of a first frame of the first imaging sensor to synchronize with the candidate target timing. BRIEF DESCRIPTION OF THE DRAWINGS In order to more clearly illustrate technical solutions in the present disclosure, the drawings required in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present disclosure. For those of ordinary skills in the art, other drawings could be obtained based on these drawings without creative efforts and should fall within the scope of the present disclosure. Among the drawings: FIG. 1 is a schematic structural view of an imaging device according to some embodiments of the present disclosure. FIG. 2 is a schematic flowchart of an imaging sensor from power-on to stream on. FIG. 3 is a timing diagram of an imaging sensor stream on according to the related art. FIG. 4 is a schematic flowchart of a fast mode transition of an imaging sensor according to some embodiments of the present disclosure. FIG. 5 is a schematic flowchart of a first-frame timing control method for a first imaging sensor of an imaging device according to some embodiments of the present disclosure. FIG. 6 is a schematic flowchart of a first-frame timing control method for a first imaging sensor of an imaging device according to some other embodiments of the pr