Search

CN-121486700-B - Quantum image sensing method and device based on exposure modulation

CN121486700BCN 121486700 BCN121486700 BCN 121486700BCN-121486700-B

Abstract

The invention discloses a quantum image sensing method and device based on exposure modulation. The sensing end carries out frame-by-frame exposure modulation on the same scene in the continuous K frames, regulates and controls the expected incident photon number of each frame, and the pixels output 1-bit binary images in each frame and obey poisson distribution. And the acquisition end carries out maximum posterior estimation on the K frames as a group to obtain the reference intensity estimation of each pixel. In order to improve the robustness and the speed, a RED-PRO iterative algorithm based on FISTA acceleration is adopted in the reconstruction process, namely, after the gradient of a data consistency term is reduced, the data consistency term is projected to a pre-training denoising manifold, and a pixel self-adaption step length is given based on Fisher information. The method can obviously improve the signal-to-noise ratio and inhibit the artifacts under the scene with low illumination, saturation and limited dynamic range, and can be compatible with the existing QIS/SPAD sensor and lens module only through exposure modulation and software reconstruction without changing a 1-bit sensing circuit architecture.

Inventors

  • FENG XIAOHUA
  • LI MINGHAN
  • WEI KAI

Assignees

  • 浙江大学

Dates

Publication Date
20260512
Application Date
20260112

Claims (6)

  1. 1. A quantum image sensing method based on exposure modulation, characterized in that the method comprises the following steps: (S1) for pixels in the same view field, applying a preset exposure modulation mode to each frame in continuous multi-frame exposure acquisition, and regulating and controlling the expected incident photon number of each frame; (S2) acquiring binary output image output of each frame, which is subjected to poisson distribution, through a quantum image sensor, establishing a likelihood function for the expected incident photon number of each pixel, and taking negative log likelihood as a fidelity term loss function of an optimization target; (S3) iteratively optimizing the reconstructed image by adopting maximum posterior estimation aiming at negative log likelihood; In the iterative reconstruction process, a RED-PRO iterative algorithm based on FISTA acceleration is used, the gradient descent process is replaced by a Nesterov acceleration gradient technology on the basis of the original RED-PRO, the convergence speed is increased, and after each gradient descent/acceleration gradient descent, the method is put forward by the RED-PRO Operator, utilizing the pre-trained denoising model to project the image/video to be reconstructed to the neural network denoising device On the defined high-dimensional flow pattern, the implicit regular and fast-convergence reconstruction effect is realized.
  2. 2. The method of claim 1, wherein the expected number of photons for the kth frame is Where N is proportional to the pixel gray scale, Is a known modulation function As a linear modulation function, i.e. Wherein The coefficient known for the kth frame is proportional to the exposure time T.
  3. 3. The method of claim 1, wherein the neural network denoising device Neural network de-noisers, pre-trained with natural image/video or target domain data, contain either image de-noiser DnCNN, DRUNet, restormer or video de-noiser FASTDVDNET, RVRT structures and act as a priori operators in RED-PRO.
  4. 4. The method of claim 1, wherein the overall video frame is divided into groups of K frames, each group being estimated independently.
  5. 5. A quantum image sensing device for implementing the method of any one of claims 1-4, comprising a modulation control unit for generating control of exposure time or incident intensity per frame, an optical lens module for acquiring an image based on a lens, a quantum image sensor array being a one-bit quantized readout structure with sub-electronic noise output, a readout and time control circuit, and a processor for maximum a posteriori estimation reconstruction process.
  6. 6. A computer readable storage medium, characterized in that a computer program is stored thereon, which program, when being executed by a processor, causes an apparatus to carry out the method of any of claims 1-4.

Description

Quantum image sensing method and device based on exposure modulation Technical Field The invention relates to the technical field of image sensors and image processing, in particular to a quantum image sensing method and device based on exposure modulation. Background The quantum image sensor (Quanta Image Sensor, QIS) is a novel single photon imaging sensing technology, and is characterized in that the exposure of each pixel is divided into a plurality of high-speed frames, and each frame only accumulates few photons, so that each frame output can be quantized into one bit of information. Specifically, in a single bit QIS, each frame reads one bit, outputting a "1" if at least one photon is detected by a pixel within the frame, and outputting a "0" otherwise. Since photon arrival has random poisson statistics, for an exposure frame with an average photon number of N, the probability that a pixel does not receive a photon isAt least one photon is received with a probability of 1-。 QIS is presented to address the performance limitations of conventional CMOS/CCD image sensors at very low illumination and high dynamic range. The conventional image sensor has difficulty in detecting a single photon due to readout noise limitation under weak light and is easily saturated under strong light, so that dynamic range is limited. In contrast, QIS consists of numerous ultra-small pixels (called "jots") with single photon detection capability and extremely high readout frame rates (> 100 khz). By digitally integrating and denoising these spatiotemporal cube data, a quality gray scale image can be restored even in extremely weak light with an average of less than 1 photon per pixel. Currently, image reconstruction for QIS relies mainly on statistical inference and regularization methods. On the one hand, since QIS readout complies with poisson-Bernoulli statistical rules, photon arrival rates of pixels can be inferred by maximum likelihood estimation, and on the other hand, effective prior regularization needs to be introduced in order to suppress noise and underqualification problems in reconstruction. Existing methods include simple frame accumulation averaging, thresholding, and more advanced bayesian estimation and iterative algorithms. However, the prior art has shortcomings in reconstruction quality and convergence speed. For example, direct accumulation cannot fully utilize nonlinear information of QIS, bayesian method is complex in calculation, and conventional iterative reconstruction is easy to converge slowly or sink into local extremum under high noise and weak signal conditions. Therefore, a new scheme combining intelligent exposure modulation and efficient reconstruction algorithm is needed to improve the image signal-to-noise ratio sacrificed by QIS for high frame rate and high dynamic range, and further promote the practical application of QIS. Disclosure of Invention In order to overcome the defects of the prior art, the embodiment of the invention provides a quantum image sensing method and a quantum image sensing device based on exposure modulation, which have similar coding effects by modulating exposure time under different frames on the premise of not increasing the complexity of sensor hardware, so that the underdetermined degree of a target optimization problem is reduced in the reconstruction process, the final imaging effect is improved, and the advantages of FISTA and RED-PRO (Regularization by Denoising via Fixed-Point project) optimization algorithms are fused, and the convergence speed in the reconstruction process is accelerated. The method flow of the invention comprises the following steps of a quantum image sensing method based on exposure modulation, which comprises the following steps: (S1) for pixels in the same view field, applying a preset exposure modulation mode to each frame in continuous multi-frame exposure acquisition, and regulating and controlling the expected incident photon number of each frame; (S2) acquiring binary output image output of each frame, which is subjected to poisson distribution, through a quantum image sensor, establishing a likelihood function for the expected incident photon number of each pixel, and taking negative log likelihood as a fidelity term loss function of an optimization target; (S3) iteratively optimizing the reconstructed image by adopting maximum posterior estimation aiming at the negative log likelihood. (S5) using FISTA-acceleration-based RED-PRO iterative algorithm in iterative reconstruction, each time gradient is reduced/gradient is accelerated, by the method proposed in RED-PRO (Regularization by Denoising via Fixed-Point project)Operator, utilizing the pre-trained denoising model to project the image/video to be reconstructed to the neural network denoising deviceOn the defined high-dimensional flow pattern, the implicit regular and fast-convergence reconstruction effect is realized. Further, the expected photon number for the kth fra