CN-122023147-A - Low-cost imaging sensor noise modeling method and system

CN122023147ACN 122023147 ACN122023147 ACN 122023147ACN-122023147-A

Abstract

The invention discloses a low-cost imaging sensor noise modeling method and system. The method comprises the steps of collecting and processing calibration data, decomposing sensor noise into time-invariant noise, stripe noise and pixel-level noise components, modeling and calibrating the time-invariant noise, stripe noise and pixel-level noise components respectively, constructing a neural proxy network taking random noise, photosensitivity and exposure time as input conditions, training the neural proxy network by utilizing high-precision pixel-level noise samples, enabling the neural proxy network to generate synthetic noise adaptive to sensor characteristics and imaging parameters, fusing the synthetic noise generated by the network with real sampling noise by adopting a random mixing strategy, superposing the synthetic noise with other calibration noise components and signal-related noise, generating a complete synthetic noise field, and finally synthesizing the real noise-carrying data for training a denoising network with a clean image. The invention can accurately describe the complex noise characteristics of the low-cost sensor, and effectively considers the reality and diversity of noise.

Inventors

WANG LIZHI
Feng Hansen
HUANG YUANFEI
HUANG HUA

Assignees

北京师范大学

Dates

Publication Date: 20260512
Application Date: 20260203

Claims (8)

1. A method for modeling noise of a low-cost imaging sensor, comprising the steps of: s1, acquiring dark frame data and flat field frame data of a sensor under different imaging parameter combinations, wherein the dark frame data and the flat field frame data are used for noise calibration; S2, based on dark frame data and flat field frame data, decomposing sensor noise into independent components and performing parameterization calibration to obtain noise parameters, wherein the noise parameters comprise a time-invariant noise model, a stripe noise model and a high-precision pixel-level noise sample; s3, constructing and training a noise neural agent network taking random noise, sensitivity and exposure time as input conditions by taking a high-precision pixel-level noise sample as a supervision signal; s4, generating complete synthesized noise by utilizing the time invariant noise model, the stripe noise model, the trained noise neural agent network and a preset mixing strategy.
2. The low cost imaging sensor noise modeling method of claim 1, wherein S1 comprises: In a dark environment, imaging is carried out according to traversal combination of a preset sensitivity calibration point set and a preset exposure time calibration point set, and a plurality of original dark frame images under each imaging parameter combination are acquired; and under the condition of uniform illumination, imaging according to the set of photosensitive calibration points and a group of exposure time, and acquiring Zhang Yuanshi more flat field frame images.
3. The low cost imaging sensor noise modeling method of claim 1, wherein S2 comprises: calibrating a time-invariant noise model comprising black level errors and fixed pattern noise by a linear regression method according to dark frame data acquired under different imaging parameter combinations; calculating the average value of rows and columns from dark frame residual error data after removing time-invariant noise so as to calibrate a stripe noise model; further removing stripe noise from the residual data, and performing high-bit reconstruction on the obtained initial pixel level noise to obtain a spatially independent high-precision pixel level noise sample; and calibrating the overall system gain of the sensor by using a photon transfer method according to the acquired flat field frame data.
4. The low cost imaging sensor noise modeling method of claim 1, wherein S3 comprises: constructing a neural agent network with input and output corresponding to Bayer array four-channel tensors, wherein the input conditions of the neural agent network comprise random noise, sensitivity and exposure time of which the front end is sampled from standard normal distribution, and a gain adjusting module for determining a scaling coefficient by interpolation of the sensitivity and the exposure time is contained in the neural agent network; Parameters of the neural agent network are trained by minimizing a distribution loss function using high precision pixel level noise samples as supervisory signals.
5. The low cost imaging sensor noise modeling method of claim 1, wherein S4 comprises: Randomly sampling real noise from a high-precision pixel-level noise sample library, generating synthetic noise from a trained neural agent network, and fusing the real noise and the synthetic noise by using a random mixing coefficient to obtain mixed pixel-level noise; Synthesizing corresponding time-invariant noise and stripe noise by using the calibrated time-invariant noise model and the stripe noise model according to the target imaging parameters; And adding the mixed pixel level noise with the synthesized time-invariant noise and the stripe noise, and combining the calibrated system gain and the shot noise model to obtain the complete synthesized noise.
6. The method of claim 4, wherein the neural agent network is a four-channel joint modeling network, and the scaling factor of the internal gain adjustment module is determined by interpolation between sensitivity and exposure time, wherein the interpolation weight is associated with a preset imaging parameter calibration point.
7. The method of claim 5, wherein the blending of the real noise and the synthetic noise using random mixing coefficients is formulated as follows: wherein alpha is a random mixing coefficient between 0 and 1 for balancing the realism and diversity, Is synthesized pixel level noise.
8. A low cost imaging sensor noise modeling system for implementing the method of any of claims 1-7, comprising an acquisition module, a decomposition module, a construction module, and a production module; the acquisition module is used for acquiring dark frame data and flat field frame data of the sensor under different imaging parameter combinations and used for noise calibration; The decomposition module is used for decomposing sensor noise into independent components based on dark frame data and flat field frame data and carrying out parameterization calibration to obtain noise parameters, wherein the noise parameters comprise a time-invariant noise model, a stripe noise model and a high-precision pixel-level noise sample; the construction module is used for constructing and training a noise neural agent network taking random noise, photosensitivity and exposure time as input conditions by taking high-precision pixel-level noise samples as supervision signals; the production module is used for generating complete synthesized noise by utilizing the time-invariant noise model, the stripe noise model, the trained noise neural agent network and a preset mixing strategy.

Description

Low-cost imaging sensor noise modeling method and system Technical Field The invention relates to the field of sensor noise modeling, in particular to a low-cost imaging sensor noise modeling method and system. Background The low-light image denoising technology has important significance for improving the imaging quality of consumer electronic equipment. At present, a denoising method based on deep learning has become a mainstream scheme, and the method realizes noise reduction by learning a mapping relation between a noisy image and a clean image, and the effect depends on large-scale high-quality training data. However, acquiring real, clean paired data for a particular imaging sensor is costly and complex, and the resulting data often still has data defects that severely limit the final performance of the denoising model. In order to overcome the bottleneck of real data acquisition, the adoption of a noise modeling technology for synthesizing training data becomes an effective way. The key of the technology is to construct a model capable of accurately reflecting the real noise distribution of the sensor. The existing method mainly comprises two types, namely a type of noise synthesis based on a physical mechanism by calibrating noise parameters and according to a Poisson-Gaussian classical model, wherein the physical meaning is clear, but the capability of characterizing complex non-stationary noise is limited, and the other type of noise synthesis based on data driving, wherein the distribution is directly learned from a noise sample by utilizing a neural network, and the training process is generally dependent on sufficient and pure noise data and has challenges in stability and generalization. When the above general method is applied to a low-cost imaging sensor, the noise characteristics are more complex due to process and cost limitations, which makes the prior modeling technology pose serious challenges. While low cost sensors exhibit significant noise characteristic differences across different color channels of bayer arrays, existing modeling methods tend to ignore such channel heterogeneity, causing cross-channel noise residuals or color distortion problems to occur easily when a synthetic data training-based denoising network processes real images. In addition, the noise intensity of the sensor is not only changed along with the sensitivity, but also is obviously influenced by the exposure time, and a gain adjustment module in the traditional neural agent model usually only considers a single factor of the sensitivity, so that the dynamic dependency relationship is difficult to accurately describe, and the synthetic noise deviates from reality under the variable parameter condition. Meanwhile, the low-cost sensor is easy to generate local mode noise which is spatially dependent and distributed in the same way, and the existing method is difficult to achieve good balance between keeping the authenticity of noise and guaranteeing the diversity of data no matter the existing method simply samples limited real noise or completely relies on a network to generate random noise. In summary, the existing noise modeling method is difficult to adapt to the unique noise characteristics of the low-cost sensor, and cannot generate synthetic data which is highly realistic and has enough diversity, so that the performance and practical process of the advanced denoising technology on the low-cost sensor are restricted. Thus, a new approach to noise modeling specifically for low cost imaging sensor designs is urgently needed. Disclosure of Invention Aiming at the problems that the existing noise modeling method has poor channel characteristic adaptability, inaccurate noise parameter adjustment, difficulty in considering noise diversity and authenticity, insufficient non-stationary noise processing capacity and the like when the existing noise modeling method is used for adapting to a low-cost imaging sensor, the invention aims to provide the noise modeling method specially designed for the low-cost imaging sensor. The method accurately describes the statistical characteristics and the association relation of noise of each channel by establishing a Bayer array four-channel joint noise model so as to solve the problem of noise residue after denoising caused by inconsistent noise among channels, improves the estimation precision of noise parameters under the dynamic imaging condition by constructing a gain adjusting module which simultaneously associates sensitivity with exposure time, and further, effectively covers the complex noise scene with non-independent and same distribution by designing a strategy for adaptively mixing real sampling noise and noise generated by a neural network while preserving the inherent local noise mode of a sensor. The method has the advantages of strong generalization capability, wide adaptability of the sensor and high authenticity of synthesized noise, can be directly applied to