CN-119693232-B - Reflective Fourier lamination method based on camera matrix

Abstract

The invention relates to the field of image super-resolution, in particular to an image reconstruction method, and specifically relates to a Fourier laminated image reconstruction method based on a deep learning network. A series of low-resolution images are shot through a camera array by utilizing a reflection type far-field Fourier stack imaging system, deep learning is combined with a traditional Fourier stack, the low-resolution images are reconstructed by utilizing a neural network, the acquisition quantity of the low-resolution images is reduced, the requirement of the traditional Fourier stack on redundancy is entrust, and the sampling speed and the reconstruction speed of the Fourier stack imaging are improved.

Inventors

• FENG YUNPENG
• HAN YUHANG
• CHENG HAOBO
• ZHANG XIAOWEI

Assignees

• 北京理工大学

Dates

Publication Date

20260512

Application Date

20241202

Claims (5)

1. 1. A reflective fourier ptychographic method based on a camera matrix, characterized by the steps of: Step one, creating an object image by using a liquid crystal spatial light modulator; Acquiring a plurality of low-resolution images with known sizes by using an array camera matrix system consisting of 4X 4 full-shutter cameras, wherein the offset between adjacent camera lenses is 33mm; calculating the system magnification by using the image size and the camera target surface size in the second step; Step four, constructing a super-resolution image reconstruction deep learning network, wherein the network consists of an initial convolution layer, 14 dense blocks, 7 transition layers, 7 up-sampling layers and a final convolution layer, wherein the dense blocks and the transition layers are referenced DenseNet, each dense block consists of a normalization layer, a convolution layer and an activation function module; step five, utilizing the image collected in the step two and the corresponding true value to manufacture a training data set and train a network; step six, evaluating and optimizing the network by utilizing the MSE, the SSIM and the BCE image quality evaluation index; step seven, inputting the low-resolution image into the network after training and optimizing in the step five to obtain a high-resolution reconstructed image; The specific method for acquiring the low-resolution image by using the camera array comprises the following steps of using laser illumination, obtaining spherical light illumination after collimation and beam expansion, reflecting light rays by a spatial light modulator, adjusting the position of the camera array to ensure that the center of the array is converged with the center of an equivalent camera aperture, and shooting the low-resolution image of a group of samples by using the camera array; the camera matrix system is illuminated with a laser equivalent coherent light source and captured by a camera array at a transfer function In the case of (a), each camera is roughly emulated as a low-pass filter over the target field, where Pupil function, F is focal length, in the same camera array, in position The transfer function of the first camera with the center is The coherent impulse response corresponding to the camera is: , Wherein the method comprises the steps of Is positioned at A camera point spread function at the center of the plane defining a phase function as: , And defining an array camera measurement model as: .
2. 2. The method for obtaining images by using a spatial light modulator according to claim 1, wherein the specific method for obtaining images by using the spatial light modulator is that a plurality of vector clippers are made, and a plurality of binary images are generated by image enhancement methods such as rotation, inversion, scaling and the like.
3. 3. The method of claim 1, wherein the step four is to perform preprocessing such as downsampling on the acquired low-resolution image to create a training dataset and a training sub-dataset.
4. 4. The method of claim 1, wherein L=L is a loss of the image reconstruction network structure in the third step BCE + βL SSIM.
5. 5. The method of claim 1, wherein the camera array captures 16 low resolution images at a time.

Description

Reflective Fourier lamination method based on camera matrix Technical Field The invention relates to the field of image super-resolution, in particular to an image reconstruction method, and specifically relates to a Fourier laminated image reconstruction method based on a deep learning network. Background The stacked imaging is a computational optical imaging method proposed in the last century and used for solving the phase problem by deconvolution, and at the beginning of the 21 st century Rodenbur proposes a coherent diffraction stacked imaging technique (PIE) which uses the stacked relationship of objects in the spatial domain to constrain and realize the reconstruction of a large range of objects. In 2013, zheng et al proposed a fourier stacked microscopy (Fourier ptychographic microscopy, FPM) technique, which, unlike PIE, has a constraint on the light field in the frequency domain of the object, thus increasing the object spectrum and equivalently improving the resolution of the object. The Fourier laminated microscopic imaging technology combines a phase recovery algorithm and an aperture synthesis technology, and the technology restricts a light field in a frequency domain of an object, so that the frequency spectrum of an image can be increased, and the resolution of the image can be equivalently improved. Whereas the idea of fourier lamination has been increasingly applied in far field imaging in recent years. In conventional stacked diffraction imaging, an object is illuminated with a spatially limited light pattern (probe) and the diffraction pattern generated in the far field is observed, whereas in the far field the fourier transform of the probe is convolved with the fourier transform of the object. The object is translated to several positions and the diffraction pattern corresponding to each position is captured and then combined using an iterative algorithm. However, the current methods all require repeated sampling, the dark field exposure time is long, and the requirement of redundant information results in long data acquisition time, and in many scenes, faster acquisition speed and frequency are required. If the image acquisition time can be reduced and the number of acquired images can be reduced, the overall imaging speed can be further improved and the quality can be higher. In recent years, with rapid development of the deep learning field, convolutional neural networks (Convolutional Neural Network, CNN) have been increasingly applied in the fields of computer vision and image processing, such as image super-resolution, image denoising, phase reconstruction, object detection, and the like. The convolutional neural network learns the difference between the input image and the output image through a large amount of training data, and then searches for the correct nonlinear mathematical mapping relation between the input image and the output image. Therefore, deep learning can be combined with traditional Fourier lamination, a neural network is utilized to reconstruct a low-resolution image, the acquisition quantity of the low-resolution image is reduced, and the sampling speed and the reconstruction speed of Fourier lamination imaging are improved. Disclosure of Invention The invention aims to solve the defects in the prior art and provides a Fourier laminated image reconstruction method based on a deep learning network, which can effectively improve sampling and system imaging efficiency. The technical scheme provided by the invention is as follows: The invention provides a super-resolution imaging sampling system, which comprises a far-field reflection type Fourier laminated imaging light path and a camera matrix, wherein the far-field reflection type Fourier laminated imaging light path and the camera matrix are used for acquiring a plurality of low-resolution image intensity diagrams with known sizes; In a second aspect, the invention also provides a training method of the image reconstruction system, an image reconstruction method and a system The technical scheme of the super-resolution imaging sampling system provided by the invention is that a Fourier laminated image reconstruction method based on a deep learning network comprises the following steps: Step one, creating an object image by using a liquid crystal spatial light modulator; Acquiring a plurality of low-resolution images with known sizes by using an array camera matrix system consisting of 4X 4 full-shutter cameras, wherein the offset between adjacent camera lenses is 33mm; Calculating the system magnification M by using the image size and the camera target surface size in the second step; Step four, constructing a super-resolution image reconstruction deep learning network, wherein the network consists of an initial convolution layer, 14 dense blocks, 7 transition layers, 7 up-sampling layers and a final convolution layer, wherein the dense blocks and the transition layers are referenced DenseNet, each dense blo