CN-121982123-A - Video-level lens-free spectrum imaging system based on color mask and method thereof

CN121982123ACN 121982123 ACN121982123 ACN 121982123ACN-121982123-A

Abstract

The invention discloses a video-grade lens-free spectrum imaging system based on a color mask and a method thereof. The system comprises a filtering window module, a dynamic adjustable spatial spectrum modulation module, a color acquisition module, a sliding window module and a fusion reconstruction module, wherein the filtering window module, the dynamic adjustable spatial spectrum modulation module and the color acquisition module are sequentially arranged in sequence, the fusion reconstruction module is respectively connected with the dynamic adjustable spatial spectrum modulation module, the color acquisition module and the sliding window module, and the sliding window module is connected with the color acquisition module. The invention effectively improves the space and spectrum quality of the reconstructed image through the fusion of the dynamic coding and the multi-frame information, realizes the continuous capturing of the hyperspectral video of the dynamic scene, and has the remarkable advantages of compact structure, flexible regulation and control and high imaging quality.

Inventors

CHEN LINSEN
WU YIKAI
WANG CHAO
SUN FEIYANG
HU LIHAO
CAO XUN

Assignees

南京大学

Dates

Publication Date: 20260505
Application Date: 20251216

Claims (10)

1. The video-level lens-free spectrum imaging system based on the color mask is characterized by comprising a filtering window module, a dynamic adjustable spatial spectrum modulation module, a color acquisition module, a sliding window module and a fusion reconstruction module, wherein the filtering window module, the dynamic adjustable spatial spectrum modulation module and the color acquisition module are sequentially arranged in sequence, the fusion reconstruction module is respectively connected with the dynamic adjustable spatial spectrum modulation module, the color acquisition module and the sliding window module, and the sliding window module is connected with the color acquisition module; the filtering window module is used for filtering interference light outside the wave band of the imaging window; the dynamic adjustable spatial spectrum modulation module comprises a random color mask and a programmable dynamic gray mask which are fixedly arranged, and is used for carrying out joint coding and modulation on an incident light field; The color acquisition module is used for detecting the coded and modulated light field to obtain a series of two-dimensional measurement values related to time; The sliding window module is used for implementing window translation operation on the time sequence and sending the continuously captured measured value sequence groups into the fusion reconstruction module; The fusion reconstruction module is used for receiving the multi-frame two-dimensional measurement values output by the color acquisition module and reconstructing a hyperspectral data cube sequence through a multi-frame reconstruction algorithm.
2. The color mask-based video-level lensless spectral imaging system of claim 1, wherein the programmable dynamic gray scale mask in the dynamically tunable spatial spectral modulation module is an LCD screen, a digital micromirror device, or an electrically tunable super surface, capable of switching different spatial coding patterns in a preset time sequence.
3. The system of claim 1, wherein the programmable dynamic gray-scale mask and the random color mask are arranged in close contact to form a compact joint coding unit.
4. The color mask based video level lensless spectral imaging system of claim 1, wherein the color acquisition module is an RGB color image sensor with bayer filter array or a custom multispectral image sensor.
5. The color mask based video grade lensless spectral imaging system of claim 1, wherein the filter window module is placed in close proximity to the random color mask while the size of the filter window module is larger than the size of the random color mask.
6. Imaging method using a color mask based video-level lensless spectral imaging system according to any of the claims 1-5, characterized in that the method comprises the steps of: s1, firstly, a scene light signal passes through a filtering window module to perform band prefiltering; S2, using a dynamic adjustable spatial spectrum modulation module to enable a programmable dynamic gray mask to switch different coding patterns according to time sequence, carrying out dynamic amplitude coding and diffraction projection on the filtered optical signals, and projecting point light sources in a scene to a color acquisition module in a multiplexing mode; S3, acquiring a two-dimensional measurement value corresponding to one frame under each coding pattern by utilizing a color acquisition module, so as to obtain a multi-frame measurement value sequence in a time window; S4, based on a point spread function corresponding to each coding pattern and a spectrum response function of the color acquisition module, which are obtained by calibration in advance, the fusion reconstruction module uses a multi-frame reconstruction algorithm to carry out joint solution on the multi-frame measured value sequences, and a hyperspectral data cube in the time window is reconstructed; S5, repeating the steps S2 to S4 through sliding a time window, and continuously outputting a hyperspectral data cube sequence to realize video-level spectral imaging.
7. The method according to claim 6, wherein in step S2, the programmable dynamic gray mask switched coding pattern comprises a random pattern, a checkerboard pattern or a stripe pattern.
8. The method of claim 6, wherein in step S4, the multi-frame reconstruction algorithm is implemented using a combination of an alternating direction multiplier method and a convolutional neural network.
9. The method according to claim 8, wherein in step S4, the multi-frame reconstruction algorithm initially uses an alternate direction multiplier method to perform preliminary reconstruction on single-frame measurement values, and then inputs multi-frame preliminary reconstruction results into a convolutional neural network for fusion and quality enhancement.
10. The method according to claim 6, wherein in step S5, the specific step of sliding the time window comprises: Firstly, defining and initializing a window, setting a time window T with a fixed length, controlling a dynamic adjustable spatial spectrum modulation module to sequentially switch T different coding patterns, and synchronously capturing corresponding T-frame two-dimensional measured values by a color acquisition module to form a first complete data window; When the data at the next moment needs to be processed, the window slides forwards for one step, the coding patterns continue to be switched in sequence, namely, the oldest measured value of one frame in the window is removed, and meanwhile, a newly acquired measured value of one frame is added, so that a new data window is formed; Each complete time window triggers a fusion reconstruction process, and the multi-frame reconstruction algorithm regards multi-frame data in the current time window as a whole to reconstruct a hyperspectral data cube corresponding to the central moment of the current time window.

Description

Video-level lens-free spectrum imaging system based on color mask and method thereof Technical Field The invention relates to a video-level lens-free spectral imaging system and method based on a color mask, and belongs to the technical field of spectral imaging. Background The snapshot spectrum imaging technology can capture three-dimensional space-spectrum information of a scene in single exposure, and has wide application value in the fields of industrial production detection, intelligent agriculture, combustion dynamics analysis, environmental resource exploration and the like. With the rapid development of technology, new applications such as wearable devices, autopilot, robotics, virtual/augmented reality, etc. place higher demands on the optical imaging system. These applications require that the spectral imaging system be compact, low cost, high speed and high spectral resolution. At present, a snapshot spectrum imaging system shows a remarkable miniaturization development trend. From early reliance on long and narrow relay lens sets and high volume beam splitters, to recent years with new materials or pixel-level filtering such as DOE and super surfaces, snapshot spectral imaging systems have been scaled down to the size of conventional lens imaging systems. However, these systems still do not get rid of the limitations of lens focusing imaging, and the overall size of the system cannot be further reduced due to the influence of the focal length of the lens. Therefore, a further miniaturization is motivated by the combination with lensless imaging techniques. The lens-free imaging 'point-to-multiple' imaging configuration and the integrated coding paradigm can further reduce the scale of the imaging system to the level of a near sensor, and become an important direction for miniaturization of spectral imaging. The prior art, such as Spectral DiffuserCam, combines frosted glass as a random phase encoder with a color filter array to achieve lensless snapshot spectral imaging. Similarly, other studies have employed Linear Variable Filters (LVFs) in combination with phase masks or fixed amplitude masks as the encoding device. While these systems have made significant progress in miniaturization, there are drawbacks in that, first, the encoding scheme is fixed and not adjustable, and that the physical structure and optical characteristics of the scattering medium or amplitude mask used in the system are fixed. This fixed encoding mode results in a system that cannot be changed once manufactured, limiting the information dimension and diversity it can acquire from a single exposure. Secondly, the reconstruction problem is highly underdetermined, the imaging precision is low, and when the inverse problem of reconstructing a high-dimensional spectrum data cube from two-dimensional measurement values is faced, the provided constraint conditions are seriously insufficient, which directly causes the problems of fuzzy details, low spectral precision, obvious noise and artifact and the like of the reconstructed image. In addition, high-quality video-level spectral imaging cannot be realized, namely a fixed coding mode and a single-frame reconstruction frame cannot provide enough information redundancy and time consistency for continuous capturing of dynamic scenes, and although video can be recorded by improving the frame rate of a camera, each frame is independently reconstructed based on single exposure with insufficient information, so that high-quality and high-frame-rate spectral video output is difficult to realize while the spectral accuracy is ensured. Disclosure of Invention The invention aims to overcome the defects of the prior art, and provides a lens-free spectrum imaging system and method which are based on high integration of a color mask, can be dynamically regulated and controlled and support multi-frame reconstruction, so as to solve the problems of low reconstruction precision, incapability of adapting to dynamic scenes and the like caused by fixed coding. The technical scheme adopted by the system of the invention is as follows: The video-level lens-free spectrum imaging system based on the color mask comprises a filtering window module, a dynamic adjustable spatial spectrum modulation module, a color acquisition module, a sliding window module and a fusion reconstruction module, wherein the filtering window module, the dynamic adjustable spatial spectrum modulation module and the color acquisition module are sequentially arranged in sequence, the fusion reconstruction module is respectively connected with the dynamic adjustable spatial spectrum modulation module, the color acquisition module and the sliding window module, the sliding window module is connected with the color acquisition module, the filtering window module is used for filtering interference light rays outside an imaging window band, the dynamic adjustable spatial spectrum modulation module comprises a random color mask and a