Search

CN-116559128-B - Structured light super-resolution microscopic imaging method and system based on deep learning

CN116559128BCN 116559128 BCN116559128 BCN 116559128BCN-116559128-B

Abstract

The invention discloses a structured light super-resolution microscopic imaging method and system based on deep learning. The system comprises a light source, a first lens, a polarization beam splitter prism, a Spatial Light Modulator (SLM), a second lens, a photographing mode switching module, a third lens, a micro objective lens, a three-dimensional electric object stage, a dichroic mirror, a barrel lens, a color filter and a camera which are sequentially distributed along the direction of a light path, wherein the photographing mode switching module comprises a 3D-SIM mode, a 2D-SIM mode and a wide field mode. The method is a structured light illumination microscopic super-resolution imaging method based on deep learning and assisted by key frames. The invention combines the wide-field imaging and the structured light illumination super-resolution microscopic imaging technology, can reduce phototoxicity and photobleaching in the structured light illumination super-resolution microscopic imaging process, and realizes long-time super-resolution dynamic observation of the living cell sub-microscopic structure.

Inventors

  • TANG YUJUN
  • LI HUI
  • WEN GANG
  • LIANG YONG
  • WANG LINBO

Assignees

  • 中国科学院苏州生物医学工程技术研究所

Dates

Publication Date
20260512
Application Date
20230428

Claims (8)

  1. 1. The structured light super-resolution microscopic imaging method based on the deep learning is used for realizing long-time super-resolution dynamic observation of a living cell sub-microscopic structure and is characterized by being realized based on a structured light super-resolution microscopic imaging system based on the deep learning, wherein the system comprises a light source, a first lens, a polarization beam splitter prism, a Spatial Light Modulator (SLM), a second lens, a photographing mode switching module, a third lens, a microscope objective, a three-dimensional electric object stage, a dichroic mirror, a barrel lens, a color filter and a camera which are sequentially distributed along the light path direction; The light emitted by the light source is vertically incident to the spatial light modulator SLM through the first lens and the polarization beam splitter prism, the diffracted light from the spatial light modulator SLM returns along an original path, the diffracted light is reflected by the polarization beam splitter prism to enter the second lens, the polarized light enters the illumination mode switching module after being collimated by the lens, and the illumination mode switching module is used for switching different illumination modes according to specific imaging requirements; the photographing mode switching module includes three modes: 1) 3D-SIM mode, which allows 0 th and plus or minus 1 st diffraction light to pass through; 2) 2D-SIM mode, which allows only the positive and negative 1 st order diffracted light to pass through; 3) A wide field mode which allows only one of the three orders 0, positive 1 or negative 1 to pass through, or by disrupting the beam interference condition, or by switching the structured light stripe illumination mode within a single exposure time of the camera; the method comprises the following steps: Step 1, switching an illumination mode of the structured light super-resolution microscopic imaging system into a 3D-SIM mode or a 2D-SIM mode, and acquiring N frames of original images modulated by the structured light illumination mode; step 2, reconstructing and obtaining a super-resolution image by an algorithm based on the original image, wherein the super-resolution image is used as a key frame; step 3, switching the illumination mode of the structured light super-resolution microscopic imaging system into a wide-field mode, and collecting a wide-field image at the current moment; step 4, inputting the key frame and the wide-field image into a neural network to obtain a super-resolution image at the current moment; step 5, collecting the wide field image at the next moment, Step 6, inputting the key frame and the wide field image at the next moment into a neural network to obtain a super-resolution image at the current moment; And 7, repeatedly executing the steps 5 to 6 until the super-resolution imaging is completed.
  2. 2. The structured light super-resolution microscopic imaging method based on deep learning according to claim 1, wherein the steps 5 to 7 are replaced by: Step 5, judging whether the morphological change of the biological structure exceeds a preset change threshold, if so, executing the step 1 and the step 2, obtaining a key frame, and then collecting a wide field image at the next moment to execute the next step; step 6, if the number of the current key frames is one, the step is executed, namely the key frames and the wide field images at the next moment are input into a neural network to obtain a super-resolution image at the current moment, otherwise, any one or more key frames and the wide field images at the next moment are input into the neural network to obtain the super-resolution image at the current moment; And 7, repeatedly executing the steps 5 to 6 until the super-resolution imaging is completed.
  3. 3. The structured light super-resolution microscopic imaging method based on deep learning according to claim 2, wherein the judging condition in the step 5 is replaced by whether a new biological structure is present.
  4. 4. The structured light super-resolution microscopic imaging method based on deep learning according to claim 3, wherein the judging condition in the step 5 is replaced by whether the time difference between the current time and the initial time reaches a preset time interval threshold.
  5. 5. The structured light super-resolution microscopic imaging method based on deep learning according to claim 2, wherein the input of the neural network in step 6 further comprises one or more super-resolution images output by the neural network at previous time instants, and the existence relationship between the super-resolution images and the key frames is sum relationship.
  6. 6. The deep learning-based structured light super-resolution microscopic imaging method according to claim 1, wherein the photographing mode switching module comprises a porous mask plate, a polarization adjusting device and a fourth lens which are sequentially arranged along the light path.
  7. 7. The deep learning-based structured light super-resolution microscopic imaging method according to claim 6, wherein the porous mask plate is used for spatial filtering, and comprises 2N pinholes distributed in a rotationally symmetrical manner, or a combination of the pinhole distribution modes, wherein N is an integer.
  8. 8. The deep learning based structured light super resolution microscopic imaging method according to claim 6, wherein the polarization adjustment means comprises a combination of a polarization rotator and a liquid crystal phase compensator, or a combination of two half wave plates.

Description

Structured light super-resolution microscopic imaging method and system based on deep learning Technical Field The invention belongs to the technical field of fluorescence microscopy imaging, and particularly relates to a method and a system for performing dynamic long-time super-resolution microscopy imaging of living cells based on mutual fusion of a wide-field illumination mode and a structural illumination mode of deep learning. Background In life science research, fluorescence microscopy is widely used because of its advantages such as specificity markability, ability to image living cells in real time, etc. However, since fluorescence microscopy is limited by diffraction limits, its use in the field of biomedical research is greatly limited. In recent years, scientific researchers have proposed super-resolution microscopy capable of breaking through diffraction limits, including stimulated emission depletion microscopy, light activated positioning microscopy, random optical reconstruction microscopy, structural illumination microscopy, and the like. The structured light illumination micro technology illuminates a sample through modulated stripe structured light, and then the resolution twice as high as the diffraction limit can be realized through image reconstruction. Compared with other super-resolution imaging technologies, the structured light obvious micro-technology has higher time resolution (about 80 Hz), has no special requirement on fluorescent dye of a marked sample, has lower phototoxicity (about 10W/cm < 2 >) and is important for dynamic observation of living cells. Therefore, the structured light illumination microscope is mainly used for in vivo observation of subcellular level, including mitochondrial dynamic change, cytoskeletal dynamic change, chromosome dynamic change, intracellular vesicle movement, movement of virus in cells, and the like. However, the structured light illumination microscopy requires multiple measurements to reconstruct a super-resolution image, and in the reconstruction algorithm of the structured light microscopy, complex operations such as frequency domain information separation, frequency domain information stitching and the like are involved. However, such reconstruction algorithms require a high signal-to-noise ratio of the original fluorescent image. Otherwise, if the super-resolution reconstruction is performed by using the original fluorescent image with low signal-to-noise ratio, the final super-resolution image has large artifacts. The existence of the artifacts can seriously affect the quality of the final super-resolution reconstructed image, so that the real sample information cannot be distinguished from the artifacts generated in the reconstruction process, and further the microscopic observation effect is affected. In order to solve the defects, a structured light illumination super-resolution microscopic imaging method based on deep learning is generated. Deep learning based approaches have met with great success in learning end-to-end image transformations from a large number of example data. Since 2019 deep learning is applied to structural light super-resolution microscopic imaging for the first time, a plurality of students have proposed corresponding structural light illumination super-resolution reconstruction algorithms based on U-Net, generation of a countermeasure network (GAN), a Residual Channel Attention Network (RCAN) and the like. These algorithms mainly solve two types of problems, 1) improving the imaging quality under low signal-to-noise ratio conditions, reducing phototoxicity, and 2) reducing the number of original images required for reconstruction. The above problems can be solved by effectively improving the time resolution of the structural illumination for obvious microimaging, reducing the photo-bleaching and phototoxicity in the imaging process and providing possibility for long-time live cell real-time dynamic imaging. However, the super-resolution image obtained from the wide-field image to the structural light modulation based on the deep learning method is still difficult to obtain the ideal effect, and the reconstructed super-resolution structure is not reliable. The wide-field imaging can further reduce phototoxicity and photobleaching in the dynamic imaging process of living cells and prolong the dynamic imaging time of the living cells. Because the diffraction-limited wide-field image does not contain resolvable super-resolution information, the traditional algorithm based on analysis models such as Wiener deconvolution and the like cannot obtain the super-resolution image through the wide-field image. The method based on the deep learning does not need an explicit analysis model, and the method based on the data driving can approximate not only the pseudo-inverse function of the image degradation process but also the random characteristic of the super-resolution solution. Therefore, it is important to study how