Search

CN-121090490-B - Full-light three-dimensional scanning confocal fluorescence microscopic imaging device and implementation method thereof

CN121090490BCN 121090490 BCN121090490 BCN 121090490BCN-121090490-B

Abstract

The invention discloses an all-optical three-dimensional scanning confocal fluorescence microscopic imaging device and an implementation method thereof. The invention processes space distribution data by adopting CNN, dynamically regulates and controls phase hologram, light intensity and phase compensation of a laser, enables focus of excitation light to have no aberration axial displacement, optimizes bidirectional parallel optical scanning track of a two-dimensional scanning system, realizes full-light three-dimensional scanning, combines LSTM and TCN to obtain time sequence dependency, integrally controls a laser light source, an adjustable diaphragm, an electric adjustable focusing lens and a photoelectric detector, efficiently and synchronously and automatically operates and axially focuses and adjusts at high speed, eliminates error and time sequence mismatch, automatically adapts different sample optical characteristics and environmental interference, keeps high-quality imaging, improves system robustness and applicable scene range, and is used for mode biological embryo real-time tracking, intracellular signal molecule dynamic visualization, cell membrane protein migration aggregation observation and intracellular organelle interaction tracking.

Inventors

  • GONG JINGRUI
  • WANG YUE
  • XU NING
  • CHEN YONGHAO
  • SHI KEBIN
  • YANG HONG

Assignees

  • 北京大学

Dates

Publication Date
20260505
Application Date
20250912

Claims (10)

  1. 1. The full-light three-dimensional scanning confocal fluorescence microscopic imaging device is characterized by comprising an excitation light source, a depth self-adaptive regulation and control device, a dichroic mirror, an adjustable diaphragm, a two-dimensional scanning system, a telecentric scanning correction system, an objective lens, an electric adjustable lens, a pinhole, a photoelectric detector and an intelligent control system, wherein the two-dimensional scanning system is connected to a data acquisition card, the photoelectric detector is connected to an image acquisition card; the method comprises the steps of (a) using an excitation light source to emit polarized Gaussian beams as excitation light, entering a depth self-adaptive regulation and control device, regulating and controlling the phase of the excitation light by loading a phase hologram, screening out 1-level diffraction light by a dichroic mirror and then using an adjustable aperture to carry out two-dimensional plane optical scanning, and focusing the light to a sample by an objective lens; The sample generates fluorescent signals, the fluorescent signals are collected through the same objective lens and reversely transmitted back to the two-dimensional scanning system along the direction from which the excitation light comes, the two-dimensional scanning system is used for scanning, and the dichroic mirror and the excitation light are used for splitting beams; The intelligent control system adopts a self-adaptive regulation and control method driven by deep learning to optimize a phase hologram calculated by a feedback aberration-free axial scanning algorithm to obtain an accurate relation between the phase hologram and an axial position, extracts local distortion characteristics, light intensity distribution and phase errors of the phase hologram through a multi-input multi-output convolution neural network to generate static correction quantity to correct pixel nonlinear response of a spatial light modulator, optimizes the light intensity of the laser according to the axial position of a focus, performs phase compensation, combines a long-term memory network and a time convolution network to obtain a time sequence dependency relation, dynamically eliminates interference of ambient temperature and light path vibration, enables the excitation light to realize aberration-free axial displacement of a rear focus of an objective lens, optimizes a bidirectional parallel optical scanning track of a two-dimensional scanning system in real time, and cooperatively controls a laser light source, an adjustable diaphragm, a two-dimensional scanning system, an electric focusing lens and a photoelectric detector to realize full-light three-dimensional scanning.
  2. 2. The fluorescence microscopy imaging device of claim 1, further comprising a beam expansion collimation device that uses a dual lens to expand and collimate the beam, wherein the expanded beam diameter matches the spatial light modulator of the depth adaptive modulation device.
  3. 3. The fluorescence microscopic imaging apparatus according to claim 1, wherein the depth-adaptive modulation device comprises a half-wave plate, a prism and a spatial light modulator, wherein the half-wave plate modulates the polarization state of the incident excitation light, the linear polarization light is adjusted to be a polarization state matched with the optimal working polarization direction of the spatial light modulator by precisely rotating the angle of the half-wave plate, the prism is matched with the spatial light modulator to perform light field modulation on the Gaussian light, the Gaussian light is reflected to the incident spatial light modulator through one side of the prism, and the spatial light modulator changes the focus of the excitation light at the axial position behind the objective lens by loading a phase hologram.
  4. 4. The fluorescence microscopy imaging device of claim 1, further comprising a telecentric scan correction system disposed between the two-dimensional scan system and the objective lens, comprising a scan lens and a first tube lens, wherein light passing through the two-dimensional scan system passes through the scan lens and then through the first tube lens, ensuring that light is relayed from the two-dimensional scan system to a back focal plane of the objective lens.
  5. 5. The fluorescence microscopy imaging device of claim 1, wherein the two-dimensional scanning system comprises a resonant galvanometer and a high precision galvanometer; the high-speed scanning is realized by adopting a resonant galvanometer vibrating mirror in the x direction, the working frequency is stabilized at the kilohertz level, the high-efficiency cosine track scanning is realized by adopting the mechanical resonance characteristic, and the programmable positioning scanning can be realized by adopting a high-precision galvanometer vibrating mirror in the y direction.
  6. 6. A method of implementing an all-optical three-dimensional scanning confocal fluorescence microscopy imaging apparatus according to claim 1, said method comprising the steps of: 1) The excitation light source emits polarized Gaussian beams as excitation light, and the polarized Gaussian beams enter the depth self-adaptive regulation and control device; 2) The spatial light modulator of the depth self-adaptive regulation and control device loads a phase hologram which is calculated by a feedback aberration-free axial scanning algorithm and is regulated in real time by a self-adaptive regulation and control method driven by the depth learning to regulate and control the phase of excitation light, so that a focus is moved along the axial aberration-free direction after an objective lens; 3) 1-order diffraction light is screened out through an adjustable aperture after passing through a dichroic mirror and then is sent to a two-dimensional scanning system, two-dimensional plane optical scanning is carried out, and the light is focused to a surface or an internal area to be observed of a sample through an objective lens; 4) Generating fluorescent signals on the surface or inside of the sample, collecting the fluorescent signals through the same objective lens, reversely transmitting the fluorescent signals back to the two-dimensional scanning system along the direction from which the excitation light comes, performing rescanning through the two-dimensional scanning system, and splitting the fluorescent signals into beams through the dichroic mirror and the excitation light; 5) Through the electric adjustable focusing lens, the focal length is adjusted according to the change of the excitation light along the axial position, so that fluorescent signals at different axial positions in a sample can be focused on the detection surface of the photoelectric detector; 6) Filtering out defocusing signals through a pinhole positioned in front of the photoelectric detector, and then receiving the defocusing signals by the photoelectric detector; 7) The intelligent control system is used for obtaining an accurate relation between the phase hologram and the axial position by adopting a self-adaptive regulation and control method driven by deep learning, optimizing the light intensity of a laser according to the axial position of a focus, eliminating the inherent aberration of an optical system, dynamically regulating and controlling the phase distribution of the depth self-adaptive regulation and control device, and the light intensity and phase compensation of the laser, so that the excitation light is subjected to aberration-free axial displacement of the focus behind an objective lens with high numerical aperture, and the bidirectional parallel optical scanning track of a two-dimensional scanning system is optimized in real time, thereby realizing full-optical three-dimensional scanning.
  7. 7. The method of claim 6, wherein the adaptive control method driven by deep learning is based on a neural network architecture constrained by a physical model, the neural network architecture adopts a multi-input multi-output convolutional neural network CNN, inputs are three types of spatial data, the CNN extracts local distortion characteristics of phase holograms, sidelobe modes of light intensity distribution and spatial distribution rules of phase errors through a convolutional layer, outputs are three types of static correction amounts after the spatial correction is carried out by pooling layer compression, the first type of input is a phase hologram loaded by a spatial light modulator SLM, the corresponding first type of output is a phase hologram for optimizing a spatial residual, the nonlinear response of SLM pixels caused by residual due to objective conditions of a device or environmental conditions is corrected, the second type of input is a focal area light intensity distribution image, the focal light intensity consistency of different axial positions is improved, the corresponding second type of output is a light intensity distribution equalization parameter, the corresponding light intensity of a laser is optimized according to the axial position of a focal point, the third type of input is a phase residual map, namely the difference distribution of an actual phase residual and a theoretical phase difference, the corresponding static aberration is eliminated, and the corresponding static aberration is a phase difference of the optical system is compensated for different optical matrixes.
  8. 8. The implementation method of claim 7, wherein the neural network architecture further comprises a long-short-term memory network LSTM and a time convolution network TCN, so as to obtain a time sequence dependency relationship, and the method is used for integrally controlling a laser light source, an adjustable diaphragm, a two-dimensional scanning system, an electric adjustable focusing lens and a photoelectric detector, controlling efficient synchronization and automation operation of multiple devices and high-speed axial focusing adjustment.
  9. 9. The implementation method of claim 8, wherein the data set of the long-short-term memory network LSTM is obtained by collecting two kinds of time sequence data according to a scanning frame rate of an all-optical three-dimensional scanning confocal fluorescence microscopy imaging device as input, wherein the three kinds of static correction sequences are CNN output, each group of sequences comprises a phase hologram for optimizing space residual, a light intensity distribution equalization parameter and a static phase compensation matrix, each group of sequences covers a complete scanning period, the two kinds of focal position error time sequence data recorded by a camera synchronously collect environment temperature and optical path vibration parameters to ensure time alignment with the static correction sequences, the focal position error time sequence data is adopted to carry out time sequence sorting correction on the static correction sequences of the output of the convolutional neural network, and the three kinds of dynamic correction parameters are output, namely a time sequence smooth phase adjustment sequence obtained by taking phase mutation reverse compensation quantity of adjacent frames, a real-time phase compensation increment obtained by actual measurement residual calculation and a position prediction correction value obtained by predicting position error and taking reverse value.
  10. 10. The implementation method of the invention of claim 9, characterized in that a time convolution network TCN models a time sequence dependency relationship and optimizes a control instruction sequence, dynamic correction parameters output by an LSTM (least squares) are used as inputs of the time convolution network together with a power adjustment instruction of a laser light source, an aperture control signal of an adjustable diaphragm, a scanning track instruction of a two-dimensional scanning system, a focal length parameter of an electric adjustable lens and a device control sequence of sampling frequency of a photoelectric detector, the TCN optimally generates a multi-device cooperative control instruction sequence through a time sequence dependency relationship among a causal convolution modeling multi-device and an expansion convolution modeling multi-device, ensures accurate synchronization of the power of the laser light source, the aperture of the adjustable diaphragm, the scanning track of the two-dimensional scanning system, the focal length of the electric adjustable lens and the phase modulation of the photoelectric detector and the SLM, and realizes efficient cooperative control of the laser light source, the adjustable diaphragm, the two-dimensional scanning system, the electric adjustable lens and the photoelectric detector.

Description

Full-light three-dimensional scanning confocal fluorescence microscopic imaging device and implementation method thereof Technical Field The invention relates to the field of microscopic imaging, in particular to an all-optical three-dimensional scanning confocal fluorescence microscopic imaging device and an implementation method thereof. Background Optical fluorescence microscopy, an important branch of the modern optical field, is continuously innovated with the breakthrough progress of optical engineering, and shows irreplaceable application value in biomedical research. The optical fluorescence microscope works based on the principles of fluorescence labeling and optical detection, and fluorescent molecules in a sample are prompted to emit fluorescent signals by excitation light of a specific wavelength. The technology becomes an indispensable important tool for modern biomedical research, and provides key technical support for revealing basic laws of vital activities. Contemporary biological imaging places three central demands on optical microscope systems, large field coverage capability, high spatial-temporal resolution, and three-dimensional volumetric imaging functionality. These technical requirements essentially stem from the fundamental need for in vivo biological research-only with high fidelity optical imaging means can accurate observations and recordings of subcellular structures and their dynamic biological processes be achieved. Especially when researching the life phenomenon of fast transient, the imaging parameter indexes are often mutually restricted, and the technical bottleneck to be broken through in the current biological optical imaging field is formed. In the prior art system, the wide-field fluorescence microscopy technology can acquire a series of two-dimensional images by means of the single two-dimensional imaging characteristic of the wide-field fluorescence microscopy technology and matching with the three-dimensional scanning of a sample. However, this technique has inherent physical limitations, namely, excitation of the primary volume region during imaging, and the inability to effectively distinguish between focal and out-of-focus fluorescent signals, resulting in acquired images containing significant background noise. This technical defect directly leads to the inability to realize three-dimensional imaging, and cannot be applied to research requiring three-dimensional structural analysis. These limitations make it difficult for conventional wide-field fluorescence microscopy to meet the stringent requirements of modern biological research on imaging quality. Confocal microscopy is an important breakthrough of modern fluorescence microscopy, and multiple technical advantages are achieved through precise optical path design. The method comprises the steps of adopting a point scanning imaging mode, enabling an excitation light source to form diffraction limit light spots to scan a sample point by point after being focused by an objective lens, enabling fluorescence signals excited at each point position to be collected by the same objective lens, enabling a coaxial structure to ensure stability of in-situ excitation and imaging of a system, enabling a tightly focused light source to form a highly-limited excitation focus inside the sample, enabling laser to be focused to a position where a focus in the sample and a focus of a fluorescence signal incident into a detector are located at a conjugation position, enabling a pinhole to be closely attached to the surface of the photoelectric detector, enabling collected fluorescence to be separated by a dichroic mirror, enabling a conjugation pinhole to filter out information of the defocus signal only of the focus, and accordingly remarkably improving signal-to-noise ratio and image contrast of the system. Secondly, the characteristic that confocal microscope pinhole filters can effectively filter out defocusing signal makes it have excellent axial resolution, can realize submicron level optical section, and this characteristic makes it especially adapted to the biological research scene that needs high accuracy two-dimensional imaging. In addition, the confocal system also has good optical chromatography capability, can clearly distinguish the structural information of different depths inside the sample, and realizes the three-dimensional imaging of the thick sample. Thus, confocal systems are three-dimensional imaging tools that are excellent in mesostructures such as cells. However, the three-dimensional imaging speed of confocal microscopes is also limited by their point scanning principle of operation. In order to realize three-dimensional reconstruction, the system must acquire three-dimensional information of the sample in a point-by-point scanning mode, and the process obviously increases the data acquisition time. The current mainstream confocal three-dimensional imaging scheme mainly adopts two types of techn