CN-118424099-B - Multi-focal-plane oblique light sheet structured light three-dimensional point cloud reconstruction method and device

CN118424099BCN 118424099 BCN118424099 BCN 118424099BCN-118424099-B

Abstract

The invention discloses a multi-focal-plane oblique light sheet structured light three-dimensional point cloud reconstruction method and device, which belong to the technical field of biomedical imaging and comprise the steps of irradiating an illumination light with an oblique light sheet, shooting and collecting corresponding image frame sequences through a plurality of cameras, determining imaging light spots of the same fluorescent molecule in the image frame sequences, fitting the imaging light spots by a two-dimensional Gaussian function to obtain mass center coordinates and standard deviation parameters, obtaining a preset calibration curve, combining the calibration curve according to the standard deviation parameters to obtain three-dimensional coordinates of the fluorescent molecule, transforming the three-dimensional coordinates into normal three-dimensional coordinates according to the included angle between the focal plane of the camera and the horizontal direction, transforming the normal three-dimensional coordinates into global three-dimensional coordinates according to the moving step length of a displacement table, reconstructing three-dimensional point clouds of the plurality of cameras, obtaining a multi-camera registration file, registering the three-dimensional point clouds of the plurality of cameras according to the multi-camera registration file, and obtaining a three-dimensional point cloud depth image. The invention realizes the rapid scanning imaging and reconstruction of a large-volume sample.

Inventors

HUANG ZHENLI
ZHANG CHEN
CAI LEI

Assignees

华中科技大学

Dates

Publication Date: 20260512
Application Date: 20230131

Claims (10)

1. The multi-focal-plane oblique light sheet structured light three-dimensional point cloud reconstruction method is characterized by comprising the following steps of: Illuminating illumination light onto a sample in a diagonal light sheet, wherein the sample moves along with a displacement table in a certain step length, continuous imaging is carried out by a plurality of cameras, a plurality of camera shooting acquisition image frame sequences are obtained, focal planes of the plurality of cameras are parallel, and the moving direction of the displacement table is parallel to the arrangement direction of the plurality of cameras; determining imaging light spots of the same fluorescent molecule in a plurality of image frame sequences, and fitting the imaging light spots by adopting a two-dimensional Gaussian function to obtain centroid coordinates and standard deviation parameters of the fluorescent molecule; Acquiring a preset calibration curve, and calculating depth information of the fluorescent molecules according to the standard deviation parameter and the calibration curve to obtain three-dimensional coordinates of the fluorescent molecules in a camera coordinate system; transforming the three-dimensional coordinates into orthographic three-dimensional coordinates according to included angles between focal planes of the cameras and the horizontal direction; Transforming the positive three-dimensional coordinate into a global three-dimensional coordinate according to the moving step length of the displacement table, and reconstructing a plurality of three-dimensional point clouds of the cameras; And acquiring a multi-camera registration file, and registering a plurality of three-dimensional point clouds of the cameras according to the multi-camera registration file to acquire a three-dimensional point cloud depth image of the sample.
2. The multi-focal-plane oblique light sheet structured light three-dimensional point cloud reconstruction method according to claim 1, wherein the preset calibration curve is determined by the following method: Continuously imaging through the cameras to obtain an image frame sequence of a sample on the cameras as calibration data; intercepting an imaging light spot of the same reference fluorescent molecule in the image frame sequence, and fitting by utilizing a two-dimensional Gaussian function to obtain a standard deviation parameter of the imaging light spot of the reference fluorescent molecule; and determining the preset calibration curve according to the relation between the standard deviation parameter of the imaging light spot and the defocus depth.
3. The multi-focal-plane oblique light sheet structured light three-dimensional point cloud reconstruction method according to claim 2, wherein determining the preset calibration curve by the relationship between the standard deviation parameter of the imaging light spot and the defocus depth comprises: Determining a variance parameter of the imaging light spot in the x direction and the y direction through a standard deviation parameter of the imaging light spot; Fitting a relation curve of the difference of variance parameters of the imaging light spots in the x direction and the y direction and the defocus depth to serve as the preset calibration curve.
4. The method for reconstructing a three-dimensional point cloud of a multi-focal-plane oblique light sheet structure light according to claim 1, wherein the obtaining a preset calibration curve and calculating depth information of the fluorescent molecules according to the standard deviation parameter in combination with the calibration curve comprises: Obtaining a plurality of preset calibration curves corresponding to different reference fluorescent molecules and a plurality of ovality curves corresponding to different reference fluorescent molecules; Substituting the standard deviation parameters into the preset calibration curves to obtain a plurality of depth information; Substituting the plurality of depth information into the ellipticity curve to calculate corresponding ellipticity, and determining the depth corresponding to the ellipticity closest to the fitting ellipticity as the depth of the fluorescent molecule.
5. The multi-focal-plane oblique light sheet structured light three-dimensional point cloud reconstruction method according to claim 2, wherein the multi-camera registration file is determined by: determining an image frame with the minimum difference absolute value of standard deviation parameters as a focusing frame of the imaging light spot; And determining the relative position and the posture of the cameras according to the barycenter coordinates of the imaging light spots in the focusing frame and the moving step length of the displacement table, and storing the relative position and the posture as a multi-camera registration file.
6. The method of claim 5, wherein the relative position poses of the plurality of cameras comprise an axial spacing and a lateral spacing, the axial spacing of the plurality of cameras is a product of a frame difference between the focused frames and the movement step, and the lateral spacing of the plurality of cameras is a difference between centroid coordinates of the imaged spots in the focused frames.
7. The multi-focal-plane oblique light sheet structured light three-dimensional point cloud reconstruction method according to claim 1, wherein the three-dimensional coordinates are transformed into orthographic three-dimensional coordinates by a formula: x’ = x y’=ycos(a)-zsin(a) z’=ysin(a)+zcos(a) wherein a is the included angle between the focal plane of the camera and the horizontal direction.
8. The method for reconstructing a three-dimensional point cloud of a multi-focal-plane oblique light sheet structure according to claim 1, wherein the global three-dimensional coordinate is a sum of a positive three-dimensional coordinate and a three-dimensional coordinate of a displacement table at the same time, and the three-dimensional coordinate of the displacement table is determined by an initial position coordinate and a moving step length.
9. The multi-focal-plane oblique light sheet structured-light three-dimensional point cloud reconstruction method of claim 6, wherein the multi-camera registration file includes a relative position parameter of the plurality of cameras in an x-direction, a relative position parameter in a y-direction, a relative position parameter in a z-direction, the relative position parameter in the x-direction and the relative position parameter in the y-direction being a difference between centroid coordinates of an imaging spot in the focused frame, the relative position parameter in the z-direction being a product of a frame number difference between the focused frames and the movement step, the three-dimensional point cloud registration of the plurality of cameras including adding a coordinate value in the x-direction in the global three-dimensional coordinate to the relative position parameter in the x-direction, adding a coordinate value in the y-direction in the global three-dimensional coordinate to the relative position parameter in the y-direction, and adding a coordinate value in the z-direction in the global three-dimensional coordinate to the relative position parameter in the z-direction.
10. The utility model provides a multi-focal plane oblique light piece structured light three-dimensional point cloud reconstruction device which characterized in that, multi-focal plane oblique light piece structured light three-dimensional point cloud reconstruction device includes: The image acquisition module is used for irradiating illumination light onto a sample in an oblique light sheet, the sample moves along with the displacement table in a certain step length, continuous imaging is carried out through a plurality of cameras, an image frame sequence shot and acquired by the plurality of cameras is obtained, focal planes of the plurality of cameras are parallel, and the moving direction of the displacement table is parallel to the arrangement direction of the plurality of cameras; the light spot fitting module is used for determining imaging light spots of the same fluorescent molecule in a plurality of image frame sequences, and fitting the imaging light spots by adopting a two-dimensional Gaussian function to obtain centroid coordinates and standard deviation parameters of the fluorescent molecules; The depth calculation module is used for obtaining a preset calibration curve, calculating the depth information of the fluorescent molecules according to the standard deviation parameter combined with the calibration curve, and obtaining three-dimensional coordinates of the fluorescent molecules in a camera coordinate system; The coordinate orthographic module is used for converting the three-dimensional coordinates into orthographic three-dimensional coordinates according to the included angles between the focal planes of the cameras and the horizontal direction; the coordinate transformation module is used for transforming the positive three-dimensional coordinate into a global three-dimensional coordinate according to the moving step length of the displacement table and reconstructing a plurality of three-dimensional point clouds of the plurality of cameras; The camera registration module is used for acquiring a multi-camera registration file, registering a plurality of three-dimensional point clouds of the cameras according to the multi-camera registration file, and acquiring a three-dimensional point cloud depth image of the sample.

Description

Multi-focal-plane oblique light sheet structured light three-dimensional point cloud reconstruction method and device Technical Field The invention relates to the technical field of biomedical imaging, in particular to a multi-focal-plane oblique light sheet structured light three-dimensional point cloud reconstruction method and device. Background At present, various ultra-high resolution optical microscopy imaging methods for breaking the optical diffraction limit are sequentially adopted, and the ultra-high resolution imaging methods can be roughly divided into two types according to the characteristics of the ultra-high resolution imaging methods, namely, one imaging technology for effectively reducing the point spread function, such as stimulated radiation loss fluorescence microscopy imaging, ground state loss super-resolution imaging, saturated structure light illumination microscopy imaging and the like, and the other imaging technology based on single molecule positioning, such as light activation positioning microscopy imaging and random optical reconstruction microscopy imaging. In the three-dimensional imaging technology based on single-molecule positioning, a cylindrical mirror or a mask plate is generally used for modulating fluorescent signals, so that shot patterns change along with depth, and the depth of fluorescent molecules is recovered according to pattern information, thereby realizing three-dimensional positioning. In cylindrical mirror three-dimensional positioning imaging, the focal points of fluorescent molecules in the X and Y directions are not located at the same position by using astigmatism brought by the cylindrical mirror, when the fluorescent molecules are located at the proper position between the two focal points, light spots obtained in imaging are symmetrical circles, and the intensity distribution of the imaging light spots at other positions is elliptical with different ellipticity. The position of fluorescent molecules from the focusing plane can be obtained according to the ellipticity of the imaging light spot by calibrating the change of the ellipticity of the imaging light spot along with the focusing depth, and whether the fluorescent molecules are positioned above or below the focusing plane is judged according to whether the ellipticity exceeds 1. However, the three-dimensional positioning method based on astigmatism is only less than 1um in reconstruction thickness of single imaging when being used for scanning a large-volume sample. On the other hand, the conventional light sheet microscope is used for cell or subcellular level imaging, and often only multi-angle imaging is used, and deconvolution post-processing is performed, so that the data volume is huge, and time and labor are wasted. Disclosure of Invention In order to solve the technical problem of low single reconstruction thickness in the existing large-volume sample scanning, the embodiment of the invention provides a multi-focal-plane oblique light sheet structured light three-dimensional point cloud reconstruction method and device. The technical scheme is as follows: the multi-focal-plane oblique light sheet structured light three-dimensional point cloud reconstruction method comprises the following steps: Illuminating illumination light onto a sample in a diagonal light sheet, wherein the sample moves along with a displacement table in a certain step length, continuous imaging is carried out by a plurality of cameras, a plurality of camera shooting acquisition image frame sequences are obtained, focal planes of the plurality of cameras are parallel, and the moving direction of the displacement table is parallel to the arrangement direction of the plurality of cameras; determining imaging light spots of the same fluorescent molecule in a plurality of image frame sequences, and fitting the imaging light spots by adopting a two-dimensional Gaussian function to obtain centroid coordinates and standard deviation parameters of the fluorescent molecule; Acquiring a preset calibration curve, and calculating depth information of the fluorescent molecules according to the standard deviation parameter and the calibration curve to obtain three-dimensional coordinates of the fluorescent molecules in a camera coordinate system; transforming the three-dimensional coordinates into orthographic three-dimensional coordinates according to included angles between focal planes of the cameras and the horizontal direction; Transforming the positive three-dimensional coordinate into a global three-dimensional coordinate according to the moving step length of the displacement table, and reconstructing a plurality of three-dimensional point clouds of the cameras; And acquiring a multi-camera registration file, and registering a plurality of three-dimensional point clouds of the cameras according to the multi-camera registration file to acquire a three-dimensional point cloud depth image of the sample. Further, the prese