CN-122023649-A - High-speed reconstruction and axial splicing method for microscopic light field based on neural network

CN122023649ACN 122023649 ACN122023649 ACN 122023649ACN-122023649-A

Abstract

The invention discloses a high-speed reconstruction and axial splicing method of a microscopic light field based on a neural network, and relates to the technical field of computational optics, wherein the method comprises the steps of shooting light field images with different depths, and obtaining light field data containing spatial position and depth information; the method comprises the steps of preprocessing a light field image sequentially through a distortion correction technology and a digital self-adaptive optical technology, digitally refocusing the preprocessed light field image to obtain a three-dimensional body formed by stacking, axially splicing the three-dimensional body formed by stacking to obtain an original three-dimensional body, inputting the original three-dimensional body into a three-dimensional convolutional neural network to perform optimization processing, outputting the axially spliced three-dimensional body, and storing the axially spliced three-dimensional body as a TIFF picture. The embodiment of the invention can realize seamless splicing of the light field three-dimensional images with a large axial range, improve splicing continuity and image resolution, and greatly improve reconstruction speed and calculation efficiency by optimizing a processing flow through a three-dimensional convolutional neural network.

Inventors

LU ZHI
LI YUAN

Assignees

清华大学

Dates

Publication Date: 20260512
Application Date: 20260123

Claims (9)

1. A method for reconstructing a microscopic light field at a high speed and splicing the microscopic light field axially based on a neural network is characterized by comprising the following steps: shooting light field images with different depths, and acquiring light field data containing spatial position and depth information; Preprocessing the light field image sequentially through a distortion correction technology and a digital self-adaptive optical technology; digitally refocusing the preprocessed light field image to obtain a stacked three-dimensional body; the three-dimensional body formed by stacking is subjected to axial splicing, so that an original three-dimensional body is obtained; Inputting the original three-dimensional body into a three-dimensional convolutional neural network for optimization treatment, and outputting an axially spliced three-dimensional body; and saving the axially spliced three-dimensional body as a TIFF picture.
2. The method of claim 1, wherein the preprocessing of the light field image by distortion correction techniques and digital adaptive optics techniques in sequence comprises: performing distortion correction on the optical field image by a distortion correction technology; the aberration of the optical field image after distortion correction is removed by a digital self-adaptive optical technology, and the aberration removing formula is expressed as follows: Wherein, the In order to correct the pixels of the image, Representing spatial coordinates and angular coordinates, respectively.
3. The method of claim 1, wherein digitally refocusing the preprocessed light field image to obtain a stacked three-dimensional volume comprises: carrying out refocusing calculation on each light field image at a plurality of different axial depth positions by adopting a digital refocusing algorithm to obtain refocused images corresponding to different focusing depths; And stacking the refocused images along the axial direction to form a three-dimensional data body with limited axial size.
4. The method of claim 1, wherein the axially stitching the stacked three-dimensional body to obtain an original three-dimensional body comprises: the method comprises the steps of carrying out weight distribution on three-dimensional bodies with limited axial sizes at different focal planes by calculating depth information of shot light field images, wherein a single-layer original weight distribution formula of a splicing mode is as follows: Wherein, the Is the distribution of weights over different distances for a single small three-dimensional volume, Is its absolute distance from the focal plane; And smoothly splicing each small three-dimensional body to form a complete original digital refocusing three-dimensional body with large axial length.
5. The method of claim 4, wherein said smoothly stitching each small three-dimensional volume to form a complete large axial length original digitally refocused three-dimensional volume comprises: calculating the normalized absolute weight of each small three-dimensional body, wherein the calculation formula is as follows: Wherein, the Is the absolute weight of each small three-dimensional volume at each depth when they are fused into a large three-dimensional volume, Is a small three-dimensional volume size associated with the height; according to the normalized absolute weight, fusing all single-layer pictures of the small three-dimensional body at the corresponding positions to obtain a complete single-layer picture of the large three-dimensional body, wherein the formula of picture fusion is as follows: Wherein, the Is a single-layer picture of a complete large three-dimensional volume, Is a single-layer picture of a small three-dimensional body at a corresponding position.
6. The method of claim 1, wherein the inputting the original three-dimensional volume into the three-dimensional convolutional neural network for optimization, outputting an axially stitched three-dimensional volume, comprises: Using a multi-layer three-dimensional convolutional neural network, the activation function is set to Extracting characteristic information in the original three-dimensional body and recovering the characteristic information into an image three-dimensional body; And restoring the original physical scale of the image three-dimensional body through bilinear interpolation, and outputting the axially spliced three-dimensional body.
7. The utility model provides a microscopic light field high-speed rebuilds and axial splicing apparatus based on neural network which characterized in that includes: The image acquisition module is used for shooting light field images with different depths and acquiring light field data containing space position and depth information; the distortion correction and digital self-adaption module is used for preprocessing the light field image sequentially through a distortion correction technology and a digital self-adaption optical technology; Digital refocusing Jiao Mokuai, which is used for carrying out digital refocusing on the preprocessed light field image to obtain a stacked three-dimensional body; The axial splicing module is used for axially splicing the three-dimensional bodies formed by stacking to obtain an original three-dimensional body; The optimizing module is used for inputting the original three-dimensional body into a three-dimensional convolutional neural network to perform optimizing treatment and outputting an axially spliced three-dimensional body; And the storage module is used for storing the axially spliced three-dimensional body as a TIFF picture.
8. A computer device comprising a processor and a memory; wherein the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, for implementing a neural network-based microscopic light field high-speed reconstruction and axial splicing method as set forth in any one of claims 1-6.
9. A non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor, implements a neural network-based method of high-speed reconstruction and axial stitching of a microscopic light field as claimed in any one of claims 1-6.

Description

High-speed reconstruction and axial splicing method for microscopic light field based on neural network Technical Field The invention relates to the technical field of computational optics, in particular to a high-speed reconstruction and axial splicing method of a microscopic light field based on a neural network. Background Computational imaging techniques are techniques that can record four-dimensional light field information and calculate and restore three-dimensional objects based thereon. The technology has wide application in the front fields of biology, medicine and the like, and has profound effects on scientific research and technical progress in various fields. However, due to the limitations of existing hardware, the depth of field of a three-dimensional image obtained based on the light field reconstruction technology is smaller in the axial direction, and the position resolution away from the focal plane is lower. This makes it impossible to ensure continuity and smoothness when performing the axial splice. In addition, the calculation speed in the light field reconstruction process is low, and the actual application efficiency is seriously affected. Zhi Lu et al in Nature Methods journal published paper "Physics-driven self-supervised learning for fast high-resolution robust 3D reconstruction of light-field microscopy"（Nat Methods 22, 1545-1555 (2025)）, propose a light field three-dimensional reconstruction method based on deep learning. According to the method, the time for reconstructing the light field is obviously shortened through the deep learning model, the reconstruction time of the traditional hour level is shortened to the second level, and the reconstruction speed is greatly improved. In paper "Computational optical sectioning with an incoherent multiscale scattering model for light-field microscopy"（Nat Commun 12, 6391 (2021)） published by Yi Zhang et al, journal Nature Communications, a computational optical reconstruction method based on a plurality of different axial light field images is presented. The method can obtain a larger axial length and realize a more complete three-dimensional reconstruction. The prior art only carries out three-dimensional reconstruction based on a single Zhang Guangchang image, and although light field reconstruction can be carried out, the effective continuous and smooth axial splicing cannot be realized due to the small depth of field and low resolution when the depth of field is far away from a focal plane. This limitation makes this approach less effective in certain application scenarios, especially in tasks requiring fine axial reconstruction. In the prior art, reconstruction is performed through a plurality of light field images with different axial positions, and a three-dimensional reconstruction body with a larger axial length can be obtained, but a traditional deconvolution method is still adopted. Moreover, the conventional RL deconvolution algorithm employed in the prior art has a long reconstruction time, typically requiring several hours for a set of axially scanned microscopic light field images. This results in a slower running speed of the method and computational efficiency that is difficult to meet when handling large-scale high-throughput reconstruction tasks. Therefore, it cannot be widely applied to actual scenes requiring quick and efficient processing. Disclosure of Invention The invention mainly aims to provide a high-speed reconstruction and axial splicing method of a microscopic light field based on a neural network. The invention further aims at providing a device for reconstructing a microscopic light field at a high speed and splicing the microscopic light field axially based on a neural network. A third object of the invention is to propose a computer device. A fourth object of the present invention is to propose a non-transitory computer readable storage medium. In order to achieve the above objective, an embodiment of a first aspect of the present invention provides a method for reconstructing a microscopic light field at a high speed and splicing the microscopic light field axially based on a neural network, including: shooting light field images with different depths, and acquiring light field data containing spatial position and depth information; Preprocessing the light field image sequentially through a distortion correction technology and a digital self-adaptive optical technology; digitally refocusing the preprocessed light field image to obtain a stacked three-dimensional body; the three-dimensional body formed by stacking is subjected to axial splicing, so that an original three-dimensional body is obtained; Inputting the original three-dimensional body into a three-dimensional convolutional neural network for optimization treatment, and outputting an axially spliced three-dimensional body; and saving the axially spliced three-dimensional body as a TIFF picture. In one embodiment of the present invention,