CN-115272789-B - Fluorescent image processing model training method and fluorescent image processing method

CN115272789BCN 115272789 BCN115272789 BCN 115272789BCN-115272789-B

Abstract

The invention provides a fluorescent image processing model training method and a fluorescent image processing method, comprising the steps of constructing an unsupervised neural network model, wherein the unsupervised neural network model comprises an image conversion module and an image recovery module; and setting training parameters, and training an unsupervised neural network model by taking a minimized loss function as a target to obtain a fluorescence image processing model. According to the fluorescence image axial resolution improvement method based on the unsupervised deep learning, which is disclosed by the invention, gold standard training data is not needed, complex physical modeling is not needed in an imaging process, the model performance and applicability are greatly improved, and various fluorescence imaging three-dimensional data can be effectively recovered.

Inventors

YUAN JING
GONG HUI
Ning Kefu
LU BOLIN
ZHANG XIAOYU

Assignees

华中科技大学苏州脑空间信息研究院

Dates

Publication Date: 20260505
Application Date: 20220713

Claims (7)

1. The fluorescence image processing model training method is characterized by comprising the following steps of: An unsupervised neural network model is built, the unsupervised neural network model comprises an image conversion module and an image recovery module, the image conversion module is used for unpaired image learning real degradation process, and the image recovery module is used for image learning resolution improvement process; the image conversion module is an image conversion module based on a generation countermeasure network, comprises two generation networks G A 、G B with the same structure and two discrimination networks D A 、D B ,G A with the same structure, wherein the generation networks G A 、G B and the two discrimination networks D A 、D B ,G A are responsible for converting an image block Y i in transverse low resolution data Y into an image block Z i in axial low resolution data Z, the G B is responsible for converting the image block Z i in the axial low resolution data Z into an image block Y i in transverse low resolution data Y, the discriminator D A is responsible for distinguishing a real image block Z i from a network generated image block G A (y i ), the discriminator D B is responsible for distinguishing the real image block Y3995 from the network generated image block G B (z i , the generation networks are structured as a residual neural network with residual connection or a U-shaped network with a downsampling path and an upsampling path, the discrimination networks are structured as a convolutional neural network formed by a plurality of convolutional layers in series, the image recovery module H based on the convolutional neural network is structured as a residual neural network with residual connection or a U-shaped network with a downsampling path and a upsampling path, the image recovery module X i in the axial low resolution data Z is responsible for recovering the image block X2, the training effect of the image restoration module forms feedback to the image conversion module, so that feedback loss is introduced, and the generation network G A 、G B can learn the mapping between the transverse low-resolution data Y and the axial low-resolution data Z better; The method comprises the steps of obtaining a three-dimensional fluorescent image of a sample, and manufacturing a non-paired training data set, wherein the obtained three-dimensional fluorescent image is sliced along the transverse direction and the axial direction respectively to obtain a transverse tangential plane and an axial tangential plane of the three-dimensional fluorescent image, namely transverse high-resolution data X; Splitting the transverse high-resolution data X, the transverse low-resolution data Y and the axial low-resolution data Z to form a series of unpaired image blocks with the same size; and setting training parameters, training the unsupervised neural network model by using the unpaired training data set with the minimum loss function as a target, and obtaining a fluorescence image processing model.
2. Training method according to claim 1, characterized in that the image block pixels in the lateral high resolution data X and the lateral low resolution data Y are in one-to-one correspondence, the image block pixels in the lateral high resolution data X and the axial low resolution data Z are not in correspondence, and the image block pixels in the lateral low resolution data Y and the axial low resolution data Z are not in correspondence.
3. The training method according to claim 2, wherein the image conversion module is optimized and then the image restoration module is optimized in each iterative optimization cycle during training, and the image conversion module is trained in a loop-consistent manner.
4. A training method as claimed in claim 3, characterized in that the loss function L comprises a cyclic coincidence loss L cycle , a generation counterloss L gan , a content loss L con , and a feedback loss L feed , namely: Where λ, ρ and σ represent weights occupied by the generation of the countermeasure loss L gan , the content loss L con , and the feedback loss L feed , respectively, λε (0, 1), ρε (0, 10) and σε (0, 1).
5. A fluorescence image processing method, characterized by comprising: acquiring a three-dimensional fluorescence image to be processed; Slicing the three-dimensional fluorescence image to be processed, and then processing the three-dimensional fluorescence image by using the fluorescence image processing model according to any one of claims 1-4 to obtain the three-dimensional fluorescence image with improved axial resolution.
6. The method according to claim 5, wherein slicing the three-dimensional fluoroscopic image to be processed comprises: And re-slicing the three-dimensional fluorescence image to be processed along the x direction and the y direction to obtain axial sections in the xz direction and the yz direction.
7. The method for processing a fluorescence image according to claim 6, wherein the processing using the fluorescence image processing model to obtain a three-dimensional fluorescence image with improved axial resolution comprises: And sequentially inputting the axial tangential planes in the xz direction and the yz direction into an image recovery module in the fluorescent image processing model to obtain xz and yz tangential planes with improved axial resolution, re-slicing the tangential planes in the xz direction and the yz direction back to the original xy direction, and averaging to finally obtain the three-dimensional fluorescent image with improved axial resolution.

Description

Fluorescent image processing model training method and fluorescent image processing method Technical Field The invention relates to the technical field of image processing, in particular to a fluorescent image processing model training method and a fluorescent image processing method. Background Most optical microscopes have anisotropic three-dimensional resolution, with axial resolution typically 2-3 times worse than lateral resolution. Anisotropic optical resolution can lead to blurring artifacts in the axial direction when reconstructing three-dimensional objects, affecting the accurate analysis and measurement of their three-dimensional structure. In order to improve the axial resolution of three-dimensional imaging, many methods in the early days were improved from the hardware, such as multi-view light sheet microscope, confocal 4Pi microscope, etc., but these methods rely on carefully calibrated light paths and complex and expensive devices, resulting in low applicability. With the development of computer vision and deep learning, there are some methods that attempt to achieve an improvement in axial resolution using neural networks. This presents difficulties for training of supervised networks because the lateral and axial data of the three-dimensional image are not naturally corresponding. Therefore they adopt a method of synthesizing data by physically modeling the imaging process to reduce the lateral data to be close to the axial data, then train the neural network with the synthesized paired data, and finally apply the network to the real axial data. The disadvantage of this approach is that the performance of the model is very dependent on the performance of the modeling, and the modeling process is only a simulation of the real scene, which inevitably deviates from the real situation, thus resulting in limited application performance of the trained model in the real scene. Disclosure of Invention In order to overcome the technical defects, the invention provides the fluorescent image processing model training method and the fluorescent image processing method, and the method does not need gold standard training data or complex physical modeling of an imaging process, so that the model performance and applicability are greatly improved, and various types of three-dimensional data of fluorescent imaging can be effectively recovered. In order to achieve the above purpose, the invention discloses a fluorescence image processing model training method, which specifically comprises the following steps: An unsupervised neural network model is built, the unsupervised neural network model comprises an image conversion module and an image recovery module, the image conversion module is used for unpaired image learning real degradation process, and the image recovery module is used for image learning resolution improvement process; acquiring a three-dimensional fluorescence image of a sample, and manufacturing a non-paired training data set; And setting training parameters, and training an unsupervised neural network model by taking a minimized loss function as a target to obtain a fluorescence image processing model. In the technical scheme, the transverse data and the axial data of the three-dimensional fluorescent image are natural and non-corresponding, and the non-paired training data set is trained by adopting the non-supervision network model, wherein the image conversion module can learn the real degradation process from the transverse data to the axial data in a non-paired manner, and then the image recovery module is utilized to learn the resolution improvement process of the image, so that the non-supervision neural network model can directly learn the distribution characteristics of the real data without modeling and estimating the imaging process, the problem of model performance reduction caused by modeling errors is avoided, and the obtained fluorescent image processing model can be suitable for processing fluorescent images which can not form the paired training set. The invention also discloses a fluorescence image processing method, which comprises the following steps: acquiring a three-dimensional fluorescence image to be processed; slicing the three-dimensional fluorescence image to be processed, and then processing by using the fluorescence image processing model to obtain the three-dimensional fluorescence image with improved axial resolution. The method is realized based on the fluorescence image processing model, can realize the axial resolution improvement of the three-dimensional data of the fluorescence image from the algorithm level, overcomes the restriction of hardware conditions, and has better performance and applicability than the existing neural network method. Drawings FIG. 1 is a flow chart of a fluorescence image processing model training method. Fig. 2 is a training flow diagram of an unsupervised neural network model. FIG. 3 is a flow chart of a fluorescence image