CN-119831833-B - Microscope image stitching method and device

CN119831833BCN 119831833 BCN119831833 BCN 119831833BCN-119831833-B

Abstract

The invention relates to the technical field of image processing and discloses a method and a device for splicing microscope images, wherein the method comprises the steps of obtaining images to be spliced, wherein the images to be spliced are images obtained when a target sample is shot through a microscope, and the images to be spliced carry corresponding object stage position information when the images are shot; inputting the images to be spliced into a pre-constructed encoder for encoding to obtain one-to-one corresponding encoding results of the images to be spliced, and inputting the one-to-one corresponding encoding results of the images to be spliced and the object stage position information into a pre-constructed generator to generate image splicing results. The microscope image stitching method adopted by the invention can be used for effectively aiming at images with larger microscope image size and high feature similarity, can be used for quickly finishing image stitching, has higher precision, and effectively solves the problems of long calculation time and poor precision.

Inventors

YANG QI
ZHANG WEI
YAO JIA
Zhang Yueye
Huang Runhu
ZHOU LIANQUN
LI JINZE
ZHANG ZHIQI
LI CHUANYU
LI CHAO

Assignees

中国科学院苏州生物医学工程技术研究所

Dates

Publication Date: 20260505
Application Date: 20241223

Claims (6)

1. A method of stitching microscopic images, the method comprising: acquiring an image to be spliced, wherein the image to be spliced is an image obtained when a target sample is shot through a microscope, and the image to be spliced carries corresponding object stage position information when the image is shot; The method comprises the steps of converting an image to be spliced into an HSV format image, extracting a brightness component based on the HSV format image, carrying out multi-scale Gaussian convolution based on the brightness component to obtain an illumination component, correcting the illumination component by a preset function to obtain a corrected image to be spliced; Inputting the images to be spliced after the correction of the light source into a pre-constructed encoder to encode, and obtaining a one-to-one corresponding encoding result of the images to be spliced, wherein the pre-constructed encoder comprises a self-attention layer, a first residual error connection and normalization layer, a feedforward network layer, a second residual error connection and normalization layer which are sequentially connected; Inputting the coding results and the object stage position information which are in one-to-one correspondence with the images to be spliced into a pre-constructed generator to generate an image splicing result, wherein the pre-constructed generator comprises a mask self-attention layer, a third residual error connection and normalization layer, a decoding self-attention layer, a fourth residual error connection and normalization layer, a feedforward network layer and a fifth residual error connection and normalization layer which are sequentially connected.
2. The method of claim 1, wherein the predetermined function is: Wherein, the To correct the brightness value of the pixel, A pixel brightness value of the image to be spliced, Are all the coordinates of the pixels, and the pixel coordinates, , wherein, And the illumination component m of the image to be spliced is the brightness average value of the illumination component.
3. A microscope image stitching device, the device comprising: The acquisition module is used for acquiring an image to be spliced, wherein the image to be spliced is an image obtained when a target sample is shot through a microscope, and the image to be spliced carries corresponding object stage position information when the image is shot; The correction module is used for carrying out light source correction on the images to be spliced by adopting a light source correction algorithm, and comprises the steps of converting the images to be spliced into HSV format images, extracting brightness components based on the HSV format images, carrying out multi-scale Gaussian convolution based on the brightness components to obtain illumination components, and correcting the illumination components by adopting a preset function to obtain corrected images to be spliced; The coding module is used for inputting the images to be spliced after the correction of the light source into a pre-constructed coder to code so as to obtain a coding result corresponding to the images to be spliced one by one, wherein the pre-constructed coder comprises a self-attention layer, a first residual error connection and normalization layer, a feedforward network layer, a second residual error connection and normalization layer which are connected in sequence; The generation module is used for inputting the coding results and the object stage position information which are in one-to-one correspondence with the images to be spliced into a pre-constructed generator to generate an image splicing result, wherein the pre-constructed generator comprises a mask self-attention layer, a third residual error connection and normalization layer, a decoding self-attention layer, a fourth residual error connection and normalization layer, a feedforward network layer and a fifth residual error connection and normalization layer which are sequentially connected.
4. A computer device, comprising: the microscope image stitching method according to claim 1 or 2, comprising a memory and a processor, the memory and the processor being communicatively connected to each other, the memory having stored therein computer instructions, the processor executing the computer instructions to perform the microscope image stitching method.
5. A computer-readable storage medium storing computer instructions for causing the computer to perform the microscopic image stitching method according to claim 1 or 2.
6. A computer program product comprising computer instructions for causing a computer to perform the microscope image stitching method of claim 1 or 2.

Description

Microscope image stitching method and device Technical Field The invention relates to the technical field of image processing, in particular to a microscope image stitching method and device. Background The microscope is an optical instrument for amplifying objects which can not be directly observed by naked eyes, such as tiny objects, cell structures, microorganisms and the like, and is widely applied to the fields of biology, medicine, materials science, chemistry and the like. However, when analyzing the whole tissue on a microscopic scale, it is necessary to satisfy both the sharpness requirement and the field of view requirement. Although the microscope can meet the precision requirement, the visual field range is correspondingly reduced, so that the tissue to be analyzed needs to be scanned and photographed by using the microscope, and then a high-precision large-range tissue image is obtained by a stitching method. At present, two methods for splicing microscope images are mainly adopted, namely, characteristic-based splicing and phase-based splicing. The feature-based stitching is to extract corner features of two images, match the features, calculate an affine transformation matrix according to feature matching results, project the images to be stitched into a reference image coordinate system according to the transformation matrix, and finally finish image stitching through a fusion algorithm. However, the feature calculating method is high in time consumption, particularly for large-size images shot by a microscope, a large amount of memory and time are consumed for feature calculation, the similarity of cell morphology in tissue images is high, and the feature-based method has a false recognition risk for images with similar features, so that the splicing result is wrong. The phase-based stitching is to process and superimpose images to be stitched by using a fast Fourier transform, and then obtain stitched images by using an inverse Fourier transform. However, the splicing method is poor in accuracy. Disclosure of Invention In view of the above, the invention provides a method and a device for splicing microscope images, which are used for solving the problems of large calculated amount and low precision of the microscope image splicing method in the prior art. In a first aspect, the present invention provides a method for stitching microscopic images, the method comprising: Acquiring an image to be spliced, wherein the image to be spliced is an image obtained when a target sample is shot through a microscope, and the image to be spliced carries corresponding objective table position information when the image is shot; inputting the images to be spliced into a pre-constructed encoder for encoding, and obtaining encoding results of the images to be spliced in one-to-one correspondence; inputting the coding results corresponding to the images to be spliced one by one and the object stage position information into a pre-constructed generator to generate an image splicing result. The invention provides a microscope image splicing method based on an encoding structure, which extracts image characteristics through an encoder, wherein the step uses GPU (graphic processing Unit) acceleration, so that the processing speed can be effectively improved, stage displacement encoding information is added into the encoder or a generator, the calculated amount can be further reduced, the processing speed and the splicing precision can be improved, and finally, a spliced image is generated through the generator. The microscope image stitching method adopted by the invention can be used for effectively aiming at images with larger microscope image size and high feature similarity, can be used for quickly finishing image stitching, has higher precision, and effectively solves the problems of long calculation time and poor precision. In an alternative embodiment, after acquiring the image to be stitched, the method further includes: and carrying out light source correction on the images to be spliced by adopting a light source correction algorithm, wherein the images to be spliced after the light source correction are used for inputting into an encoder. In this embodiment, before encoding the image to be spliced, the gamma light source correction algorithm is further used to correct the light source non-uniformity of the image, so that the corrected image can effectively eliminate the non-uniformity of the light source, and the consistency of the brightness and contrast of the image is improved, thereby reducing errors caused by illumination difference. Therefore, the method is not only beneficial to enhancing the overall quality of the image, but also can obviously improve the accuracy of image matching in the splicing process and ensure the definition and accuracy of the final composite image. In an alternative embodiment, the light source correction algorithm is used to perform light source correction on the ima