EP-4550255-B1 - IMAGE STITCHING METHOD, AND GENE SEQUENCING SYSTEM AND CORRESPONDING GENE SEQUENCER

EP4550255B1EP 4550255 B1EP4550255 B1EP 4550255B1EP-4550255-B1

Inventors

LIU, Huanlin
LI, MEI
HUANG, Zirui
CHEN, BICHAO
HONG, YAN
LI, YuXiang

Dates

Publication Date: 20260506
Application Date: 20220629

Claims (14)

A computer implemented method for image stitching, wherein the method comprises: obtaining multiple first images of a sample and template information about a track line or track cross of the sample (S101); the template information comprises index numbers of multiple track lines on the sample, the track lines are distributed horizontally and vertically in each of the multiple first images, and the intersection point of the track lines is the track cross; pre-stitching the multiple first images to obtaint pre-stitching coordinates of the multiple first images (S102); selecting at least one second image from the multiple first images based on a characteristic of track line or track cross, and determining the index numbers corresponding to each track line in the second image based on a distance between adjacent track lines in the second image, and deriving a global template based on the index numbers of the second image and pre-stitching coordinates of the second image, so as to derive the global template based on the template information of the sample and the pre-stitching coordinates of the second image (S103); and for a third image other than the second image among the multiple first images, calculating a offset between the pre-stitching coordinates of the third image and the corresponding template coordinates of the third image in the global template, and adjusting the coordinates of the third image based on the offset, so as to stitch and generate a stitched image about the sample (S104).
The computer implemented method for image stitching according to claim 1, characterized in that , said pre-stitching the multiple first images to obtain the pre-stitching coordinates of the multiple first images comprises (S102): scanning the multiple first images and obtaining the pre-stitching coordinates of the multiple first images based on the offset and overlap between adjacent first images obtained during the scanning process.
The computer implemented method for image stitching according to claim 2, characterized in that , the offset and overlap between adjacent first images obtained are obtained by scanning multiple first images using Fast Fourier Transform or Scale Invariant Feature Transform.
The computer implemented method for image stitching according to claim 1, characterized in that , the second image is the image with the most obvious track line features among the multiple first images, or the image with the most track crosses among the multiple first images.
The computer implemented method for image stitching according to claim 1, characterized in that , the offset of the third image is a vector formed from the track cross on the third image to the template point in the global template corresponding to the track cross on the third image.
The computer implemented method for image stitching according to claim 1, characterized in that said adjusting the coordinates of the third image comprises adjusting the coordinates of the third image containing track crosses in a manner of adding the pre-stitching coordinates of each third image containing track crosses to the respective unique offset for the third image containing track crosses; or said adjusting the coordinates of the third image further comprises performing seam fusion processing between adjacent third images.
The computer implemented method for image stitching according to claim 6, characterized in that , the unique offset is obtained in a manner of: for each third image containing the track crosses, obtaining the respective offset of each of track crosses in the third image, and selecting an offset with the median angle or median distance from the offsets as the unique offset of the third image; or, for each third image containing the track crosses, obtaining the respective offset of each of track crosses in the third image, sorting the offsets based on the vector length of the offsets, and selecting an offset at the preset sorting position as the unique offset of the third image.
The computer implemented method for image stitching according to claim 7, characterized in that , said adjusting the coordinates of the third image based on the offset further comprises using a nearest neighbor adjustment method to adjust the coordinates of third image without track crosses, wherein the nearest neighbor adjustment method comprises: for each third image without track crosses, adjusting the coordinates of the third image without track crosses based on the unique offset of a third image containing track crosses, as well as based on the offset and overlap between the third image without track crosses and the third image containing track crosses; wherein the third image containing track crosses is closest to the third image without track crosses.
A gene sequencing system, wherein the gene sequencing system comprises: an image acquisition module configured to obtain multiple first images of a sample and template information about a track line or track cross of the sample; the template information comprises index numbers of multiple track lines on the sample, the track lines are distributed horizontally and vertically in each of the multiple first images, and the intersection point of the track lines is the track cross; an image pre-stitching module configured to pre-stitch the multiple first images to obtain pre-stitching coordinates of the multiple first images; a global template derivation module configured to select at least one second image from the multiple first images based on a characteristic of track line or track cross, and determine the index numbers corresponding to each track line in the second image based on a distance between adjacent track lines in the second image, and derive a global template based on the index numbers of the second image and pre-stitch coordinates of the second image, so as to derive the global template based on the template information of the sample and the pre-stitching coordinates of the second image; and a stitching image generation module configured to, for a third image other than the second image among the multiple first images, calculate a offset between the pre-stitching coordinates of the third image and the corresponding template coordinates of the third image in the global template, and adjust the coordinates of the third image based on the offset, so as to stitch and generate a stitched image about the sample.
The gene sequencing system according to claim 9, characterized in that , the offset of the third image is a vector formed from the track cross on the third image to the template point in the global template corresponding to the track cross on the third image.
The gene sequencing system according to claim 9, characterized in that , the stitching image generation module is further configured to adjust the coordinates of the third image containing track crosses in a manner of adding the pre-stitching coordinates of each third image containing track crosses to the respective unique offset for the third image containing track crosses.
The gene sequencing system according to claim 11, characterized in that , the stitching image generation module is further configured to obtain the unique offset in a manner of: for each third image containing the track crosses, obtaining the respective offset of each of track crosses in the third image, and selecting an offset with the median angle or median distance from the offsets as the unique offset of the third image; or, for each third image containing the track crosses, obtaining the respective offset of each of track crosses in the third image, sorting the offsets based on the vector length of the offsets, and selecting an offset at the preset sorting position as the unique offset of the third image.
The gene sequencing system according to claim 9, characterized in that , the stitching image generation module is further configured to use a nearest neighbor adjustment method to adjust the coordinates of third image without track crosses, wherein the nearest neighbor adjustment method comprises: for each third image without track crosses, adjusting the coordinates of the third image without track crosses based on the unique offset of a third image containing track crosses, as well as based on the offset and overlap between the third image without track crosses and the third image containing track crosses; wherein the third image containing track crosses is closest to the third image without track crosses.
A computer device, wherein comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to implement the image stitching method according to any one of claims 1-8 when executing the computer program.

Description

FIELD The present disclosure relates to the technical field of image processing, and more particularly, to an image stitching method, corresponding gene sequencing system, gene sequencer, computer device, and computer storage medium. BACKGROUND Gene sequencing refers to the analysis of the base sequence of specific DNA fragments, namely the arrangement of adenine (A), thymine (T), cytosine (C), and guanine (G). The above four types of bases carry four different fluorophores respectively, which emit fluorescence of different wavelengths (colors) when excited. One of the commonly used sequencing methods is to identify the type of synthesized base by recognizing the fluorescence wavelength, thereby modifying the base sequence. The second-generation sequencing technology uses a high-resolution microscopy imaging system to capture fluorescent molecular images of DNA Nanoballs (i.e. DNBs) on the collected samples (such as gene sequencing chips), and input the fluorescent molecular images to base recognition software to decode the image signal and obtain the base sequence. When using a high-power microscope to observe the obtained base sequence, only a magnified image of the local area of the observed object can be obtained. In view of this, it is usually achieved by continuously taking multiple shots to collect the optical signal of the entire sample. After data collection is completed, it is also necessary to obtain complete and high-precision global sample images, which is of great significance for identifying and analyzing problems in the future. Existing image stitching schemes typically utilize pixel values or feature points between images to perform alignment, fusion, and other operations to complete the stitching process. Joe Chalfoun et al. published an article entitled "MIST: Accurate and Scalable Microscopy Image Stitching Tool with Stage Modeling and Error Minimization" in SCIENTIFIC REPORTS (vol. 7, no. 1, 1 December 2017, XP055849263, DOI: 10.1038/s41598-017-04567-y). The article disclosed that automated microscopy can image specimens larger than the microscope's field of view (FOV) by stitching overlapping image tiles and enables time-lapse studies of entire cell cultures in multiple imaging modalities. MIST (Microscopy Image Stitching Tool) was created for rapid and accurate stitching of large 2D time - lapse mosaics in the article. MIST estimates the mechanical stage model parameters (actuator backlash, and stage repeatability 'r') from computed pairwise translations and then minimizes stitching errors by optimizing the translations within a (4r)2 square area. MIST has a performance-oriented implementation utilizing multicore hybrid CPU/GPU computing resources, which can process terabytes of time-lapse multi-channel mosaics 15 to 100 times faster than existing tools. However, most existing stitching methods are based on feature similarity, which leads to a lack of measurement standards and makes it difficult to achieve high accuracy. Therefore, it is necessary to provide an improved image stitching method and corresponding gene sequencing system for stitching sample images. SUMMARY The image stitching method and corresponding gene sequencing system provided by the present invention are dedicated to solving at least one of the above-mentioned problems. The invention is defined by the independent claims 1 and 9. Embodiments result from the dependent claims and the below description. In particular, according to a first aspect of the present invention, a method for image stitching is provided. The method comprises: obtaining multiple first images of a sample and template information about the track line or track cross of the sample;pre-stitching the multiple first images to obtain the pre-stitching coordinates of the multiple first images;selecting at least one second image from the multiple first images based on the characteristic of track line or track cross, and deriving a global template based on the template information of the sample and the pre-stitching coordinates of the second image; andfor a third image other than the second image among the multiple first images, calculating the offset between the pre-stitching coordinates of the third image and the corresponding template coordinates of the third image in the global template, and adjusting the coordinates of the third image based on the offset, so as to stitch and generate a stitched image about the sample. According to a second aspect of the present invention, a gene sequencing system is provided. The gene sequencing system comprises: an image acquisition module configured to obtain multiple first images of a sample and template information about the track line or track cross of the sample;an image pre-stitching module configured to pre-stitch the multiple first images to obtain the pre-stitching coordinates of the multiple first images;a global template derivation module configured to select at least one second image from the multiple firs