EP-3455789-B1 - A METHOD TO COMBINE BRIGHTFIELD AND FLUORESCENT CHANNELS FOR CELL IMAGE SEGMENTATION AND MORPHOLOGICAL ANALYSIS USING IMAGES OBTAINED FROM IMAGING FLOW CYTOMETER (IFC)

EP3455789B1EP 3455789 B1EP3455789 B1EP 3455789B1EP-3455789-B1

Inventors

LI, ALAN
VAIDYANATHAN, Shobana

Dates

Publication Date: 20260506
Application Date: 20170605

Claims (15)

A computer-automated cell classification system (100) comprising: an imaging flow cytometer (105) acquiring a multispectral plurality of images (121) of a moving cell in a sample, the multispectral plurality of images being simultaneously acquired across multiple different imaging modes including a side scatter image, each of the multispectral plurality of images being spatially well aligned with each other; a classifier engine (107) executed by a computer processor (84) and coupled to the imaging flow cytometer (105) to receive the multispectral plurality of images, the classifier engine: (A) segmenting (125) one image of the multispectral plurality of images into primary cellular components, the one image providing morphology of the moving cell, the primary cellular components being formed of image subcomponents representing cell parts of the moving cell; (B) locating the cell parts in the one image by: (i) segmenting (127) the other images of the multispectral plurality of images, including the side scatter image, into secondary cellular components corresponding to the primary cellular components segmented from the one image, the secondary cellular components segmented from the other images serve as subcomponent masks of the image subcomponents of the one image, (ii) spatially correlating (128) the subcomponent masks to the one image, and using (129) the subcomponent masks as foreground object markers of the respective cell parts for the one image, and (iii) applying (130) a graph cut segmentation algorithm to the one image in a manner that generates image data of the one image having increased accuracy of location of the cell parts therein; andw (C) reprocessing (137) the one image with the subcomponent masks by using a multi-label graph cuts algorithm to position the subcomponent masks in the one image, the reprocessing further identifying cell morphology from the one image and thereby classifying cell type of the moving cell; and an output unit rendering indications of the identified cell morphology and/or cell type classification from the classifier engine for the moving cell.
The cell classification system of Claim 1 wherein the one image is a brightfield image and some of the other images are fluorescent images of different fluorescent channels.
The cell classification system of Claim 2 wherein the step (B) of locating cell parts in the brightfield image iteratively applies (i) through (iii) to each of the different fluorescent images.
The cell classification system of Claim 1 wherein the classifier engine classifies cell types including any one or more of: sperm cells having disorders, sickle cells, cancer cells, and other cells indicative of disease or health disorder.
The cell classification system of Claim 1 wherein the output unit is any of a computer display monitor, a diagnostic test indicator, or other digital rendering.
The cell classification system of Claim 1 wherein the classifier engine classifies defective sperm cells, and the output unit is a diagnostic indicator rendering indication of infertility, optionally wherein the classification engine classifies a defective sperm cell by automatically identifying one or more of: presence of cytoplasmic droplets, occurrence of Distal Midpiece Reflex (DMR), and shape of sperm head.
A computer-based diagnostic tool (100) for identifying cell morphology and classifying cell type, the tool comprising: an input assembly to receive a multispectral plurality of images (121) of a moving cell in a sample, the multispectral plurality of images being simultaneously acquired by an imaging flow cytometer (105) across multiple different imaging modes including a side scatter image, each of the multispectral plurality of images being spatially well aligned with each other; and a classifier engine (107) communicatively coupled to the input assembly to receive the multispectral plurality of images, the classifier engine being executed by a computer processor (84) and processing the multispectral plurality of images by: (A) segmenting (125) one image of the multispectral plurality of images into primary cellular components, the one image providing morphology of the moving cell, the primary cellular components being formed of image subcomponents representing cell parts of the moving cell; (B) locating the cell parts in the one image by: (i) segmenting (127) the other images of the multispectral plurality of images, including the side scatter image, into secondary cellular components corresponding to the primary components segmented from the one image, the secondary cellular components segmented from the other images serve as subcomponent masks of the image subcomponents of the one image, (ii) spatially correlating (128) the subcomponent masks to the one image, and using (129) the subcomponent masks as foreground object markers of the respective cell parts for the one image, and (iii) applying (130) a graph cut segmentation algorithm to the one image in a manner that generates image data of the one image having increased accuracy of location of the cell parts therein; and (C) reprocessing (137) the one image with the subcomponent masks by using a multi-label graph cuts algorithm to position the subcomponent masks in the one image, the reprocessing further identifying cell morphology from the one image and thereby classifying cell type of the moving cell, wherein the classifier engine further provides an output indication of the identified cell morphology and/or cell type classification for the moving cell.
The diagnostic tool of Claim 7 wherein the one image is a brightfield image and some of the other received images are fluorescent images of different fluorescent channels.
The diagnostic tool of Claim 8 wherein the step (B) of locating cell parts in the brightfield image iteratively applies (i) through (iii) to the different fluorescent images.
The diagnostic tool of Claim 7 wherein the classifier engine classifies defective sperm cells and provides an output indication of infertility, optionally wherein the classifier engine classifies a defective sperm cell by enabling automated identification of one or more of: presence of cytoplasmic droplets, occurrence of Distal Midpiece Reflex (DMR), and shape of sperm head.
A computer-implemented method of identifying cell morphology and classifying cell types, the method comprising: using an imaging flow cytometer (105) to acquire a multispectral plurality of images (121) of a moving cell in a sample, the multispectral plurality of images being simultaneously acquired across multiple different imaging modes including a side scatter image, each of the multispectral plurality of images being spatially well aligned with each other; with a digital processor (84) in communication with the imaging flow cytometer, responsively processing the multispectral plurality of images by: (A) segmenting (125) one image of the multispectral plurality of images into primary cellular components, the one image providing morphology of the moving cell, the primary components being formed of image subcomponents representing cell parts of the moving cell; (B) locating the cell parts in the one image by: (i) segmenting (127) the other images of the multispectral plurality of images, including the side scatter image, into secondary cellular components corresponding to the primary components segmented from the one image, the secondary cellular components segmented from the other images serve as subcomponent masks of the image subcomponents of the one image, (ii) spatially correlating (128) the subcomponent masks to the one image, and using (129) the subcomponent masks as foreground object markers of the respective cell parts for the one image, and (iii) applying (130) a graph cut segmentation algorithm to the one image in a manner that generates image data of the one image having increased accuracy of location of the cell parts therein; and (C) reprocessing (137) the one image with the subcomponent masks by using a multi-label graph cuts algorithm to position the subcomponent masks in the one image, the reprocessing further identifying cell morphology from the one image and thereby classifying cell type of the moving cell; and outputting an indication of the identified cell morphology and/or cell type classification for the moving cell.
The method of Claim 11 wherein the one image is a brightfield image and some of the other images are fluorescent images of different fluorescent channels, optionally wherein the step (B) of locating cell parts in the brightfield image iteratively applies (i) through (iii) to each of the different fluorescent images.
The method of Claim 11 wherein the reprocessing classifies cell types including one or more of: sperm cells having disorders, sickle cells, cancer cells, and other cells indicative of disease or health disorder.
The method of Claim 11 wherein the output indication is any of a diagnostic test result indicator or other digital rendering.
The method of Claim 11 wherein the responsively processing includes classifying defective sperm cells; and the outputting renders an indication of infertility, optionally wherein the classifying a defective sperm cell includes the digital processor automating identification of one or more of: presence of cytoplasmic droplets, occurrence of Distal Midpiece Reflex (DMR), and shape of sperm head.

Description

The present invention relates to a computer-automated cell classification system, a computer-based diagnostic tool for identifying cell morphology and classifying cell type, and a computer-implemented method of identifying cell morphology and classifying cell types, BACKGROUND Cytometry is the measurement of the characteristics of cells including cell size, cell count, cell morphology (shape and structure), cell cycle phase, DNA content, and the existence or absence of specific proteins on the cell surface or in the cytoplasm. Cytometry is used to characterize and count blood cells in common blood tests. In a similar fashion, cytometry is also used in cell biology research and in medical diagnostics to characterize cells in a wide range of applications associated with diseases such as cancer and other disorders. Cytometric devices include image cytometers which operate by statically imaging a large number of cells using optical microscopy. Prior to analysis, cells may be stained to enhance contrast or to detect specific molecules by labeling these with fluorochromes. The cells may be viewed within a hemocytometer to aid manual counting. The introduction of the digital camera has led to the automation of image cytometers including automated image cytometers for cell counting and automated image cytometers for sophisticated high-content screening systems. Another cytometric device is the flow cytometer. In a flow cytometer, cells are suspended in a stream of fluid and passed by an electronic detection apparatus. The cells are characterized optically or by the use of an electrical impedance method called the Coulter principle. To detect specific molecules when optically characterized, cells are in most cases stained with the same type of fluorochromes that are used by image cytometers. Flow cytometers generally provide less data than image cytometers, but have a significantly higher throughput. For example, in biotechnology, flow cytometry is a laser- or impedance-based, biophysical technology and is employed in cell counting, cell sorting, biomarker detection and protein engineering. The flow cytometer allows simultaneous multiparametric analysis of the physical and chemical characteristics of up to thousands of particles (cells) per second. Flow cytometry is routinely used in the diagnosis of health disorders, especially blood cancers. However flow cytometry has other applications in research, clinical practice and clinical trials. A common variation is to physically sort cell particles based on their properties, so as to purify cell populations of interest. Assessment of morphology is critical for identification of different cell types. It also plays a vital role in evaluating the health of the cell. However, accurately classifying complex morphologies such as sickle cells, diatoms, and spermatozoa can be a challenge due to the heterogeneous shapes within a cell. Current methodology involves (i) obtaining images from a microscope, (ii) then manually locating the subcellular components, and (iii) estimating the coordinates for spatial location of that cellular component and subcomponents in order to approximately map them back to a brightfield image. CA-A-2935473 discloses a medical image analysis method for identifying biomarker-positive tumor cells, the method comprising identifying a plurality of nuclei and positional information of said nuclei by analyzing the light intensities in a first digital image; identifying cell membranes which comprise the biomarker by analyzing the light intensities in a second digital image, the first and second digital images depicting the same area of a slide comprising multiple tumor cells having being stained with a first stain selectively staining nuclei and a second stain selectively staining a biomarker indicative of a tumor cell belonging to a particular cancer-subtype. Bradbury L et al, "A spectral k-means approach to bright-field cell image segmentation", 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology Society: (EMBC 2010) ; Buenos Aires, Argentina, 31 August-4 September 2010, IEEE, Piscataway, NJ, USA, pages 4748-4751, doi:10.1109/IEMBS.2010.5626380, ISBN:978-1-4244-4123-5, XP032108401 discloses a segmentation method which combines spectral and k-means clustering techniques to locate cells in bright-field images. Fluorescent images may be matched to bright-field images and an active contour method is used to locate the nucleus of the cell in the fluorescent image. US-A-2009/245598 discloses methods for quantifying the location, strength and percent of expressed target molecules or other biological markers in immunohistochemically stained biological samples. The samples may be automatically segmented, for example into subcellular compartments, from images of compartmental markers. Then, the distribution of a target molecule on each of these compartments is calculated that includes the percentage and strength of expression. SUMMARY