US-12626365-B2 - Systems and methods to process electronic images to selectively hide structures and artifacts for digital pathology image review

US12626365B2US 12626365 B2US12626365 B2US 12626365B2US-12626365-B2

Abstract

A computer-implemented method for processing digital pathology images, the method including receiving a plurality of digital pathology images of at least one pathology specimen, the pathology specimen being associated with a patient. The method may further include determining, using a machine learning system, whether artifacts or objects of interest are present on the digital pathology images. Once the machine learning system has determined that an artifact or object of interest is present, the system may determine one or more regions on the digital pathology images that contain artifacts or objects of interest. Once the system determines the regions on the digital pathology images that contain artifacts or objects of interest, the system may use a machine learning system to inpaint or suppress the region and output the digital pathology images with the artifacts or objects of interest inpainted or suppressed.

Inventors

Navid Alemi
Christopher Kanan

Assignees

PAIGE.AI, Inc.

Dates

Publication Date: 20260512
Application Date: 20220923

Claims (17)

1 . A computer-implemented method for processing digital pathology images, comprising: receiving a plurality of digital pathology images of at least one pathology specimen, the pathology specimen being associated with a patient; determining, using a first machine learning system, whether artifacts are present on the digital pathology images, the first machine learning system using artifact-agnostic learning techniques; upon determining that an artifact is present, determining one or more regions on the digital pathology images that contain artifacts; upon determining the one or more regions on the digital pathology images that contain artifacts, using a second machine learning system to inpaint or suppress the one or more regions; and outputting the digital pathology images with the artifacts inpatined or suppressed.
2 . The method of claim 1 , wherein the artifact-agnostic learning techniques include applying classification-based learning techniques.
3 . The method of claim 2 , wherein applying the classification-based learning techniques to determine whether artifacts are presents further includes: assigning a score to each patch within the plurality of digital pathology images; and determining that the assigned scores are above a threshold value to classify a respective patch as including artifacts.
4 . The method of claim 1 , wherein the artifact-agnostic learning techniques include applying segmentation-based learning techniques.
5 . The computer-implemented method of claim 1 , wherein the artifacts includes writing utensil marking, hair, blur, scanlines, and/or bubbles displayed on the plurality of digital pathology images.
6 . The method of claim 1 , further comprising: determining a segmentation map of the one or more regions on the digital pathology images that contain artifacts.
7 . A system for processing electronic medical images, the system comprising: at least one memory storing instructions; and at least one processor configured to execute the instructions to perform operations comprising: receiving a plurality of digital pathology images of at least one pathology specimen, the pathology specimen being associated with a patient; determining, using a first machine learning system, whether artifacts are present on the digital pathology images, the first machine learning system using artifact-agnostic learning techniques; upon determining that an artifact is present, determining one or more regions on the digital pathology images that contain artifacts; upon determining the one or more regions on the digital pathology images that contain artifacts, using a second machine learning system to inpaint or suppress the one or more regions; and outputting the digital pathology images with the artifacts inpatined or suppressed.
8 . The system of claim 7 , wherein the artifact-agnostic learning techniques include applying classification-based learning techniques.
9 . The system of claim 8 , wherein applying the classification-based learning techniques to determine whether artifacts are presents further includes: assigning a score to each patch within the plurality of digital pathology images; and determining that the assigned scores are above a threshold value to classify a respective patch as including artifacts.
10 . The system of claim 7 , wherein the artifact-agnostic learning techniques include applying segmentation-based learning techniques.
11 . The system of claim 7 , wherein the artifacts includes writing utensil marking, hair, blur, scanlines, and/or bubbles displayed on the plurality of digital pathology images.
12 . The system of claim 7 , further comprising: determining a segmentation map of the one or more regions on the digital pathology images that contain artifacts.
13 . A system for processing electronic medical images, the system comprising at least one memory storing instructions; and at least one processor configured to execute the instructions to perform operations comprising: receiving a plurality of digital pathology images of at least one pathology specimen, the pathology specimen being associated with a patient; determining, using a first machine learning system, whether artifacts are present on the digital pathology images, the first machine learning system using artifact-specific learning techniques; receiving, from one or more users, a first artifact type to search for and remove; upon determining that the first artifact type is present, determining one or more regions on the digital pathology images that contain the first artifact type; upon determining the one or more regions on the digital pathology images that contain the first artifact type, using a second machine learning system to inpaint or suppress the one or more regions; and outputting the digital pathology images with the first artifact type inpatined or suppressed.
14 . The system of claim 13 , wherein the artifacts includes writing utensil marking, hair, blur, scanlines, and/or bubbles displayed on the plurality of digital pathology images.
15 . The system of claim 13 , wherein the first machine learning system using the artifact-specific learning techniques applies learning techniques based on a shape of one or more artifacts.
16 . The system of claim 13 , wherein the first machine learning system using the artifact-specific learning techniques applies learning techniques based on an appearance of one or more artifacts.
17 . The system of claim 13 , wherein the first artifact type is blur.

Description

RELATED APPLICATION(S) This application claims priority to U.S. Provisional Application No. 63/261,706 filed Sep. 27, 2021, the entire disclosure of which is hereby incorporated herein by reference in its entirety. FIELD OF THE DISCLOSURE Various embodiments of the present disclosure pertain generally to image processing methods. More specifically, particular embodiments of the present disclosure relate to systems and methods to selectively hide artifacts during digital review. BACKGROUND In human and animal pathology, visual examination of tissue under a microscope may be vital to diagnostic medicine, e.g., to diagnose cancer or in drug development (such as in assessing toxicity). With current pathology techniques, tissue samples may undergo multiple preparation steps so that different tissue structures may be differentiated visually by the human eye. These steps may consist of: (i) preserving the tissue using fixation; (ii) embedding the tissue in a paraffin block; (iii) cutting the paraffin block into thin sections (e.g., 3-5 micrometers or μm); (iv) mounting the sections on glass slides; and (v) staining mounted tissue sections to highlight important components or structures. With the use of stains and dyes, histology allows pathologists to visualize tissue structures and/or tissues, chemical elements within cells, and even microorganisms. However, some structures (e.g., hair, ink, bubbles, etc.) on a slide and/or appearing in an image of the slide may interfere with a visualization experience. The background description provided herein is for the purpose of generally presenting the context of the disclosure. Unless otherwise indicated herein, the materials described in this section are not prior art to the claims in this application and are not admitted to be prior art, or suggestions of the prior art, by inclusion in this section. SUMMARY According to certain aspects of the present disclosure, systems and methods are disclosed for processing electronic medical images, comprising: receiving a plurality of digital pathology images of at least one pathology specimen, the pathology specimen being associated with a patient; determining, using a machine learning system, whether artifacts or objects of interest are present on the digital pathology images; upon determining that an artifact or object of interest is present, determining one or more regions on the digital pathology images that contain artifacts or objects of interest; upon determining the regions on the digital pathology images that contain artifacts or objects of interest, using a machine learning system to inpaint or suppress the region; and outputting the digital pathology images with the artifacts or objects of interest inpainted or suppressed. A system for processing electronic medical images, the system including: at least one memory storing instructions; and at least one processor configured to execute the instructions to perform operations including: receiving a plurality of digital pathology images of at least one pathology specimen, the pathology specimen being associated with a patient; determining, using a machine learning system, whether artifacts or objects of interest are present on the digital pathology images; upon determining that an artifact or object of interest is present, determining one or more regions on the digital pathology images that contain artifacts or objects of interest; upon determining the regions on the digital pathology images that contain artifacts or objects of interest, using a machine learning system to inpaint or suppress the region; and outputting the digital pathology images with the artifacts or objects of interest inpainted or suppressed. A non-transitory computer-readable medium storing instructions that, when executed by a processor, perform operations processing electronic medical images, the operations including: receiving a plurality of digital pathology images of at least one pathology specimen, the pathology specimen being associated with a patient; determining, using a machine learning system, whether artifacts or objects of interest are present on the digital pathology images; upon determining that an artifact or object of interest is present, determining one or more regions on the digital pathology images that contain artifacts or objects of interest; upon determining the regions on the digital pathology images that contain artifacts or objects of interest, using a machine learning system to inpaint or suppress the region; and outputting the digital pathology images with the artifacts or objects of interest inpainted or suppressed. BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate various exemplary embodiments and together with the description, serve to explain the principles of the disclosed embodiments. FIG. 1A illustrates an exemplary block diagram of a system and network for processing images, fo