EP-4738250-A2 - SYSTEMS AND METHODS FOR ANATOMIC STRUCTURE SEGMENTATION IN IMAGE ANALYSIS

EP4738250A2EP 4738250 A2EP4738250 A2EP 4738250A2EP-4738250-A2

Abstract

Systems and methods are disclosed for anatomic structure segmentation in image analysis, using a computer system. One method includes: receiving image data of an anatomic structure of a patient, fitting a shape model to the anatomic structure, obtaining an estimation of a boundary of the anatomic structure and one or more keypoints at one or more known locations in the anatomic structure, wherein one or more of the estimation of the boundary or the one or more keypoints are determined based on the shape model fit to the anatomic structure, and using a trained machine-learning model, predicting a location of the boundary of the anatomic structure by generating a regression of distances from the one or more keypoints to the estimation of the boundary.

Inventors

GRADY, LEO
PETERSEN, Peter, Kersten
SCHAAP, MICHIEL
LESAGE, DAVID

Assignees

HeartFlow, Inc.

Dates

Publication Date: 20260506
Application Date: 20180509

Claims (15)

A computer-implemented method of machine-learning based anatomic structure segmentation in image analysis, comprising: receiving image data of an anatomic structure of a patient; fitting a shape model to the anatomic structure; obtaining an estimation of a boundary of the anatomic structure and one or more keypoints at one or more known locations in the anatomic structure, wherein one or more of the estimation of the boundary or the one or more keypoints are determined based on the shape model fit to the anatomic structure; and using a trained machine-learning model, predicting a location of the boundary of the anatomic structure by generating a regression of distances from the one or more keypoints to the estimation of the boundary.
The computer-implemented method of claim 1, wherein: the location of the boundary predicted via the trained machine-learning model includes a point-cloud representation of the boundary; and the computer-implemented method further comprises obtaining a surface of the anatomic structure using the point-cloud representation.
The computer-implemented method of claim 1, wherein: the image data is formed from pixels or voxels; and the location of the boundary predicted via the trained machine-learning model has a sub-pixel or sub-voxel accuracy.
The computer-implemented method of claim 1, wherein the image data includes a plurality of successive frames that are orthogonal to a centerline of the anatomic structure.
The computer-implemented method of claim 4, wherein predicting the location of the boundary of the anatomic structure includes generating a respective boundary portion for each frame of the plurality of successive frames.
The computer-implemented method of claim 1, wherein the shape model fitted to the anatomic structure is based on an annotation of the image data.
The computer-implemented method of claim 1, wherein the anatomic structure includes a blood vessel.
A system for machine-learning based anatomic structure segmentation in image analysis, comprising: at least one memory storing instructions and a trained machine-learning model; and at least one processor operatively connected to the at least one memory and configured to execute the instructions to perform operations, including: receiving image data of an anatomic structure of a patient; fitting a shape model to the anatomic structure; obtaining an estimation of a boundary of the anatomic structure and one or more keypoints at one or more known locations in the anatomic structure, wherein one or more of the estimation of the boundary or the one or more keypoints are determined based on the shape model fit to the anatomic structure; and using the trained machine-learning model, predicting a location of the boundary of the anatomic structure by generating a regression of distances from the one or more keypoints to the estimation of the boundary.
The system of claim 8, wherein: the location of the boundary predicted via the trained machine-learning model includes a point-cloud representation of the boundary; and the operations further include obtaining a surface of the anatomic structure using the point-cloud representation.
The system of claim 8, wherein: the image data is formed from pixels or voxels; and the location of the boundary predicted via the trained machine-learning model has a sub-pixel or sub-voxel accuracy.
The system of claim 8, wherein the image data includes a plurality of successive frames that are orthogonal to a centerline of the anatomic structure.
The system of claim 11, wherein predicting the location of the boundary of the anatomic structure includes generating a respective boundary portion for each frame of the plurality of successive frames.
The system of claim 8, wherein the shape model fitted to the anatomic structure is based on an annotation of the image data.
The system of claim 8, wherein the anatomic structure includes a blood vessel.
A non-transitory computer-readable medium comprising instructions for machine-learning based anatomic structure segmentation in image analysis, the instructions executable by one or more processors to perform operations of computer-implemented method according to claims 1 to 7.

Description

RELATED APPLICATION This application claims priority to U.S. Provisional Application No. 62/503,838, filed on May 9, 2017, the entire disclosure of which is hereby incorporated by reference in its entirety. TECHNICAL FIELD Various embodiments of the present disclosure relate generally to medical imaging and related methods. Specifically, particular embodiments of the present disclosure relate to systems and methods for anatomic structure segmentation in image analysis. BACKGROUND The problem of partitioning an image into multiple segments commonly occurs in computer vision and medical image analysis. A currently used approach is to automate this process using a convolutional neural network (CNN), which is trained to predict a class label for each image element (e.g., pixel or voxel). CNNs typically include multiple convolutional layers, which pass the input (e.g., an image or a portion of an image) through a set of learnable filters and nonlinear activation functions. The use of convolutional operations makes CNNs equivariant to translations. For example, translated versions of the input may lead to proportionally translated versions of the predicted segmentation labels. The set of layers with convolutions of different strides may enable CNNs to express long-range interactions in the image in terms of local, short range statistics. The segmentation boundary of current CNNs, however, may be accurate up to the level of an image element (e.g., a pixel or a voxel). In many imaging applications, a quantization error may be introduced by placing the segmentation boundary at pixel or voxel locations. In some cases, it may be known (e.g., as a priori) that a structure of interest does not contain holes and may exist as one connected component. However, these assumptions may not be integrated into the CNN such that the predicted labels may have spurious components and holes in the segmented objects. Thus, there is a desire to build models such as CNNs that can achieve sub-pixel or sub-voxel accurate segmentations and can predict labels for single connected components without holes or disconnected structures. The present disclosure is directed to overcoming one or more of the above-mentioned problems or interests. SUMMARY According to certain aspects of the present disclosure, systems and methods are disclosed for anatomic structure segmentation in image analysis. One method of anatomic structure segmentation in image analysis includes: receiving an annotation and a plurality of keypoints for an anatomic structure in one or more images; computing distances from the plurality of keypoints to a boundary of the anatomic structure; training a model, using data in the one or more images and the computed distances, for predicting a boundary in the anatomic structure in an image of a patient's anatomy; receiving the image of the patient's anatomy including the anatomic structure; estimating a segmentation boundary in the anatomic structure in the image of the patient's anatomy; and predicting, using the trained model, a boundary location in the anatomic structure in the image of the patient's anatomy by generating a regression of distances from keypoints in the anatomic structure in the image of the patient's anatomy to the estimated boundary. According to another embodiment, a system is disclosed for anatomic structure segmentation in image analysis. The system includes a data storage device storing instructions for anatomic structure segmentation in image analysis; and a processor configured to execute the instructions to perform a method including the steps of: receiving an annotation and a plurality of keypoints for an anatomic structure in one or more images; computing distances from the plurality of keypoints to a boundary of the anatomic structure; training a model, using data in the one or more images and the computed distances, for predicting a boundary in the anatomic structure in an image of a patient's anatomy; receiving the image of the patient's anatomy including the anatomic structure; estimating a segmentation boundary in the anatomic structure in the image of the patient's anatomy; and predicting, using the trained model, a boundary location in the anatomic structure in the image of the patient's anatomy by generating a regression of distances from keypoints in the anatomic structure in the image of the patient's anatomy to the estimated boundary. In accordance with yet another embodiment, a non-transitory computer readable medium for use on a computer system containing computer-executable programming instructions for performing a method of anatomic structure segmentation in image analysis is provided. The method includes: receiving an annotation and a plurality of keypoints for an anatomic structure in one or more images; computing distances from the plurality of keypoints to a boundary of the anatomic structure; training a model, using data in the one or more images and the computed distances, for predict