CA-3124400-C - SYSTEMS AND METHODS FOR GENERATING CANCER PREDICTION MAPS FROM MULTIPARAMETRIC MAGNETIC RESONANCE IMAGES USING DEEP LEARNING

Abstract

Various example embodiments are described in which an anisotropic encoder-decoder convolutional neural network architecture is employed to process multiparametric magnetic resonance images for the generation of cancer predication maps. In some example embodiments, a simplified anisotropic encoder-decoder convolutional neural network architecture may include an encoder portion that is deeper than a decoder portion. In some example embodiments, simplified network architectures may be combined with test-time-augmentation in order to facilitate training and testing with a minimal number of test subjects.

Inventors

• SHARON CLARKE
• ALESSANDRO GUIDA
• Peter Lee
• Chris Bowen
• David Hoar
• Steve Patterson

Assignees

• NOVA SCOTIA HEALTH AUTHORITY

Dates

Publication Date

20260505

Application Date

20191211

Priority Date

20181221

Claims (20)

1. 32 THEREFORE WHAT IS CLAIMED IS: 1. A method of implementing a convolutional neural network for generating cancer prediction maps based on processing of multiparametric magnetic resonance images, the method comprising: defining an anisotropic encoder-decoder convolutional neural network for processing multiparametric magnetic resonance images, wherein the anisotropic encoder-decoder convolutional neural network is configured such that an output thereof is a pixelated prediction image map, with each pixel of the pixelated prediction image map configured to provide a binary determination of a presence or absence of cancer; performing supervised transfer learning to pretrain the anisotropic encoder-decoder convolutional neural network; training the anisotropic encoder-decoder convolutional neural network with a plurality of multiparametric magnetic resonance training image sets and respective ground truth training image sets, each multiparametric magnetic resonance training image set corresponding to a given subject and comprising a plurality of multiparametric magnetic resonance image slices; and employing test-time-augmentation to test the anisotropic encoder-decoder convolutional neural network with a plurality of multiparametric magnetic resonance test image sets and respective ground truth test image sets; wherein test-time-augmentation is performed, when processing a given slice of a multiparametric magnetic resonance test image set, according to the steps of: CA 3124400 33 applying a plurality of transformations to the multiparametric magnetic resonance images corresponding to the given slice, thereby obtaining a plurality of transformed multiparametric magnetic resonance images; employing the anisotropic encoder-decoder convolutional neural network to generate, for each transformed multiparametric magnetic resonance image, an intermediate cancer prediction map; rectifying each intermediate cancer prediction map by applying a respective inverse transformation, thereby obtaining a plurality of rectified intermediate cancer prediction maps; and combining the plurality of rectified intermediate cancer prediction maps to generate a composite cancer prediction map.
2. 2. The method according to claim 1 further comprising deploying the anisotropic encoder-decoder convolutional neural network to process a multiparametric magnetic resonance image set from a subject and generate a plurality of cancer prediction maps for the subject.
3. 3. The method according to claim 1 or 2 wherein the encoder portion of the anisotropic encoder-decoder convolutional neural network includes a greater number of layers than the decoder portion of the anisotropic encoder-decoder convolutional neural network.
4. 4. The method according to any one of claims 1 to 3 wherein the anisotropic encoder-decoder convolutional neural network includes at least one short-cut 34 connection.
5. 5. The method according to any one of claims 1 to 4 wherein the encoder portion of the anisotropic encoder-decoder convolutional neural network includes three or fewer convolution blocks, each convolution block comprising two convolution layers and a maxpool layer.
6. 6. The method according to any one of claims 1 to 5 wherein the decoder portion of the anisotropic encoder-decoder convolutional neural network comprises three or fewer transposed convolution layers.
7. 7. The method according to any one of claims 1 to 6 wherein the composite cancer prediction map is generated with a 1x1 pixel mapping relationship to the multiparametric magnetic resonance images.
8. 8. The method according to any one of claims 1 to 7 wherein the multiparametric magnetic resonance training image sets and the multiparametric magnetic resonance test image sets are multiparametric magnetic resonance image sets of the prostate.
9. 9. The method according to any one of claims 1 to 7 wherein the multiparametric magnetic resonance training image sets and the multiparametric magnetic resonance test image sets are multiparametric magnetic resonance image sets of the pancreas.
10. 10. The method according to any one of claims 1 to 9 wherein the multiparametric magnetic resonance training image sets and the multiparametric magnetic resonance test image sets are obtained from less than 100 subjects.
11. 11. The method according to any one of claims 1 to 9 wherein the multiparametric magnetic resonance training image sets and the multiparametric magnetic resonance test image sets are obtained from less than 50 subjects.
12. 12. The method according to any one of claims 1 to 9 wherein the multiparametric magnetic resonance training image sets and the multiparametric magnetic resonance test image sets are obtained from less than 20 subjects.
13. 13. A method of employing an anisotropic encoder-decoder convolution neural network to generate cancer prediction maps based on processing of a multiparametric magnetic resonance image set associated with a subject, the anisotropic encoder-decoder convolution neural network being configured such that an output thereof is a pixelated prediction image map, with each pixel of the pixelated prediction image map configured to provide a binary determination of a presence or absence of cancer; the method comprising employing test-time-augmentation to process the multiparametric magnetic resonance image set via the anisotropic encoder-decoder convolutional neural network; CA 3124400 36 wherein test-time-augmentation is performed, when processing a given slice of the multiparametric magnetic resonance image set, according to the steps of: applying a plurality of transformations to the multiparametric magnetic resonance images corresponding to the given slice, thereby obtaining a plurality of transformed multiparametric magnetic resonance images; employing the anisotropic encoder-decoder convolutional neural network to generate, for each transformed multiparametric magnetic resonance image, an intermediate cancer prediction map; rectifying each intermediate cancer prediction map by applying a respective inverse transformation, thereby obtaining a plurality of rectified intermediate cancer prediction maps; and combining the plurality of rectified intermediate cancer prediction maps to generate a composite cancer prediction map.
14. 14. A system for processing multiparametric magnetic resonance images for cancer segmentation, the system comprising: processing circuitry comprising at least one processor and associated memory, wherein the memory stores instructions executable by the at least one processor for performing operations comprising: employing test-time-augmentation to process a multiparametric magnetic resonance image set via an anisotropic encoder-decoder convolutional neural network, the anisotropic encoder-decoder convolution neural network anisotropic encoder-decoder convolutional neural network 37 being configured such that an output thereof is a pixelated prediction image map, with each pixel of the pixelated prediction image map configured to provide a binary determination of a presence or absence of cancer; wherein test-time-augmentation is performed, when processing a given slice of the multiparametric magnetic resonance image set, according to the steps of: applying a plurality of transformations to the multiparametric magnetic resonance images corresponding to the given slice, thereby obtaining a plurality of transformed multiparametric magnetic resonance images; employing the anisotropic encoder-decoder convolutional neural network to generate, for each transformed multiparametric magnetic resonance image, an intermediate cancer prediction map; rectifying each intermediate cancer prediction map by applying a respective inverse transformation, thereby obtaining a plurality of rectified intermediate cancer prediction maps; and combining the plurality of rectified intermediate cancer prediction maps to generate a composite cancer prediction map.
15. 15. The system according to claim 14 wherein the processing circuitry is configured such that the encoder portion of the anisotropic encoder-decoder convolutional neural network includes a greater number of layers than the decoder portion of the anisotropic encoder-decoder convolutional neural network. CA 3124400 38
16. 16. The system according to any one of claims 14 or 15 wherein the processing circuitry is configured such that the anisotropic encoder-decoder convolutional neural network includes at least one short-cut connection.
17. 17. The system according to any one of claims 14 to 16 wherein the processing circuitry is configured such that the encoder portion of the anisotropic encoder-decoder convolutional neural network includes three or fewer convolution blocks, each convolution block comprising two convolution layers and a maxpool layer.
18. 18. The system according to any one of claims 14 to 17 wherein the processing circuitry is configured such that the decoder portion of the anisotropic encoder-decoder convolutional neural network comprises three or fewer transposed convolution layers.
19. 19. The system according to any one of claims 14 to 18 wherein the processing circuitry is configured such that the composite cancer prediction map is generated with a 1x1 pixel mapping relationship to the multiparametric magnetic resonance images.
20. 20. The system according to any one of claims 14 to 19 wherein the processing circuitry is configured such that the multiparametric magnetic resonance training image sets and the multiparametric magnetic resonance test image sets are multiparametric magnetic resonance image sets of the prostate. CA 3124400 39

Description

1 SYSTEMS AND METHODS FOR GENERATING CANCER PREDICTION MAPS FROM MULTIPARAMETRIC MAGNETIC RESONANCE IMAGES USING DEEP LEARNING CROSS-5 REFERENCE TO RELATED APPLICATION This application claims priority to U.S. Provisional Patent Application No. 62/783,734, titled “SYSTEMS AND METHODS FOR GENERATING CANCER PREDICTION MAPS FROM MULTIPARAMETRIC MAGNETIC RESONANCE IMAGES USING DEEP LEARNING” and filed on December 10 21, 2018. BACKGROUND The present disclosure relates to the detection and classification of cancer in medical images. More particularly, the present disclosure relates to 15 automated cancer segmentation from multiparametric MR images. Prostate cancer was the second most frequently diagnosed cancer in men and the fifth leading cause of cancer death worldwide in 2012 (Torre et al. 2015). The incidence in developed countries is on the rise and is associated with a significant socioeconomic burden (Roehrborn 20 and Black 2011; Sanyal et al. 2013). Evaluation of a patient suspected of having prostate cancer most commonly involves systematic random transrectal ultrasound-guided (TRUS) core biopsy. This approach, however, has several known limitations including failure to sample clinically significant cancer and under estimation of Gleason score. Due to 25 these disadvantages, there is increasing interest in prostate cancer detection and staging with multiparametric MRI. In 2014, the National Institute of Clinical Excellence guidelines for prostate cancer management were modified to include consideration of multiparametric magnetic resonance imaging (MRI) after a negative TRUS biopsy to determine if 5 another biopsy is needed, and in men with histologically proven cancer when changes in tumor (T) or nodal (N) stage would alter management. The Prostate Imaging Reporting and Data System (Pl-RADS) recommends that the multiparametric MRI examination consist of triplanar T2 weighted (T2w), diffusion weighted imaging (DWI) with 10 associated apparent diffusion coefficient (ADC) map, and dynamic contrast enhanced (DCE) sequences; the DCE series should image the prostate for 3-5 minutes post-contrast administration with a temporal resolution of <7 seconds per volume (Weinreb et al. 2016). Thus, the multiparametric MRI examination generates hundreds, if not thousands, of 15 images for radiologic review. Consequently, interpretation is time consuming and diagnostic accuracy is dependent upon the expertise of the reporting radiologist (Garcia-Reyes et al. 2015). Although Pl-RADS provides a standardized lexicon for interpreting and reporting multiparametric MRI, there remains considerable inter-observer variability 20 (Rosenkrantz et al. 2016; Muller et al. 2015). Machine learning has the potential to reliably and objectively integrate a large amount of MRI data to create a map of cancer probability. When used as a diagnostic aid by the radiologist, such an approach has been shown to increase efficiency and accuracy while 25 reducing inter-variability (Hambrock and MBCh 2013; Giannini et al. 2 2017). SUMMARY Various example embodiments are described in which an anisotropic encoder-decoder convolutional neural network architecture is employed to process multiparametric magnetic resonance images for the generation of cancer predication maps. In some example embodiments, a simplified anisotropic encoder-decoder convolutional neural network architecture may include an encoder portion that is deeper than a decoder portion. In some example embodiments, simplified network architectures may be combined with test-time-augmentation in order to facilitate training and testing with a minimal number of test subjects. Accordingly, in a first aspect, there is provided a method of implementing a convolutional neural network for generating cancer prediction maps based on processing of multiparametric magnetic resonance images, the method comprising: defining an anisotropic encoder-decoder convolutional neural network for processing multiparametric magnetic resonance images, wherein the anisotropic encoder-decoder convolutional neural network is configured such that an output thereof is a pixelated prediction image map, with each pixel of the pixelated prediction image map configured to provide a binary determination of a presence or absence of cancer; performing supervised transfer learning to pretrain the anisotropic encoder-decoder convolutional neural network; training the anisotropic encoder-decoder convolutional neural network 3 with a plurality of multiparametric magnetic resonance training image sets and respective ground truth training image sets, each multiparametric magnetic resonance training image set corresponding to a given subject and comprising a plurality of multiparametric magnetic resonance image slices; and employing test-time-augmentation to test the anisotropic encoderdecoder convolutional neural network with a plurality of multiparametric magnetic resonance test image sets and respective ground truth test image s