US-20260127795-A1 - GENERALIZED PATCH-BASED INFERENCE FOR DENOISING DIFFUSION MODELS FOR PLUG-AND-PLAY MEDICAL IMAGE RESTORATION/RECONSTRUCTION

Abstract

Systems and methods for image reconstruction that uses patch-based processing of images during each evaluation for inverse problems, while remaining independent of specialized neural network architectures or specialized training of a diffusion prior. Embodiments use a grid sampling strategy to determine patches that includes a shifted-grid approach and a reflective padding approach in order to avoid artifacts in the resulting estimations.

Inventors

• Saikat Roy
• Mahmoud Mostapha
• Radu Miron
• Mariappan S. Nadar
• Matthew Holbrook

Assignees

• Siemens Healthineers Ag

Dates

Publication Date

20260507

Application Date

20250326

Priority Date

20241106

Claims (20)

1. 1 . A method for image reconstruction of medical imaging data, the method comprising: acquiring a medical image of a patient; iteratively refining the medical image using a diffusion PnP model comprising a plurality of iterations, wherein for each iteration of the plurality of iterations, predictions for the medical image occur on a set of patches sampled from a grid of the medical image; and outputting the refined medical image.
2. 2 . The method of claim 1 , wherein the patches are sampled using shifting-grid-based patch sampling to resolve grid artifacts.
3. 3 . The method of claim 1 , wherein reflection padding is used on the patches sampled from the grid to eliminate foreground-to-padded-background transitions.
4. 4 . The method of claim 1 , wherein the diffusion PnP model comprises DIFFPnP.
5. 5 . The method of claim 1 , wherein each of the patches of the set of patches are 128×128 pixels in size.
6. 6 . The method of claim 1 , wherein the medical image is acquires using magnetic resonance imaging (MRI), computed tomography (CT), photon counting CT (PCCT), ultra-high-resolution CT (UHR PCCT), or spectral CT.
7. 7 . The method of claim 1 , wherein the diffusion PnP model is trained using a dataset of medical images comprising MRI scans sourced from multiple anatomical regions including at least brain, knee, and prostate regions.
8. 8 . The method of claim 1 , wherein the diffusion PnP model uses measurement data for regularization.
9. 9 . The method of claim 1 , wherein the patches are sampled by: dividing the medical image into a plurality of patches, wherein each patch of the plurality of patches undergoes independent inference by leveraging a trained generalized diffusion prior, wherein multiple inference passes are conducted with systematically shifted patch grids, wherein reflection padding is used to mirrors pixel values at boundaries of the medical image.
10. 10 . A system for image reconstruction of medical imaging data, the system comprising: a medical imaging system configured to acquire medical imaging data; a diffusion PnP model configured to use patch-based sampling to reconstruct a medical image from the medical imaging data, wherein instead of predicting an entire denoised image, predictions occur on foreground patches sampled from a grid; and an interface configured to display the medical image.
11. 11 . The system of claim 10 , wherein the patch-based sampling uses shifting-grid-based patch sampling to resolve grid artifacts.
12. 12 . The system of claim 10 , wherein reflection padding is used on patches sampled from the grid to eliminate foreground-to-padded-background transitions.
13. 13 . The system of claim 10 , wherein the diffusion PnP model comprises DIFFPnP.
14. 14 . The system of claim 10 , wherein patches of the patch-based sampling are 128×128 pixels in size.
15. 15 . The system of claim 10 , wherein the medical imaging system comprises a magnetic resonance imaging system.
16. 16 . The system of claim 10 , wherein the patch-based sampling comprises a plurality of patches created by dividing the medical imaging data using the grid; wherein each patch of the plurality of patches undergoes independent inference by leveraging a trained generalized diffusion prior, wherein at least one additional inference passes are conducted with the grid shifted, wherein reflection padding is used to mirrors pixel values at boundaries of the medical image.
17. 17 . The system of claim 10 , wherein the diffusion PnP model is trained using a dataset of medical images comprising MRI scans sourced from multiple anatomical regions including at least brain, knee, and prostate regions.
18. 18 . A method for image reconstruction of a medical image, the method comprising: acquiring medical imaging data of a patient; iteratively refining the medical imaging data using a diffusion PnP model, wherein each iteration comprises: dividing the medical imaging data into a first set of patches using a first grid, wherein reflection padding mirrors pixel values at image boundaries of the medical image; performing inference on one or more patches from the first set using a trained generalized diffusion prior; dividing the medical imaging data into a second set of patches using a second grid, the second grid shifted from the first grid, wherein reflection padding mirrors pixel values at image boundaries of the medical image; performing inference on one or more patches from the second set using a trained generalized diffusion prior; reincorporating the medical imaging data from the one or more patches from the first set and the one or more patches from the second set for which inference was performed, the reincorporated medical imaging data uses to solve a data proximal subproblem for regularization; deriving a state for a next iteration by adding noise back; and outputting the refined medical imaging data.
19. 19 . The method of claim 18 , wherein the medical imaging data is acquired using magnetic resonance imaging (MRI).
20. 20 . The method of claim 19 , wherein the diffusion PnP model is trained using a dataset of medical images comprising MRI scans sourced from multiple anatomical regions including at least brain, knee, and prostate regions.

Description

CROSS REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. provisional application Ser. No. 63/716,749, filed Nov. 6, 2024, and European Patent Application EP24465590.8, filed Nov. 6, 2024, both of which are entirely incorporated by reference. FIELD This disclosure relates to medical imaging. BACKGROUND Magnetic resonance imaging, or MRI, is a noninvasive medical imaging test that can generate detailed images of almost every internal structure in the human body, including, for example organs, bones, muscles, and blood vessels. The process of transforming the acquired MRI data to images is called image reconstruction. Image reconstruction transforms the data into interpretable images using signal processing techniques to improve image quality and speed up scans. Deep learning-based approaches have been proposed that use neural networks to enhance image reconstruction, improving speed and accuracy. For example, plug-and-play approaches to solving inverse problems in complex MRI data have recently benefitted from Diffusion-based generative priors. In such a scheme, a diffusion model is used to model the prior distribution and may be used in a number of inverse tasks such as denoising or super-resolution without the need to train individual models for each task. This has led to exceptional performance of such diffusion based inverse solvers in CT or complex MRI data while retaining perceptual quality and reconstruction faithfulness. In the context diffusion models, Neural Function Evaluations (NFEs) refer to the number of times the underlying neural network, which parametrizes the system dynamics, is evaluated during the numerical integration of the ODE solver. Existing diffusion models process the entire image at once during each NFE, necessitating large amounts of GPU memory. This can quickly become infeasible in images with large resolutions. SUMMARY By way of introduction, the preferred embodiments described below include methods, systems, instructions, and/or computer readable media for generalized patch-based inference for denoising diffusion models for plug-and-play medical image restoration/reconstruction. In a first aspect, a method for image reconstruction of medical imaging data, the method comprising: acquiring a medical image of a patient; iteratively refining the medical image using a diffusion PnP model comprising a plurality of iterations, wherein for each iteration of the plurality of iterations, predictions for the medical image occur on a set of patches sampled from a grid of the medical image; and outputting the refined medical image, wherein the patches are sampled using shifting-grid-based patch sampling to resolve grid artifacts, wherein reflection padding is used on the patches sampled from the grid to eliminate foreground-to-padded-background transitions. In a second aspect, a system for image reconstruction of medical imaging data, the system comprising: a medical imaging system configured to acquire medical imaging data; a diffusion PnP model configured to use patch-based sampling to reconstruct a medical image from the medical imaging data, wherein instead of predicting an entire denoised image, predictions occur on foreground patches sampled from a grid; and an interface configured to display the medical image. In a third aspect, a method for image reconstruction of a medical image, the method comprising: acquiring medical imaging data of a patient; iteratively refining the medical imaging data using a diffusion PnP model, wherein each iteration comprises: dividing the medical imaging data into a first set of patches using a grid, wherein reflection padding mirrors pixel values at image boundaries of the medical image; performing inference on one or more patches from the first set using a trained generalized diffusion prior; dividing the medical imaging data into a second set of patches using a second grid, the second grid shifted from the first grid, wherein reflection padding mirrors pixel values at image boundaries of the medical image; performing inference on one or more patches from the second set using a trained generalized diffusion prior; reincorporating the medical imaging data from the patches from the first set and second set for which inference was performed, the reincorporated medical imaging data uses to solve a data proximal subproblem for regularization; deriving a state for a next iteration by adding noise back; and outputting the refined medical imaging data. Any one or more of the aspects described above may be used alone or in combination. These and other aspects, features and advantages will become apparent from the following detailed description of preferred embodiments, which is to be read in connection with the accompanying drawings. The present invention is defined by the following claims, and nothing in this section should be taken as a limitation on those claims. Further aspects and advantages of the invention are discussed below in c