Search

EP-4177841-B1 - PSEUDO-DATA GENERATION APPARATUS, PSEUDO-DATA GENERATION METHOD, LEARNING APPARATUS AND LEARNING METHOD

EP4177841B1EP 4177841 B1EP4177841 B1EP 4177841B1EP-4177841-B1

Inventors

  • TAKESHIMA, HIDENORI

Dates

Publication Date
20260506
Application Date
20221104

Claims (15)

  1. A pseudo-data generation apparatus (1) comprising processing circuitry (2) configured to: acquire (21) one or more pieces of partial observation image data that form part of whole observation image data, the pseudo-data generation apparatus (1) being characterized in that the processing circuitry (2) is further configured to: generate (22) pseudo-whole observation image data by inputting the one or more pieces of partial observation image data to a function, the pseudo-whole observation image data being pseudo-data of the whole observation image data, wherein the function is optimized by training so that partial observation image data for training and pseudo-partial observation image data for training resemble each other, the pseudo-partial observation image data for training being obtained by converting the pseudo-whole observation image data for training, wherein the pseudo-whole observation image data for training is generated using the function, from the partial observation image data for training.
  2. The pseudo-data generation apparatus (1) according to claim 1, wherein the function is a generator (G) trained using a conditional generative adversarial network or a decoder trained using a conditional variational auto encoder or a model trained using a conditional diffusion model.
  3. The pseudo-data generation apparatus (1) according to claim 1, wherein the processing circuitry (2) is configured to: acquire (21) as multiple pieces of partial observation image data, a plurality of images captured while shifting a focus, and generate (22) three-dimensional volume data as the pseudo-whole observation image data from the multiple pieces of partial observation image data, the three-dimensional volume data including depth information.
  4. The pseudo-data generation apparatus (1) according to claim 1, wherein the processing circuitry is configured to: convert the pseudo-whole observation data into pseudo-first partial observation data for training that is pseudo-data of first partial observation data among multiple pieces of partial observation data; discriminate the pseudo-first partial observation data for training based on the pseudo-first partial observation data for training, the first partial observation data, and other partial observation data among the multiple pieces of partial observation data by using the function; and optimize a parameter of the function so that the pseudo-first partial observation data for training resembles the first partial observation data.
  5. The pseudo-data generation apparatus (1) according to claim 4, wherein the multiple pieces of partial observation data include the first partial observation data and second partial observation data differing from the first partial observation data, the processing circuitry being configured to: convert the pseudo-whole observation data into pseudo-second partial observation data for training, discriminate the pseudo-first partial observation data for training based on the pseudo-first partial observation data for training, the first partial observation data, and the second partial observation data, discriminate the pseudo-second partial observation data for training based on the pseudo-second partial observation data for training, the second partial observation data, and the first partial observation data, and optimize the parameter so that the pseudo-first partial observation data for training resembles the first partial observation data and the pseudo-second partial observation data for training resembles the second partial observation data.
  6. The pseudo-data generation apparatus (1) according to claim 4, wherein the processing circuitry is configured to: convert the pseudo-whole observation data into pseudo-missing partial data if there is missing partial observation data among the partial observation data that likely constitute the whole observation data, the pseudo-missing partial data being pseudo-data corresponding to the missing partial observation data, and optimize the parameter by using the acquired partial observation data and the pseudo-missing partial data.
  7. The pseudo-data generation apparatus (1) according to claim 4, wherein the processing circuitry is configured to optimize the parameter so that regularization is performed on the pseudo-whole observation data.
  8. The pseudo-data generation apparatus (1) according to claim 7, wherein the processing circuitry is configured to perform regularization by minimizing L1 norm or L2 norm concerning the pseudo-whole observation data.
  9. The pseudo-data generation apparatus (1) according to claim 7, wherein the processing circuitry is configured to implement regularization by adding a penalty term as a loss function at a time of training, the penalty term decreasing in value as the pseudo-whole observation data is more symmetrical in a horizontal direction and/or in a vertical direction.
  10. The pseudo-data generation apparatus (1) according to claim 4, wherein the pieces of partial observation data include a three-dimensional magnetic resonance (MR) image before contrast imaging, a three-dimensional MR image after contrast imaging, and a moving image of an imaging target site, and the whole observation data is a three-dimensional contrast moving image.
  11. The pseudo-data generation apparatus (1) according to claim 4, wherein the pieces of partial observation data include a magnetic resonance (MR) image or an MR moving image, and a one-dimensional MR spectroscopy (MRS), and the whole observation data is a two-dimensional chemical shift imaging (CSI) image if the MR image is acquired, and the whole observation data is a three-dimensional CSI image if the MR moving image is acquired.
  12. The pseudo-data generation apparatus (1) according to claim 4, wherein the pieces of partial observation data include a computed tomography (CT) image for each energy distribution, and the whole observation data is a material discrimination image relating to a plurality of materials.
  13. The pseudo-data generation apparatus (1) according to claim 4, wherein the pieces of partial observation data include a first resolution image and a second resolution image having a resolution higher than a resolution of the first resolution image, and the whole observation data is a super-resolved image.
  14. A computer-implemented pseudo-data generation method, comprising: acquiring (S201) one or more pieces of partial observation image data that form part of whole observation image data, the pseudo-data generation method being characterized in that the method further comprises: generating (S202) pseudo-whole observation image data by inputting the one or more pieces of partial observation image data to a function, the pseudo-whole observation image data being pseudo-data of the whole observation image data, wherein the function is optimized by training so that partial observation image data for training and pseudo-partial observation image data for training resemble each other, the pseudo-partial observation image data for training being obtained by converting the pseudo-whole observation image data for training, wherein the pseudo-whole observation image data for training is generated using the function, from the partial observation image data for training.
  15. A pseudo-data generation program, causing a computer to perform: acquiring one or more pieces of partial observation image data that form part of whole observation image data, the pseudo-data generation program being characterized by further causing a computer to perform: generating pseudo-whole observation image data by inputting the one or more pieces of partial observation image data to a function, the pseudo-whole observation image data being pseudo-data of the whole observation image data, wherein the function is optimized by training so that partial observation image data for training and pseudo-partial observation image data for training resemble each other, the pseudo-partial observation image data for training being obtained by converting the pseudo-whole observation image data for training, wherein the pseudo-whole observation image data for training is generated using the function, from the partial observation image data for training.

Description

FIELD Embodiments described herein relate generally to a pseudo-data generation apparatus, a pseudo-data generation method, a learning apparatus and a learning method. BACKGROUND Machine learning such as a deep neural network has been applied to many fields, and various approaches have also been applied to the medical field. Machine learning presupposes the use of large amounts of data for training, and thus faces the problem of expected performance not being achieved without sufficient data yields. In the medical field especially, privacy protection rules and the like render it difficult to collect large amounts of various types of medical data, including a medical image. Also, in the medical field, for example, some medical data are either hard to obtain or impossible to physically obtain, such as a three-dimensional volume magnetic resonance (MR) moving image (cine image). If such medical data can be utilized for correct data and the like, machine learning can be applied more widely, for which reason there is a need for utilization of such medical data. Isola Phillip et al: "Image-to-Image Translation with Conditional Adversarial Networks", 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society, US 21 July 2017, pages 5967-5976 relates to the use of conditional adversarial networks for image-to-image translation. The networks not only learn a mapping from the input image to the output image, but also learn a loss function. SUMMARY In relation to foregoing embodiments, the following matters are disclosed as one aspect and selected features of the present invention. In a first aspect, a pseudo-data generation apparatus is provided as recited in claim 1. In an embodiment according to claim 2, the function may be a generator trained using a conditional generative adversarial network or a decoder trained using a conditional variational auto encoder or a model trained using a conditional diffusion model. In an embodiment according to claim 3, the processing circuitry may be configured to acquire as multiple pieces of partial observation data, a plurality of images captured while shifting a focus. The processing circuitry may generate three-dimensional volume data as the pseudo-whole observation data from the multiple pieces of partial observation data, the three-dimensional volume data including depth information. Further embodiments of the pseudo-data generation apparatus are defined in claims 4-13. In a second aspect, a pseudo-data generation method is provided as recited in claim 14. In a third aspect, a pseudo-data generation program is provided as recited in claim 15. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing a learning apparatus and a pseudo-data generation apparatus to a first embodiment.FIG. 2 is a flowchart illustrating a training process of a learning apparatus according to the first embodiment.FIG. 3 is a conceptual diagram illustrating a generation process according to the first embodiment.FIG. 4 is a conceptual diagram illustrating a discrimination process according to the first embodiment.FIG. 5 is a flowchart illustrating a training process of a learning apparatus according to a second embodiment.FIG. 6 is a conceptual diagram illustrating a generation process according to the second embodiment.FIG. 7 is a conceptual diagram illustrating a discrimination process according to the second embodiment.FIG. 8 is a conceptual diagram illustrating a modification of the generation process according to the second embodiment.FIG. 9 is a diagram showing a generation example of partial observation data. DETAILED DESCRIPTION In general, a pseudo-data generation apparatus includes processing circuitry. The processing circuitry acquires one or more pieces of partial observation data that form part of whole observation data. The processing circuitry generates pseudo-whole observation data by inputting the one or more pieces of partial observation data to a function, the pseudo-whole observation data being pseudo-data of the whole observation data. The function is optimized by training so that partial observation data for training and pseudo-partial observation data for training resemble each other, the pseudo-partial observation data for training being obtained by converting the pseudo-whole observation data for training. Hereinafter, a pseudo-data generation apparatus, a pseudo-data generation method, a pseudo-data generation program, a learning apparatus, a learning method, and a learning program according to the present embodiment will be described with reference to the drawings. In the following embodiments, elements assigned the same reference numeral perform the same operation, and repeat descriptions will be omitted as appropriate. Hereinafter, an embodiment will be described with reference to the accompanying drawings. (First Embodiment) A learning apparatus and a pseudo-data generation apparatus according to a first embodiment will be desc