EP-4740831-A1 - DIAGNOSTIC IMAGING ASSISTANCE DEVICE, DIAGNOSTIC IMAGING ASSISTANCE SYSTEM, AND DIAGNOSTIC IMAGING ASSISTANCE METHOD

EP4740831A1EP 4740831 A1EP4740831 A1EP 4740831A1EP-4740831-A1

Abstract

An image diagnosis assistance apparatus 5 includes: a captured image acquisition unit 514 configured to acquire a captured image captured by an imaging device 2 configured to image a subject; a movement determination unit 510 configured to determine magnitude of relative movement between the imaging device 2 and the subject; an image processing unit 515 configured to execute image processing on the captured image to output a processed image; an image generation unit 516 configured to generate an image for diagnosis based on the processed image; and an estimation unit 517 configured to estimate a diagnostic candidate area serving as a diagnostic candidate in the processed image by executing an estimation process for the processed image using a trained model. An input frame rate for the processed image on which high-definition processing is sequentially executed at the image generation unit 516 and an input frame rate for the processed image input sequentially to the estimation unit 517 differ from each other according to the magnitude of the movement.

Inventors

SUZUKI, HIROSHI
MATSUSHITA, AKIRA
MIYAZAWA, SHINGO
YAJI, TSUYOSHI

Assignees

Olympus Medical Systems Corp.

Dates

Publication Date: 20260513
Application Date: 20230706

Claims (20)

An image diagnosis assistance apparatus, comprising: a captured image acquisition unit configured to acquire a captured image captured by an imaging device configured to image a subject; a movement determination unit configured to determine magnitude of relative movement between the imaging device and the subject; an image processing unit configured to execute image processing on the captured image to output a processed image; an image generation unit configured to generate, based on the processed image, an image for diagnosis; and an estimation unit configured to estimate a diagnostic candidate area serving as a diagnostic candidate in the processed image, by executing an estimation process for the processed image using a trained model, wherein an input frame rate for the processed image with which the image generation unit sequentially executes a process of generating the image for diagnosis, and an input frame rate for the processed image sequentially input to the estimation unit differ from each other according to the magnitude of the movement.
The image diagnosis assistance apparatus according to claim 1, wherein the image generation unit is configured to generate the image for diagnosis by executing high-definition processing according to image quality of the processed image using the trained model.
The image diagnosis assistance apparatus according to claim 1, wherein the image generation unit is configured to generate, as the image for diagnosis, a high-definition image higher in definition than the processed image by executing high-definition processing on the processed image.
The image diagnosis assistance apparatus according to claim 1, further comprising a display control unit configured to generate a display image based on the image for diagnosis and the diagnostic candidate area.
The image diagnosis assistance apparatus according to claim 4, wherein the image generation unit is configured to generate, as the image for diagnosis, a high-definition image higher in image quality than the processed image by executing high-definition processing on the processed image.
The image diagnosis assistance apparatus according to claim 1, further comprising a frame selection unit configured to differ the input frame rate for the processed image with which the image generation unit sequentially executes the process of generating the image for diagnosis, and the input frame rate for the processed image sequentially input to the estimation unit from each other, according to the magnitude of the movement.
The image diagnosis assistance apparatus according to claim 1, further comprising an imaging control unit configured to change an imaging mode of the imaging device according to the magnitude of the movement.
The image diagnosis assistance apparatus according to claim 7, wherein the imaging control unit is configured to change a frame rate of imaging by the imaging device according to the magnitude of the movement.
The image diagnosis assistance apparatus according to claim 1, further comprising a light source control unit configured to change, according to the magnitude of the movement, an illumination mode of a light source device configured to supply illumination light to the subject.
The image diagnosis assistance apparatus according to claim 1, wherein when the magnitude of the movement is at a predetermined threshold or more, the input frame rate for the processed image sequentially input to the estimation unit is larger than the input frame rate for the processed image with which the image generation unit sequentially executes the process of generating the image for diagnosis.
The image diagnosis assistance apparatus according to claim 10, wherein when the magnitude of the movement is at the predetermined threshold or more, the image generation unit is configured to thin out frames of the processed image input and set the input frame rate for the processed image with which the image generation unit sequentially executes the process of generating the image for diagnosis to be smaller than the input frame rate for the processed image sequentially input to the estimation unit.
The image diagnosis assistance apparatus according to claim 1, further comprising: a display control unit configured to generate a display image based on the image for diagnosis and the diagnostic candidate area, wherein the display control unit is configured to, based on the image for diagnosis and the diagnostic candidate area that result from processing of the processed image of a same frame by the image generation unit and the estimation unit, generate the display image having the diagnostic candidate area superposed on the image for diagnosis.
The image diagnosis assistance apparatus according to claim 1, further comprising: a display control unit configured to generate a display image based on the image for diagnosis and the diagnostic candidate area, wherein the display control unit is configured to, based on the image for diagnosis and the diagnostic candidate area that result from processing of the processed image of a different frame by the image generation unit and the estimation unit, generate the display image including the image for diagnosis and information indicating a frame from which the diagnostic candidate area has been estimated.
The image diagnosis assistance apparatus according to claim 1, further comprising: a display control unit configured to generate a display image based on the image for diagnosis and the diagnostic candidate area, wherein the display control unit is configured to, based on the image for diagnosis and the diagnostic candidate area that result from processing of the processed image of a different frame by the image generation unit and the estimation unit, generate the display image having the diagnostic candidate area superposed on the image for diagnosis generated from the processed image of a frame nearest to the processed image of a frame from which the diagnostic candidate area has been estimated.
The image diagnosis assistance apparatus according to claim 1, wherein the movement determination unit is configured to determine the magnitude of the movement based on output of at least one of an acceleration sensor or an angular velocity sensor provided in the imaging device.
The image diagnosis assistance apparatus according to claim 1, wherein the movement determination unit is configured to determine the magnitude of the movement based on the processed image.
The image diagnosis assistance apparatus according to claim 1, wherein the estimation unit is configured to change the trained model according to the magnitude of the movement.
The image diagnosis assistance apparatus according to claim 1, wherein the estimation unit is configured to estimate, as the diagnostic candidate area, an area having reliability at a reliability threshold or more, among reliability of each predetermined area in the processed image output by the estimation process, and change the reliability threshold according to the magnitude of the movement.
The image diagnosis assistance apparatus according to claim 1, wherein the estimation unit is configured to estimate, as the diagnostic candidate area, an area where reliability is consecutively at a reliability threshold or more for a predetermined number of frames among reliability of each predetermined area in the processed image output by the estimation process, and change the predetermined number of frames according to the magnitude of the movement.
An image diagnosis assistance apparatus, comprising: a captured image acquisition unit configured to acquire a captured image captured by an imaging device configured to image a subject; a movement determination unit configured to determine magnitude of relative movement between the imaging device and the subject; an image processing unit configured to execute image processing on the captured image to output a processed image; an image generation unit configured to generate a high-definition image higher in definition than the processed image by executing high-definition processing on the processed image; an estimation unit configured to estimate a diagnostic candidate area serving as a diagnostic candidate in the processed image by executing an estimation process for the processed image using a trained model; and a display control unit configured to generate a display image based on the high-definition image and the diagnostic candidate area, wherein the display control unit is configured to change a form of the display image according to the magnitude of the movement.

Description

Field The present invention relates to an image diagnosis assistance apparatus, an image diagnosis assistance system, and an image diagnosis assistance method. Background Image recognition techniques based on artificial intelligence (AI) have been proposed recently in the medical field (see, for example, Patent Literature 1). In a technique described in Patent Literature 1, an estimation process using a trained model for a capture image captured by an endoscope is executed, and a diagnostic candidate area of, for example, a lesion in the captured image is thereby estimated. Citation List Patent Literature Patent Literature 1: International Publication Pamphlet No. WO 2022/004056 Summary Technical Problem In a case where an image recognition technique based on AI is utilized, for an observation area of low importance, a medical doctor may increase the observation speed and withdraw the endoscope by relying on the image recognition technique based on AI. However, in a case where the endoscope is withdrawn rapidly, precision of estimation of a diagnostic candidate area of, for example, a lesion by the image recognition technique based on AI is reduced by, for example, image blur due to the rapid movement. Therefore, the endoscope is unable to be withdrawn rapidly from the observation area of low importance and the burden on the medical doctor is unable to be reduced. The present invention has been made in view of the above, and an object thereof is to provide an image diagnosis assistance apparatus, an image diagnosis assistance system, and an image diagnosis assistance method that enable assistance for diagnosis, the assistance reducing burdens on medical doctors. Solution to Problem To solve the problem described above and to achieve the object, an image diagnosis assistance apparatus according to the present invention includes: a captured image acquisition unit configured to acquire a captured image captured by an imaging device configured to image a subject; a movement determination unit configured to determine magnitude of relative movement between the imaging device and the subject; an image processing unit configured to execute image processing on the captured image to output a processed image; an image generation unit configured to generate, based on the processed image, an image for diagnosis; and an estimation unit configured to estimate a diagnostic candidate area serving as a diagnostic candidate in the processed image, by executing an estimation process for the processed image using a trained model. An input frame rate for the processed image with which the image generation unit sequentially executes a process of generating the image for diagnosis, and an input frame rate for the processed image sequentially input to the estimation unit differ from each other according to the magnitude of the movement. An image diagnosis assistance apparatus according to the present invention includes: a captured image acquisition unit configured to acquire a captured image captured by an imaging device configured to image a subject; a movement determination unit configured to determine magnitude of relative movement between the imaging device and the subject; an image processing unit configured to execute image processing on the captured image to output a processed image; an image generation unit configured to generate a high-definition image higher in definition than the processed image by executing high-definition processing on the processed image; an estimation unit configured to estimate a diagnostic candidate area serving as a diagnostic candidate in the processed image by executing an estimation process for the processed image using a trained model; and a display control unit configured to generate a display image based on the high-definition image and the diagnostic candidate area. The display control unit is configured to change a form of the display image according to the magnitude of the movement. An image diagnosis assistance system according to the present invention includes: an imaging device configured to generate a captured image by imaging a subject; and an image diagnosis assistance apparatus configured to process the captured image. The image diagnosis assistance apparatus includes: a captured image acquisition unit configured to acquire the captured image; a movement determination unit configured to determine magnitude of relative movement between the imaging device and the subject; an image processing unit configured to execute image processing on the captured image to output a processed image; an image generation unit configured to generate a high-definition image higher in definition than the processed image by executing high-definition processing on the processed image; and an estimation unit configured to estimate a diagnostic candidate area serving as a diagnostic candidate in the processed image by executing an estimation process for the processed image using a trained model, and an inpu