US-12620211-B2 - Processing system, image processing method, learning method, and processing device

US12620211B2US 12620211 B2US12620211 B2US 12620211B2US-12620211-B2

Abstract

A processing system includes a processor with hardware. The processor is configured to perform processing of acquiring a detection target image captured by an endoscope apparatus, controlling the endoscope apparatus based on control information, detecting a region of interest included in the detection target image based on the detection target image for calculating estimated probability information representing a probability of the detected region of interest, identifying the control information for improving the estimated probability information related to the region of interest within the detection target image based on the detection target image, and controlling the endoscope apparatus based on the identified control information.

Inventors

Satoshi Ohara

Assignees

OLYMPUS CORPORATION

Dates

Publication Date: 20260505
Application Date: 20241119

Claims (14)

1 . A processing system comprising: a processor including hardware, wherein the processor is configured to: acquire a first detection target image generated based on a first control information; calculate a detection result of a region of interest and estimated probability of the detected result based on the first detection target image; determine whether or not the estimated probability is equal to or higher than a first threshold; output the detection result of the region of interest in a case where the estimated probability is equal to or higher than the first threshold; and perform update of the first control information in a case where the estimated probability is less than the first threshold, wherein in performing the update of the first control information, the processor is configured to: identify a second control information and a third control information based on the first detection target image and the first control information; acquire a second detection target image based on the second control information; acquire a third detection target image based on the third control information; detect the region of interest based the second detection target image and calculates a second estimated probability representing probability of a detection result; detect the region of interest based the third detection target image and calculate a third estimated probability representing probability of a detection result; and perform an update using control information used for outputting one of the second estimated probability and the third estimated probability that is higher as the first control information.
2 . An endoscope system comprising: the processing system according to claim 1 , wherein the first control information includes at least one of light source control information for controlling a light source which irradiates an object with illumination light, imaging control information for controlling an imaging condition for capturing the detection target image, and image processing control information for controlling image processing to a signal of the captured image.
3 . The endoscope system according to claim 2 , wherein the light source control information includes at least one of a wavelength, light quantity ratio, light quantity, duty, and light distribution of a light source.
4 . The endoscope system according to claim 2 , wherein the imaging control information is an imaging frame rate.
5 . The endoscope system according to claim 2 , wherein the image processing control information includes at least one information of a color matrix, structure highlighting, noise reduction and automatic gain control.
6 . The processing system according to claim 1 , wherein the processor is configured to output the detection result of the region of interest and the estimated probability by inputting the first detection target image to a first trained model.
7 . The processing system according to claim 6 , wherein the processor is configured to output a parameter type that should be preferentially changed for improving the estimated probability by inputting the first detection target image and the first control information to a second trained model.
8 . The processing system according to claim 7 , wherein the processor is configured to identify the second control information and the third control information based on the parameter type.
9 . An endoscope system comprising: the processing system according to claim 1 , wherein the processor is configured to generate a display image based on image processing control information.
10 . The endoscope system according to claim 9 , wherein the processor is configured to acquire the first detection target image and the display image alternately.
11 . The endoscope system according to claim 10 , wherein the processor is configured to superpose the detection result of the region of interest and information of the estimated probability on the display image.
12 . The processing system according to claim 1 , wherein the processor is configured to: extract the region of interest from each frame of the first detection target image; and continuously output the detection result of the region of interest.
13 . A method performed by a processor, the method comprising: acquiring a first detection target image generated based on a first control information; calculating a detection result of a region of interest and estimated probability of the detected result based on the first detection target image; determining whether or not the estimated probability is equal to or higher than a first threshold; outputting the detection result of the region of interest in a case where the estimated probability is equal to or higher than the first threshold; and performing update of the first control information in a case where the estimated probability is less than the first threshold, wherein performing the update of the first control information comprises: identifying a second control information and a third control information based on the first detection target image and the first control information; acquiring a second detection target image based on the second control information; acquiring a third detection target image based on the third control information; detecting the region of interest based the second detection target image and calculates a second estimated probability representing probability of a detection result; detecting the region of interest based the third detection target image and calculate a third estimated probability representing probability of a detection result; and performing an update using control information used for outputting one of the second estimated probability and the third estimated probability that is higher as the first control information.
14 . A non-transitory computer-readable storage medium storing instructions that cause a computer to at least perform: acquiring a first detection target image generated based on a first control information; calculating a detection result of a region of interest and estimated probability of the detected result based on the first detection target image; determining whether or not the estimated probability is equal to or higher than a first threshold; outputting the detection result of the region of interest in a case where the estimated probability is equal to or higher than the first threshold; and performing update of the first control information in a case where the estimated probability is less than the first threshold, wherein performing the update of the first control information comprises: identifying a second control information and a third control information based on the first detection target image and the first control information; acquiring a second detection target image based on the second control information; acquiring a third detection target image based on the third control information; detecting the region of interest based the second detection target image and calculates a second estimated probability representing probability of a detection result; detecting the region of interest based the third detection target image and calculate a third estimated probability representing probability of a detection result; and performing an update using control information used for outputting one of the second estimated probability and the third estimated probability that is higher as the first control information.

Description

CROSS REFERENCE TO RELATED APPLICATION This application is a continuation application of U.S. patent application Ser. No. 17/940,224 filed Sep. 8, 2022, which is a continuation of International Patent Application No. PCT/JP2020/010541, having an international filing date of Mar. 11, 2020, which designated the United States, the entirety of each of which are incorporated herein by reference. BACKGROUND It has been known that the image diagnosis support device for supporting a physician's diagnosis by means of an endoscopic image is configured to utilize machine learning to perform processing of detecting the lesion and acquiring estimated probability indicating the degree of detection accuracy. The neural network has been known as the trained model generated by machine learning. International Publication No. WO 2019/088121 discloses the system which provides support information by estimating information concerning name/position of a lesion, and probability thereof based on a CNN (Convolutional Neural Network) so that the estimated information is superposed on an endoscopic image. SUMMARY In accordance with one of some aspect, there is provided a processing system comprising a processor including hardware, wherein the processor is configured to perform processing of: acquiring a detection target image captured by an endoscope apparatus; controlling the endoscope apparatus based on control information; detecting a region of interest included in the detection target image based on the detection target image for calculating estimated probability information representing a probability of the detected region of interest; identifying the control information for improving the estimated probability information related to the region of interest within the detection target image based on the detection target image; and controlling the endoscope apparatus based on the identified control information. In accordance with one of some aspect, there is provided an image processing method comprising: acquiring a detection target image captured by an endoscope apparatus; detecting a region of interest included in the detection target image to calculate estimated probability information representing a probability of the detected region of interest based on the detection target image; and when using information for controlling the endoscope apparatus as control information, identifying the control information for improving the estimated probability information related to the region of interest within the detection target image based on the detection target image. In accordance with one of some aspect, there is provided a learning method for generating a trained model, comprising: acquiring an image captured by an endoscope apparatus as an input image; when using information for controlling the endoscope apparatus as control information, acquiring first control information as the control information for acquiring the input image; acquiring second control information as the control information for improving estimated probability information which represents a probability of the region of interest detected from the input image; and generating trained model by performing machine learning of a relationship among the input image, the first control information, and the second control information. In accordance with one of some aspect, there is provided a processing device comprising a processor including hardware, wherein the processor is configured to perform processing of: acquiring a detection target image captured by an endoscope apparatus; detecting a region of interest included in the detection target image to calculate estimated probability information which represents a probability of the detected region of interest based on the detection target image; identifying the control information for improving the estimated probability information related to the region of interest within the detection target image based on the detection target image; and outputting the identified control information. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 illustrates a configuration example of a processing system; FIG. 2 illustrates an external appearance of an endoscope system; FIG. 3 illustrates a configuration example of an endoscope system; FIG. 4 illustrates a control information example; FIG. 5 illustrates a configuration example of a learning device; FIGS. 6A and 6B each illustrate a configuration example of a neural network; FIG. 7A illustrates a training data example for NN1; FIG. 7B illustrates an example of input/output operation of NN1; FIG. 8 illustrates a flowchart representing NN1 learning processing; FIG. 9A illustrates a training data example for NN2; FIG. 9B illustrates a data example for acquiring training data; FIG. 9C illustrates an example of input/output operation of NN2; FIG. 10 illustrates a flowchart representing NN2 learning processing; FIG. 11 illustrates a flowchart representing detection processing and control information