JP-7856792-B2 - Endoscope control processing device, endoscope system, method of operating the endoscope control processing device, program, and endoscope device

JP7856792B2JP 7856792 B2JP7856792 B2JP 7856792B2JP-7856792-B2

Inventors

河野秀太郎

Assignees

オリンパスメディカルシステムズ株式会社

Dates

Publication Date: 20260511
Application Date: 20221227

Claims (16)

An endoscope control processing device that acquires and processes image signals frame by frame from an endoscope having an image sensor equipped with an electronic shutter function that controls the exposure time, Equipped with a processor, The aforementioned processor, The image signal acquired in the first frame is subjected to dimming detection to calculate the target exposure time for the second frame, which is time-dependent on the first frame. Based on the target exposure time value, the electronic shutter control value for the second frame is calculated. The actual exposure time is calculated from the aforementioned electronic shutter control value. The ratio of the actual exposure time to the target exposure time is calculated. Based on the electronic shutter control value, the image sensor is made to take an image, and the image signal of the second frame is acquired from the endoscope. An endoscope control processing apparatus characterized in that, in the image processing of the image signal of the second frame, it is configured to adjust the gain based on the exposure time ratio.
The aforementioned processor, The aforementioned electronic shutter control value is compared with a threshold value, The endoscope control apparatus according to claim 1, characterized in that it is configured to adjust the gain based on the exposure time ratio depending on the magnitude of the threshold.
The image sensor is capable of controlling the electronic shutter on a line-by-line basis. The aforementioned electronic shutter control value is the number of lines that are shielded from light during exposure. The aforementioned processor, If the electronic shutter control value is greater than or equal to the threshold, the gain is adjusted based on the exposure time ratio during image processing of the image signal of the second frame. The endoscope control apparatus according to claim 2, characterized in that, if the electronic shutter control value is less than the threshold, the gain is not adjusted based on the exposure time ratio during image processing of the image signal of the second frame.
The endoscope control apparatus according to claim 3, characterized in that the threshold value is set to a value smaller than or equal to the value obtained by subtracting "100 / target dimming accuracy" lines from the total number of lines of the image sensor.
An endoscope having an image sensor equipped with an electronic shutter function that controls the exposure time, An endoscope control processing device having a processor that acquires and processes image signals from the endoscope on a frame-by-frame basis, Equipped with, The aforementioned processor, The image signal acquired in the first frame is subjected to dimming detection to calculate the target exposure time for the second frame, which is time-dependent on the first frame. Based on the target exposure time value, the electronic shutter control value for the second frame is calculated. The actual exposure time is calculated from the aforementioned electronic shutter control value. The ratio of the actual exposure time to the target exposure time is calculated. Based on the electronic shutter control value, the image sensor is made to take an image, and the image signal of the second frame is acquired from the endoscope. An endoscope system characterized in that, in the image processing of the image signal of the second frame, the gain is adjusted based on the exposure time ratio.
The aforementioned processor, The aforementioned electronic shutter control value is compared with a threshold value, The endoscope system according to claim 5, characterized in that it is configured to adjust the gain based on the exposure time ratio depending on the magnitude of the threshold.
The image sensor is capable of controlling the electronic shutter on a line-by-line basis. The aforementioned electronic shutter control value is the number of lines that are shielded from light during exposure. The aforementioned processor, If the electronic shutter control value is greater than or equal to the threshold, the gain is adjusted based on the exposure time ratio during image processing of the image signal of the second frame. The endoscope system according to claim 6, characterized in that, if the electronic shutter control value is less than the threshold, the system is configured not to adjust the gain based on the exposure time ratio during image processing of the image signal of the second frame.
The endoscope system according to claim 7, characterized in that the threshold value is set to a value smaller than or equal to the value obtained by subtracting "100 / target dimming accuracy" lines from the total number of lines of the image sensor.
A method for operating an endoscope control processing device that acquires and processes image signals frame by frame from an endoscope having an image sensor equipped with an electronic shutter function that controls the exposure time, The image signal acquired in the first frame is subjected to dimming detection to calculate the target exposure time for the second frame, which is time-dependent on the first frame. Based on the target exposure time value, the electronic shutter control value for the second frame is calculated. The actual exposure time is calculated from the aforementioned electronic shutter control value. The ratio of the actual exposure time to the target exposure time is calculated. Based on the electronic shutter control value, the image sensor is made to take an image, and the image signal of the second frame is acquired from the endoscope. A method for operating an endoscope control processing device, characterized in that, in image processing of the image signal of the second frame, the gain is adjusted based on the exposure time ratio.
The aforementioned electronic shutter control value is compared with a threshold value, The method for operating an endoscope control processing device according to claim 9, characterized in that the gain is adjusted or not adjusted based on the exposure time ratio depending on the magnitude of the threshold.
The image sensor is capable of controlling the electronic shutter on a line-by-line basis. The aforementioned electronic shutter control value is the number of lines that are shielded from light during exposure. If the electronic shutter control value is greater than or equal to the threshold, the gain is adjusted based on the exposure time ratio during image processing of the image signal of the second frame. The method for operating the endoscope control processing device according to claim 10, characterized in that if the electronic shutter control value is less than the threshold, the gain is not adjusted based on the exposure time ratio in the image processing of the image signal of the second frame.
The method for operating the endoscope control processing device according to claim 11, characterized in that the threshold value is set to a value smaller than or equal to the value obtained by subtracting "100 / target dimming accuracy" lines from the total number of lines of the image sensor.
A program for a computer that acquires and processes image signals frame by frame from an endoscope having an image sensor equipped with an electronic shutter function that controls the exposure time, The aforementioned program, The image signal acquired in the first frame is subjected to dimming detection to calculate the target exposure time for the second frame, which is time-dependent on the first frame. Based on the target exposure time value, the electronic shutter control value for the second frame is calculated. The actual exposure time is calculated from the aforementioned electronic shutter control value. The ratio of the actual exposure time to the target exposure time is calculated. Based on the electronic shutter control value, the image sensor is made to take an image, and the image signal of the second frame is acquired from the endoscope. In the image processing of the image signal of the second frame, the gain is adjusted based on the exposure time ratio. A program characterized by causing the computer to perform the processing.
The aforementioned electronic shutter control value is compared with a threshold value, Depending on the magnitude of the threshold, the gain is adjusted or not adjusted based on the exposure time ratio. The program according to claim 13, characterized in that it causes the computer to perform further processing.
It is an endoscope device, Equipped with a processor, The aforementioned processor, The first imaging signal, generated by the image sensor reading the charge corresponding to the first frame, is then processed to generate a first image signal. By dimming and detecting the first image signal, the target exposure time for the second frame, which is the frame following the first frame, is determined. Based on the target exposure time, an electronic shutter control value is determined to adjust the exposure time in the second frame. The exposure time ratio is calculated as the ratio of the actual exposure time adjusted by the electronic shutter control value to the target exposure time in the second frame. A second image signal is generated by image processing the second imaging signal, which is generated by the image sensor, controlled based on the electronic shutter control value in the second frame, reading out the charge corresponding to the second frame. An endoscope device characterized in that, in the image processing of the second imaging signal, the gain is adjusted based on the calculated exposure time ratio in the second frame.
The endoscope apparatus according to claim 15, characterized in that the processor compares the electronic shutter control value with a threshold, and if the electronic shutter control value is greater than the threshold, it applies gain adjustment based on the exposure time ratio, and if the electronic shutter control value is less than the threshold, it does not apply gain adjustment based on the exposure time ratio.

Description

The present invention relates to an endoscope control processing device, an endoscope system, and a method for operating the endoscope control processing device, which acquire and process image signals from an image sensor equipped with an electronic shutter function. Conventionally, aperture, electronic shutter, and gain have been used for dimming. The aperture controls dimming by changing the brightness of the optical image. The electronic shutter controls dimming by changing the exposure time, i.e., the charge accumulation time. Gain controls dimming by amplifying the image signal. For example, Japanese Patent Publication No. 2018-14680 describes a technique for achieving a wide dynamic range of exposure in a digital still camera. The technique described in this publication involves the following exposure control during a change in the subject from a dark state to a bright state. When the subject is at its darkest, the aperture is opened wide, the gain is maximized, and the degree to which the electronic shutter operates slower than the frame rate is controlled. When the subject is slightly dark, the aperture is opened wide, the electronic shutter operates at the frame rate, and the gain is controlled between maximum and zero. When the subject is slightly bright, the electronic shutter operates at the frame rate, the gain is set to zero, and the aperture is controlled. When the subject is bright, the gain is set to zero, the aperture is stopped down to its maximum, and the degree to which the electronic shutter operates faster than the frame rate is controlled. Thus, in the above-mentioned publication, exposure control is performed in the following order as the subject changes from a dark state to a bright state: "electronic shutter control" → "gain control" → "aperture control" → "electronic shutter control". In other words, when exposure control is performed using only one of the electronic shutter, gain, or aperture, the technique changes which of the electronic shutter, gain, or aperture is used according to the brightness of the subject. Therefore, the above-mentioned publication does not describe the use of two or more controls simultaneously. Incidentally, exposure control using an electronic shutter generally involves discrete exposure times. In an electronic shutter, the shading state is achieved, for example, by sweeping away unwanted charges from pixels. The exposure time is the time from when sweeping stops and charge accumulation begins until the charge from the pixel is read out. In an image sensor where pixels are arranged in lines, exposure control by the electronic shutter is performed by determining how many lines are exposed and how many lines are left shaded at a given time. In other words, exposure control using an electronic shutter quantizes the exposure time into units of one line, resulting in quantization errors relative to the desired brightness. The proportion of quantization error in the image signal increases as the exposure time decreases. That is, the effect of quantization error is greater when there are fewer exposed lines than when there are more lines in the exposed state (exposure lines). The state where the number of exposed lines is minimized (the state where the number of light-blocking lines is maximized) is called aperture fully closed. In endoscopic systems, subjects with a wide dynamic range of light control are observed, from the dark areas at the back of the lumen to the mucosa at close range. In particular, when observing the surface of the mucosa at close range (i.e., when the subject is bright) in video, the electronic shutter may be controlled near its maximum aperture. Also, when a laser is irradiated onto a lesion, for example, the amount of light in the field of view may increase rapidly. In this case, the electronic shutter may be controlled near its maximum aperture to suppress halation in the image. However, as mentioned above, the accuracy of light control becomes rough near the maximum aperture, causing the brightness of the video to fluctuate from frame to frame, resulting in flickering and hindering image observation. On the other hand, one method to suppress flicker is to increase the number of exposure lines corresponding to the full aperture of the electronic shutter, thereby lowering the upper limit of the proportion of quantization error in the image signal. For example, if the number of exposure lines corresponding to the full aperture of the electronic shutter is set to 20 lines, even if a quantization error of one line occurs, the variation in brightness from frame to frame can be suppressed to 5% or less. However, using this method results in a narrower dynamic range for dimming. The present invention has been made in view of the above circumstances, and aims to provide an endoscope control processing device, an endoscope system, and a method for operating the endoscope control processing device that can ensure a wide dynamic range of d