Search

JP-7856485-B2 - Endoscope device, frame image extraction method, program, and endoscopic system

JP7856485B2JP 7856485 B2JP7856485 B2JP 7856485B2JP-7856485-B2

Inventors

  • 臼井 省吾

Assignees

  • ワブテック・インスペクション・テクノロジーズ・ジャパン株式会社

Dates

Publication Date
20260511
Application Date
20220520

Claims (15)

  1. An insertion part containing an image sensor is inserted into the inside of the subject, which is a machine , An operation reception unit that accepts operations, Control unit and Equipped with, The control unit, During the recording process of a video including multiple frame images generated based on the imaging signal output from the image sensor, a tagging unit adds information about the operation as a tag to the frame image corresponding to the timing when the operation reception unit receives an operation, and adds information about the specific feature image as a tag to the frame image recognized as a specific feature image representing a component contained inside the machine . An extraction unit that extracts frame images from the multiple frame images contained in the video based on at least one of two or more tags selected from the multiple types of tags, An endoscope device for the industrial field , characterized by including the following features.
  2. The endoscope apparatus according to claim 1, characterized in that the specific feature image is determined in accordance with the feature image selection operation received by the operation reception unit before the video recording process.
  3. The endoscope apparatus according to claim 1, characterized in that the specific feature image is determined based on the content of a tag related to the specific feature image, which is added by the tagging unit to a frame image included in a video recorded by a recording process previously performed on a subject of the same type.
  4. The endoscope apparatus according to claim 1, characterized in that the aforementioned specific feature image is obtained by machine learning the features of frame images to which a specific type of tag has been attached, which are included in videos recorded by recording processing performed in the past on the same type of subject.
  5. The endoscope apparatus according to claim 1, characterized in that the two or more types of tags are selected in accordance with the tag selection operation received by the operation reception unit.
  6. A display unit that displays tags attached to frame images included in the aforementioned video. Furthermore, The endoscope apparatus according to claim 5, characterized in that the tag selection operation is an operation to select two or more types of tags from among the tags displayed by the display unit.
  7. The endoscope apparatus according to claim 6, further characterized in that the display unit displays a time bar for the video and displays tags attached to the frame images included in the video in an identifiable manner on the time bar.
  8. A display unit that displays the time bar of the aforementioned video and displays the tags attached to the frame images included in the video in an identifiable manner on the time bar. Furthermore, The endoscope apparatus according to claim 5, characterized in that the tag selection operation is an operation to select two or more types of tags by specifying a region from among the tags that are identifiable on the time bar displayed by the display unit.
  9. The endoscope apparatus according to claim 1, characterized in that, among the two or more types of tags, some types of tags are pre-selected, and other types of tags are selected according to the tag selection operation received by the operation reception unit.
  10. The endoscope apparatus according to claim 1, characterized in that the extraction unit extracts a corresponding section from the video for each of the two or more types of tags, and performs the extraction based on that section.
  11. A display unit that displays the frame image extracted by the extraction unit. The endoscopic apparatus according to claim 1 , further comprising the following:
  12. Using endoscope equipment for industrial applications, During the recording process of a video including multiple frame images generated based on imaging signals output from an image sensor contained in an insertion part inserted into the inside of a machine ( the subject), information about the operation is added as a tag to the frame image corresponding to the timing when the operation reception unit receives an operation, and information about the specific feature image is added as a tag to the frame image recognized as a specific feature image representing a component contained inside the machine . A frame image extraction method characterized by extracting a frame image from the multiple frame images contained in the video based on at least one tag selected from two or more tags chosen from among multiple types of tags.
  13. During the recording process of a video including multiple frame images generated based on imaging signals output from an image sensor contained in an insertion part inserted into the inside of a machine ( the subject), information about the operation is added as a tag to the frame image corresponding to the timing when the operation reception unit receives an operation, and information about the specific feature image is added as a tag to the frame image recognized as a specific feature image representing a component contained inside the machine . Based on at least one of two or more tags selected from among multiple types of tags, a frame image is extracted from the multiple frame images contained in the video. A program characterized by causing a processor to perform processing either mounted on or located outside an endoscope device used in the industrial field .
  14. Endoscopic devices for the industrial sector , Control device and Equipped with, The aforementioned endoscope device is An insertion part containing an image sensor is inserted into the inside of the subject, which is a machine , An operation reception unit that accepts operations, Includes, The control device is During the recording process of a video including multiple frame images generated based on the imaging signal output from the image sensor, a tagging unit adds information about the operation as a tag to the frame image corresponding to the timing when the operation reception unit receives an operation, and adds information about the specific feature image as a tag to the frame image recognized as a specific feature image representing a component contained inside the machine . An extraction unit that extracts frame images from the multiple frame images contained in the video based on at least one of two or more tags selected from the multiple types of tags, An endoscopic system characterized by including the following:
  15. The endoscope apparatus according to claim 1, wherein the extraction unit identifies the interval from the start time to the end time in the video for each selected tag.

Description

This disclosure relates to an endoscopic device, a frame image extraction method, a program, and an endoscopic system. Endoscopic examinations involve recording video of the inside of the patient's body and reviewing it afterward. This process becomes increasingly difficult as the recorded video length increases. Therefore, there is a need for technology that can efficiently review only the important parts (key frames) within the recorded video. Patent Document 1 describes an image processing device for processing images acquired during examinations using a capsule endoscope. This image processing device compares the imaging times of multiple types of feature images (e.g., pyloric images, clip images, papilla images, and Bauhin's valve images) extracted from the image group acquired during the examination (the current image group) with the imaging times of corresponding multiple types of feature images extracted from a group of images acquired in past examinations of the same subject (the past image group). If the time interval between feature images in the current image group is longer than a certain threshold compared to the time interval between corresponding feature images in the past image group, a flag is added to the images acquired within that time interval in the current image group to indicate that they are images requiring attention. Furthermore, when displaying the current image group, the flagged images are displayed in a different format from other images to draw the attention of medical professionals and encourage focused observation. Patent No. 5568196 This figure illustrates the external configuration of an endoscope device 1 according to one embodiment.This figure illustrates the internal configuration of an endoscope device 1 according to one embodiment.This flowchart illustrates the processing flow performed by the control unit 34 in response to a video recording start operation.This flowchart illustrates the processing flow performed by the control unit 34 in response to a video selection operation.This diagram illustrates the condition selection screen displayed in S220 and where a tag has been selected in S230.This diagram illustrates a specific example of extraction in S240.This figure shows a modified example of the condition selection screen displayed in S220.This figure shows a modified example of the condition selection screen displayed in S220.This figure shows an example of selecting tags by specifying a region.This diagram illustrates the hardware configuration of computer 100. The embodiments of the present invention will be described below with reference to the drawings. Figure 1 illustrates the external configuration of an endoscope device 1 according to one embodiment. The endoscope device 1 illustrated in Figure 1 is used for endoscopic examinations in the industrial field and comprises an insertion section 10, an operating section 20, and a main body section 30. The insertion portion 10 has an elongated shape that allows it to be inserted into the interior of a specimen such as a turbine or engine, and includes a tip portion 11, a bendable portion 12, and a long, flexible tube portion 13. The operation unit 20 includes a joystick (bending control element) 21 that receives inputs for bending the curved section 12 in a desired direction (bending operation), and a plurality of buttons (not shown) that receive inputs for various purposes. The main unit 30 includes a display unit 31 and an external interface 32. The display unit 31 is a display device such as an LCD (Liquid Crystal Display), and displays various screens such as images of the subject's internal organs and condition selection screens, and plays back recorded videos. The display unit 31 also has a touch panel 31a that accepts touch operations for various inputs. The touch panel 31a and the aforementioned operation unit 20 are examples of operation reception units that accept user (examiner) operations. The external interface 32 is connected to an external device such as an external storage device (e.g., a USB (Universal Serial Bus) memory). Figure 2 illustrates the internal configuration of an endoscope device 1 according to one embodiment. As illustrated in Figure 2, in the endoscope device 1, the tip portion 11 of the insertion portion 10 includes an imaging optical system 11a, an image sensor 11b, a light-emitting element 11c, and an illumination optical system 11d. The imaging optical system 11a forms an image of the subject onto the image sensor 11b. The image sensor 11b captures (photoelectrically converts) the image of the subject formed by the imaging optical system 11a, generates an imaging signal, and outputs it to the main unit 30 (image generation unit 33). The image sensor 11b is a CCD (Charge Coupled Device) image sensor, a CMOS (Complementary Metal Oxide Semiconductor) image sensor, etc. The light-emitting element 11c emits illumination light to illuminate the subject. The light-emitting element 1