Search

CN-122028835-A - AI-based imaging mode selection in endoscopy

CN122028835ACN 122028835 ACN122028835 ACN 122028835ACN-122028835-A

Abstract

Systems, devices, and methods for determining or adjusting an imaging mode for examining tissue or foreign matter during an endoscopic procedure are disclosed. An exemplary endoscope system includes an endoscope and a controller circuit. The endoscope includes an imaging system to obtain images or video streams of the target anatomy during an endoscopic procedure. The controller circuit may generate endoscopic image or video features characterizing abnormalities in the target anatomy from the obtained image or video stream and determine a personalized pathology-specific imaging mode to enhance the discernability of the abnormalities from the image or video stream. A trained machine learning model may be used to determine the recommended imaging mode. The recommended imaging mode may be provided to a user or a processing device to facilitate manual or automatic endoscopic imaging mode adjustment during an endoscopic procedure.

Inventors

  • Sailesh Kangjie Di
  • LIU DAWEI
  • Michael. Ryan

Assignees

  • 捷锐士阿希迈公司(以奥林巴斯美国外科技术名义)

Dates

Publication Date
20260512
Application Date
20240911
Priority Date
20230912

Claims (20)

  1. 1. An endoscope system, comprising: an endoscope including an imaging system configured to obtain an image or video stream of a target anatomy in a patient during an endoscopic procedure, and The controller circuitry may be configured to control the operation of the controller circuitry, the controller circuit is configured to: Analyzing the obtained image or video stream to generate an endoscopic image or video feature characterizing an abnormality in the target anatomy; Automatically determining a target or recommended imaging mode of the imaging system based at least in part on the endoscopic image or video feature and one or more assist features other than the endoscopic image or video feature to enhance discernability of the anomaly from an image or video stream of the target anatomy in subsequent imaging of the target anatomy, and The target or recommended imaging mode is provided to a user or robotic system to facilitate manual or automatic endoscopic imaging mode adjustment during the endoscopic procedure.
  2. 2. The endoscopic system of claim 1, wherein the target or recommended imaging mode comprises one or more of a lighting modality, a zoom setting, or a viewing angle relative to the anomaly.
  3. 3. The endoscopic system of any of claims 1-2, wherein the target or recommended imaging mode comprises one of a narrowband imaging (NBI) mode, a Red Dichroism Imaging (RDI) mode, a White Light Imaging (WLI) mode, or a texture and image enhancement (TXI) mode.
  4. 4. An endoscope system according to any of claims 1-3 and wherein said endoscope is a colonoscope, Wherein the imaging system is configured to obtain images or video streams of each of the different colon segments during a colonoscopy procedure, Wherein the controller circuitry is configured to determine a respective target or recommended imaging mode for use in subsequent imaging of the different colon segment for enhancing discernment of anomalies from the image or video stream.
  5. 5. The endoscope system of any of claims 1-4, wherein the controller circuit is configured to: Detecting the abnormality in the target anatomy using the endoscopic image or video feature, and The target or recommended imaging mode is determined based at least in part on the results of the anomaly detection.
  6. 6. The endoscopic system of claim 5, wherein detecting the abnormality comprises detecting one or more of a presence or absence of the abnormality and a type, size, shape, location, or number of pathological tissue or obstructing mucosa in the target anatomy.
  7. 7. The endoscope system of any of claims 5-6, wherein the controller circuit is configured to detect the anomaly using a first trained Machine Learning (ML) model.
  8. 8. The endoscope system of any of claims 1-7, wherein the controller circuit is configured to: Determining a position of the endoscope in the target anatomy in substantially real-time based at least in part on the endoscopic image or video feature; registering the position of the endoscope to a pre-generated template of the target anatomy, and The position of the endoscope in the target anatomy is displayed on a user interface.
  9. 9. The endoscope system of claim 8 wherein the controller circuit is configured to identify anatomical landmarks using the endoscope images or video features and determine the position of the endoscope based on the identified anatomical landmarks.
  10. 10. The endoscope system of any of claims 8-9, wherein the controller circuit is configured to determine the target or recommended imaging mode further using the determined position of the endoscope in the target anatomy.
  11. 11. The endoscope system of any of claims 1-10, wherein to determine the target or recommended imaging mode, the controller circuit is configured to apply the endoscopic image or video feature to a second trained Machine Learning (ML) model trained to establish a correspondence between the endoscopic image or video feature and one of a set of candidate imaging modes.
  12. 12. The endoscope system of claim 11 wherein the second trained ML model is further trained using the one or more assist features, the one or more assist features include one or more of: a profile of an endoscopist performing the endoscopy procedure; Patient information and medical history data for the patient; endoscope system setup information, or Pre-procedural imaging study data, Wherein to determine the target or recommended imaging mode, the controller circuit is configured to apply an auxiliary input and an enhancement input comprising the endoscopic image or video feature to the second trained ML model.
  13. 13. The endoscope system of any of claims 11-12, wherein the second trained ML model is trained to predict, for each of a plurality of candidate imaging modes, a probability representing a likelihood that the corresponding imaging mode is selected as the target or recommended imaging mode, Wherein the controller circuit is configured to determine the target or recommended imaging mode based at least in part on a probability corresponding to the candidate imaging mode.
  14. 14. The endoscope system of any of claims 1-13, wherein, in response to the target or recommended imaging modality being different from an existing imaging modality used to obtain the image or video stream of the target anatomy, the controller circuit is configured to provide a recommendation to the user to switch to the target or recommended imaging modality to recapture the image or video stream of the target anatomy.
  15. 15. The endoscope system of any of claims 1-14, wherein, in response to the target or recommended imaging modality being different from an existing imaging modality used to obtain the image or video stream of the target anatomy, the controller circuit is configured to generate control signals to cause the imaging system to automatically switch to the target or recommended imaging modality and re-capture the image or video stream of the target anatomy.
  16. 16. A method of determining or adjusting an endoscopic imaging mode during an endoscopic procedure, the method comprising: obtaining an image or video stream of a target anatomy during an endoscopic procedure using an imaging system associated with the endoscope; Analyzing the obtained image or video stream to generate an endoscopic image or video feature characterizing an abnormality in the target anatomy; Automatically determining a target or recommended imaging mode of the imaging system based at least in part on the endoscopic image or video feature and one or more assist features other than the endoscopic image or video feature to enhance discernability of the anomaly from an image or video stream of the target anatomy in subsequent imaging of the target anatomy, and The target or recommended imaging mode is provided to a user or robotic system to facilitate manual or automatic endoscopic imaging mode adjustment during the endoscopic procedure.
  17. 17. The method of claim 16, wherein the target or recommended imaging mode includes one or more of a lighting modality, a zoom setting, or a viewing angle relative to the anomaly.
  18. 18. The method of any of claims 16-17, wherein the target or recommended imaging mode comprises one of a narrowband imaging (NBI) mode, a red bi-color imaging (RDI) mode, a White Light Imaging (WLI) mode, or a texture and image enhancement (TXI) mode.
  19. 19. The method of any of claims 16 to 18, further comprising: Detecting the abnormality in the target anatomy using the endoscopic image or video feature, and The target or recommended imaging mode is determined based at least in part on the results of the anomaly detection.
  20. 20. The method of any of claims 16 to 19, further comprising: identifying anatomical landmarks using the endoscopic image or video feature; Determining a position of the endoscope in the target anatomy in substantially real-time based at least in part on the identified anatomical landmark; registering the position of the endoscope to a pre-generated template of the target anatomy, and Displaying the position of the endoscope in the target anatomy on a user interface, Wherein determining the target or recommended imaging mode is further based on the determined position of the endoscope in the target anatomy.

Description

AI-based imaging mode selection in endoscopy Priority statement The present application claims the benefit of priority from U.S. provisional patent application serial No. 63/582,029 filed on 9/2023, the contents of which are incorporated herein by reference. Technical Field This document relates generally to endoscopic systems and, more particularly, to a system and method for determining or adjusting an imaging mode for examining tissue or foreign matter during an endoscopic procedure. Background Endoscopes have been used in a variety of clinical procedures including, for example, illuminating, imaging, detecting and diagnosing one or more disease states, providing fluid delivery (e.g., saline or other formulation via a fluid channel) toward an anatomical region, providing one or more treatment devices or biological material collection devices with access (e.g., via a working channel) for sampling or treating the anatomical region, and providing aspiration access for collecting fluid (e.g., saline or other formulation), among other procedures. Examples of such anatomical regions may include the gastrointestinal tract (e.g., esophagus, stomach, duodenum, cholangiopancreatic duct, intestine, colon, etc.), renal regions (e.g., kidneys, ureters, bladder, urethra), and other internal organs (e.g., reproductive system, sinus cavities, submucosa regions, respiratory tract), etc. Some endoscopes include a working channel through which an operator may perform aspiration, placement of a diagnostic or therapeutic device (e.g., a brush, biopsy needle or forceps, a stent, basket, or balloon), or minimally invasive procedures such as tissue sampling or removal of unwanted tissue (e.g., benign or malignant strictures) or foreign matter (e.g., stones). Some endoscopes may be used with laser or plasma systems to deliver energy to an anatomical target (e.g., soft or hard tissue or stones) to achieve a desired treatment. For example, lasers have been used in tissue ablation, coagulation, vaporization, fragmentation and lithotripsy applications to break up stones in the kidneys, gall bladder, ureters, and other stone forming areas or to ablate larger stones into smaller fragments. One example of endoscopy is colonoscopy for reducing the incidence and mortality of colorectal cancer. Colonoscopy is typically performed by rapidly advancing a colonoscope to the cecum, and then performing a thorough examination during exit to detect abnormalities (e.g., polyps) and necessary treatments (e.g., polypectomy). Endoscopes typically include an imaging sensor (e.g., an imaging device) that can acquire real-time images or video streams during an endoscopic procedure. The imaging sensor may operate in a preset imaging mode to obtain real-time images or video streams. Imaging modes may include illumination modes, optical magnification, viewing angles of the imaging sensor, and other settings of the imaging sensor and illumination system. Different imaging modes have been used for real-time endoscopy and optical diagnostics, such as high definition White Light Imaging (WLI), dye-based pigment endoscopy techniques, or virtual CE, texture and color enhanced (TXI) imaging, red Dichroism Imaging (RDI), etc., such as narrowband imaging (NBI). Optical magnification is recommended for examination of lesions or pathologies with different characteristics. Proper selection of imaging modes may improve image or video quality and facilitate examination and diagnosis of foreign objects or abnormal tissue of interest during a procedure. Disclosure of Invention Various specialized imaging modalities exist and have been used to enhance detection and/or diagnosis of certain types of pathologies or abnormalities during an endoscopic procedure. For example, image-enhanced endoscopy (IEE) techniques have enabled better detection and management of colorectal cancer and other pathologies. Although there are currently specialized imaging modality recommendations available depending on the type of pathology studied, the transition between such imaging modalities during an endoscopic procedure is still highly manual and depends on the clinician both identifying the current type of pathology being viewed and recall which imaging modality is recommended for viewing such pathology. The operator's experience with operator preferences and reading these advanced endoscopy modes is highly dependent, and the use of these advanced endoscopy modes can lead to variability in diagnostic and therapeutic results. While training on advanced endoscopic imaging can potentially improve overall anomaly detection and enable more accurate optical diagnostics and effective treatment, training can be expensive. The use of IEE techniques in colonoscopy is still operator dependent and requires a great deal of specialized training. The inventors of the present disclosure have identified an unmet need for real-time recommendations of personalized pathology-specific imaging modalities for exam