Search

EP-4346544-B1 - AUGMENTED REALITY URETEROSCOPE SYSTEM

EP4346544B1EP 4346544 B1EP4346544 B1EP 4346544B1EP-4346544-B1

Inventors

  • CHENG, Jason, Jishen
  • TAN-FAHED, Brendan
  • SANGIORGIO, John, D.
  • YANG, Wanfei
  • HESS, Paul, R.

Dates

Publication Date
20260506
Application Date
20220603

Claims (15)

  1. A ureteroscope system (100) comprising: a ureteroscope (110) configured for insertion into a urinary tract of a patient body, the ureteroscope (110) comprising: an elongate flexible shaft (115), and a camera disposed at a distal end of the shaft (115); and an image processing module (120) operatively coupled with the ureteroscope (110), the image processing module (120) comprising a console (125) including one or more processors (130) and a non-transitory computer-readable medium (140) having stored thereon logic (141, 142) that, when executed by the one or more processors (130), is configured to perform operations comprising: receiving imaging data including first images (201), wherein the first images (201) include a plurality of objects (205, 211-214) including a first subset of the plurality of objects (205, 211-214) that obstructs visibility of one or more objects (211-214) of a second subset of the plurality of objects (205, 211-214), the plurality of objects (205, 211-214) including fragments of a kidney stone, and generating at least one second image (202) including the second subset of the plurality of objects (205, 211-214) in which the second subset of the plurality of objects are visibly unobstructed, and causing rendering of the first images (201) or the at least one second image (202) on a display (105), wherein the second subset includes objects (211-214) that are persistent within the first images (201), and the first subset includes objects (205) that are transient within the first images (201).
  2. The system (100) of claim 1, wherein the operations further comprise causing the rendering of the at least one second image (202) on the display (105).
  3. The system (100) of any of claims 1-2, wherein: the imaging data comprises video imaging data, and the first and second images (201, 202) comprise video images.
  4. The system (100) of any of claims 1-3, wherein the operations further comprise removing the first subset from the first images (201) to define the at least one second image (202).
  5. The system (100) of any of claims 1-4, wherein the operations further comprise tracking the locations of one or more objects within the first images (201) or the at least one second image (202).
  6. The system (100) of claim 5, wherein the operations further comprise defining a tracking image overlay, the tracking image overlay including tracking indicia associated with the tracked objects, preferably wherein the operations further comprise causing rendering of the tracking image overlay on top of the first images (201) or the at least one second image (202) on the display (105).
  7. The system (100) of any of claims 1-6, wherein the operations further comprise identifying circumferential edges of one or more objects within the first images (201) or the at least one second image (202).
  8. The system (100) of claim 7, wherein: the operations further comprise defining an edge-highlighting image overlay, and the circumferential edges are highlighted in the edge-highlighting image overlay.
  9. The system (100) of claim 8, wherein the operations further comprise causing rendering of the edge-highlighting image overlay on top of the first images (201) or the at least one second image (202) on the display (105).
  10. The system (100) of any of claims 7-9, wherein the operations further comprise defining sizes of one or more objects within the first images (201) or the at least one second image (202), the size defined by calculating an area enclosed by the circumferential edge.
  11. The system (100) of claim 10, wherein: the operations further comprise defining a sizing image overlay, the sizing image overlay including size indicia associated with the sized objects, and each size indicium provides a visual indication of the respective size of the object.
  12. The system (100) of claim 11, wherein the operations further comprise causing rendering of the sizing overlay on top of the first images (201) or the at least one second image (202) on the display (105), preferably wherein the operations further comprise: comparing each calculated area with an area limit stored in the non-transitory computer-readable medium (140), and modifying the size indicium if the respective calculated area exceeds the area limit.
  13. The system (100) of any of claims 7-12, wherein the operations further comprise defining maximum lengths of one or more objects within the first images (201) or the at least one second image (202), each maximum length defined by a maximum distance between two points of the circumferential edge of the respective object.
  14. The system (100) of claim 13, wherein the operations further comprise defining a length image overlay, the length overlay including a line indicium visually representing each respective maximum length.
  15. The system (100) of claim 14, wherein the operations further comprise causing rendering of the length image overlay on top of the first images (201) or the at least one second image (202) on the display (105).

Description

BACKGROUND Kidney stones may be treated in various ways. Small kidney stones may pass through the urinary tract without treatment. Larger kidney stones or kidney stones that block the urinary tract may need to be removed via a medical procedure. Laser lithotripsy is a procedure for removing a calculus (e.g., a kidney stone) from the urinary tract of the patient. Laser lithotripsy includes inserting a laser optical fiber through the urinary tract to the calculus. The laser is then activated to break the calculus into small pieces that can be passed naturally by the patient or removed by a retrieval instrument. A typical procedure includes inserting a ureteroscope through the urethra, bladder, ureter and if necessary, into the kidney so that a distal tip of the scope is positioned adjacent the calculus. The laser optical fiber is inserted through a working channel of the ureteroscope to the calculus. The laser is then activated to break up the calculus into fragments small enough to be retrieved via a retrieval device such as a basket device or to be passed naturally by the patient through the urinary tract. During laser ablation of the kidney stone, small fragments (e.g., fast moving dust particles) may be separated from the stone and suspended in urinary fluid. The small fragments may be so numerous so as to affect the visibility of objects through the urinary fluid. In some instances, the small fragments may significantly obstruct the view of larger fragments causing difficulty in further performance of the lithotripsy process. Such difficulty may inhibit identification and tracking of larger stone fragments that require further ablation. In some instances, an operator may need to pause the ablation process to reacquire visibility and retarget larger stone fragments. Prior art document US 2021/150706 discloses systems and methods for processing electronic images from a medical device, such as a ureteroscope, comprising receiving an image frame from the medical device, and determining a first colour channel and a second colour channel in the image frame. A location of an electromagnetic beam halo may be identified by comparing the first colour channel and second colour channel. Edges of an electromagnetic beam may be determined based on the electromagnetic beam halo, and size metrics of the electromagnetic beam may be determined based on the edges of the electromagnetic beam. A visual indicator on the image frame may be displayed based on the size metrics of the electromagnetic beam. In prior art document US 2013/002844 an endoscope apparatus is employed, which is provided with an image generating portion that generates an image of an subject; an image-saving memory portion that saves a real-time image; a forceps-region extracting portion that extracts a forceps region, in which forceps exist, from the real-time image; an image-position aligning portion that aligns positions of the saved image saved in the image-saving memory portion and the real-time image; a forceps-region extracting portion that extracts a region corresponding to the forceps region from the saved image saved in the image-saving memory portion; and an image combining portion that combines an image of the region extracted by the forceps-region extracting portion and the real-time image. Accordingly, disclosed herein are ureteroscope systems and methods that enhance the visibility of objects such as kidney stone fragments via a ureteroscope, tracking the fragments, and assessing a size of fragments during a laser lithotripsy procedure. SUMMARY OF THE INVENTION The invention is defined in the appended claims. Briefly summarized, the invention relates to a ureteroscope system including a ureteroscope configured for insertion into a urinary tract of a patient body. The ureteroscope includes an elongate flexible shaft having a camera disposed at a distal end thereof and an image processing module operatively coupled with the ureteroscope. The module includes a console having one or more processors and a non-transitory computer-readable medium. Logic stored on the medium, when executed by the one or more processors, is configured to perform various operations as summarized below. The operations include receiving imaging data including first images. The first images includes a plurality of objects including a first subset of the plurality of objects that obstructs the visibility of one or more objects of a second subset of the plurality of objects in the first image. The operations include generating at least one second image including the second subset of the plurality of objects in which the second subset of the plurality of objects are visibly unobstructed. The operations include rendering the first image or the second image on a display of the system. The imaging data may include video imaging data, and the first and second images may include video images. The objects include fragments of a kidney stone The operations may further include remo