Search

US-20260124004-A1 - SURGICAL ASSISTANT SYSTEM BASED ON IMAGE DATA OF THE OPERATIVE FIELD

US20260124004A1US 20260124004 A1US20260124004 A1US 20260124004A1US-20260124004-A1

Abstract

A method and system for assisting a physician compute the 3D location of the tip of a surgical device inserted into a patient is described. A trained model computes the 3D location of the hidden portion of the surgical device based on live image data of the operative field. A display shows the tip of the surgical device and a 3D model of the body organ in a fused arrangement.

Inventors

  • Thomas M. Keast

Assignees

  • BRONCUS MEDICAL INC.

Dates

Publication Date
20260507
Application Date
20251231

Claims (20)

  1. 1 . A system for assisting a physician compute the 3D location of a tip of a surgical device inserted into an organ of a patient comprising: a processor programmed and operable to compute the 3D location of the device tip based on operative-field image data arising from a plurality of cameras arranged outside of the patient.
  2. 2 . The system as recited in claim 1 , further comprising a detection module and classification module for, respectively, detecting and classifying objects in the operative field.
  3. 3 . The system as recited in claim 2 , wherein the objects are selected from the group comprising a bronchoscope, the surgical device, markers applied to the patient, bronchoscope or surgical device, patient and physician's anatomy.
  4. 4 . The system as recited in claim 3 , wherein the processor is operable to extract features from the objects, and the 3D location computation is based on the evaluating the extracted features from the objects.
  5. 5 . The system as recited in claim 4 , wherein the processor is further programmed and operable to compute a motion signature based on tracking one the extracted features over time, and the 3D location computation is based on evaluating the motion signature.
  6. 6 . The system as recited in claim 5 , wherein the object is an ablation device, and the feature is a portion of a handle of the ablation device.
  7. 7 . The system as recited in claim 4 , wherein the 3D location computation is further based on user input.
  8. 8 . The system as recited in claim 4 , further comprising an external camera tracking sensor for computing the location of the cameras, and wherein the 3D location computation is further based on tracking the location of the plurality of cameras during the procedure.
  9. 9 . The system of claim 1 , wherein one or more of the plurality of cameras are visible light spectrum cameras.
  10. 10 . The system as recited in claim 1 , wherein the 3D location computation is further based on live fluoroscopic image data and tracking the location of the fluoroscopic camera.
  11. 11 . The system as recited in claim 1 , further comprising a library of prior device data comprising a plurality of different device profiles, and wherein the 3D location computation is based on a device profile from the library.
  12. 12 . The system as recited in claim 2 , wherein the processor is further programmed and operable to transpose the operative field image data to a common perspective view prior to the detecting.
  13. 13 . The system as recited in claim 1 , wherein the processor is further programmed and operable to evaluate and weight operative-field image data of each of the plurality of cameras for reliability, and to base the 3D location computation on the weighted operative-field image data.
  14. 14 . The system as recited in claim 13 , wherein evaluating and weighting the operative-field image data of each of the plurality of cameras for reliability comprises determining (a) the number of objects present in each camera's FOV, (b) the number of markers present in each camera's FOV, (c) the number of features present in each camera's FOV, and/or (d) the unobstructed area of objects in each camera's FOV.
  15. 15 . A method for assisting a physician perform a medical procedure on a patient with tracking and guidance of a medical device comprises locating a hidden portion of the medical device in an organ of the patient based on live camera images of the operative field.
  16. 16 . The method as recited in claim 15 , wherein the step of locating comprises locating a tip of a bronchoscope, and locating a portion of a medical device advanced through the bronchoscope.
  17. 17 . The method as recited in claim 15 , wherein the locating is based on at least one visible marker applied to an unburied portion of the medical device, anatomy of the patient, or anatomy of the physician.
  18. 18 . The method as recited in claim 15 , further comprising, prior to the locating, selecting a device-trained model from a library of different device-trained models based on the type of medical device being advanced into the organ of the patient.
  19. 19 . The method as recited in claim 18 , further comprising extracting at least one feature from the object, and computing a motion signature of the extracted feature based on tracking the extracted feature over time, and wherein the locating step is based on the motion signature.
  20. 20 . The method as recited in claim 15 , further comprising evaluating and weighting operative-field image data of each of the plurality of cameras for reliability, and basing the locating on the weighted operative-field image data, and wherein the evaluating and weighting operative-field image data of each of the plurality of cameras for reliability comprises determining (a) the number of objects present in each camera's FOV, (b) the number of markers present in each camera's FOV, (c) the number of features present in each camera's FOV, and/or (d) the unobstructed area of objects in each camera's FOV.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This is a continuation application of application Ser. No. 17/973,661, filed Oct. 26, 2022, entitled “SURGICAL ASSISTANT SYSTEM BASED ON IMAGE DATA OF THE OPERATIVE FIELD” and claims priority to provisional application No. 63/276,393, filed Nov. 5, 2021, entitled “SURGICAL ASSISTANT SYSTEM BASED ON IMAGE DATA OF THE OPERATIVE FIELD.” FIELD OF THE INVENTION The present invention relates to surgical procedures and in particular, to assisting physicians with tracking and guidance during bronchoscopy-based surgical procedures. BACKGROUND OF THE INVENTION Minimally invasive surgery is surgery performed with only a small incision or no incision at all and is typically performed with an endoscope, bronchoscope, laparoscope, or like instrument. In a bronchoscopic procedure, for example, a bronchoscope is inserted through the nose or mouth of the patient, advanced through the trachea and into a desired airway. The surgery may then be performed through the working lumen of the bronchoscope. A light source and camera at the tip of the bronchoscope enables the physician to observe the airway wall in real time. A skilled physician can identify their location along the airway and navigate to the desired location along the airway wall. It is often desirable, however, to supplement endoscopic visualization with radiological guidance (e.g., by taking real time X-ray images of the region with a fluoroscope). In certain procedures radiologic guidance is necessary. In a transbronchial needle aspiration (TBNA) procedure, for example, a long flexible catheter comprising a needle at the tip is advanced through the working lumen of the bronchoscope to the target site. In sampling lymph nodes, an ultrasound equipped scope (e.g., an endobronchial ultrasound (EBUS)-type bronchoscope) can be used to visualize the needle entering the lymph node in close proximity to the airway wall. For other targets, or if an EBUS scope is not available, a standard bronchoscope can be used. If desired, the needle is advanced through the airway wall outside of view of the bronchoscope to aspirate a sample of tissue. It is highly desirable or necessary to have fluoroscopy or an alternative means to view and track the needle once it is outside of view of the bronchoscope or ultrasound vision. Locating or tracking the location of devices inside the patient using a fluoroscope, however, is not straightforward. To track a device, multiple 2D X-ray images from at least two different fluoroscopic camera views are taken. Based on the information provided by these two images, the physician determines the position of the device. Determining the position based on two 2D X-rays relies on the skill and experience of the physician. Even for the most skilled physicians there is a considerable degree of uncertainty. This is undesirable. One approach to address the above-mentioned problem is described in U.S. Pat. No. 6,947,788 to Gilboa. According to the Abstract, “[a] catheter, including: a housing having a transverse inner dimension of at most about two millimeters; and a coil arrangement including five coils and five solid cores. Each of the coils is wound around one of the solid cores. The coils are non-coaxial. The coil arrangement is mounted inside the housing.” The different sensors are used for sensing different components of an electromagnetic field. A controller/processor implements an algorithm to infer the position and orientation of probe. A drawback to the technique described in Gilboa, however, is the requirement of setting up an electromagnetic field system in the operating room. Another approach to address the above-mentioned problem is described in U.S. Pat. No. 9,265,468 to Rai. According to the Abstract, “[a] method for assisting a physician track a surgical device in a body organ of a subject during a procedure includes fluoroscopic based registration, and tracking. An initial registration step includes receiving a 3D image data of a subject in a first body position, receiving a real time fluoroscopy image data, and estimating a deformation model or field to match points in the real time fluoro image with a corresponding point in the 3D model. A tracking step includes computing the 3D location of the surgical device based on a reference mark present on the surgical device, and displaying the surgical device and the 3D model of the body organ in a fused arrangement.” A drawback to the technique described in Rai, however, is the reliance on x-ray imaging. It is desirable to limit the use of x-ray imaging. Notwithstanding the above, a method and system to assist surgeons track surgical devices in a body organ such as the lung and that does not suffer the above identified drawbacks is therefore desired. SUMMARY OF THE INVENTION A bronchoscopic method for assisting a physician determine the location of the working end of a medical apparatus or device inserted into a patient is based on live images of the operativ