JP-7855597-B2 - Surgical assistance system and display method equipped with a surgical microscope and camera

JP7855597B2JP 7855597 B2JP7855597 B2JP 7855597B2JP-7855597-B2

Inventors

アミールサルベスタニ

Assignees

ビー．ブラウンニューベンチャーズゲーエムベーハー

Dates

Publication Date: 20260508
Application Date: 20220131
Priority Date: 20210201

Claims (18)

A surgical support system used in surgical interventions on patients, wherein the support system is A surgical microscope, A movable microscope head, including a housing and an optical system adapted to provide optical magnification of the area of interest in the direction of the optical microscope axis and to generate a digital microscope image via a downstream microscope image unit, A microscope comprising: a movable microscope arm connected to a base, on which a movable microscope head is mounted, and which is adapted to adjust the position and/or orientation of the microscope head; At least one display device for displaying visual content, The system comprises a control unit adapted to process the microscope image and appropriately control the display device for display, The surgical microscope further comprises a camera system having at least one ambient camera positioned on the microscope head, The surrounding camera is configured such that its field of view includes the field of view of the optical system in order to detect both the region targeted by the optical system and the surrounding area around the targeted region, and to provide a surrounding image. The control unit is further adapted to process the ambient image, generate a composite view having the microscope image and the ambient image, and output it visually by the display device. The surgical support system further comprises a storage unit having preoperative 3D image data, A surgical assistance system characterized in that the control unit of the assistance system is adapted to spatially detect the 3D structure of the patient using the camera system and to associate the detected 3D structure with the 3D image data in order to register the patient.
To detect a 3D structure, the camera system includes a 3D camera and/or a 2D camera. The surgical assistance system according to claim 1, characterized in that, in the case of the 2D camera, the 2D camera is moved over the patient's region of interest to obtain various views of the region of interest, and the control unit is adapted to calculate a 3D structure from the various views using image analysis.
The auxiliary system is configured to detect the position of the microscope head and provide it to the control unit. The control unit is configured to store the position of the microscope head in the storage unit as a registered position and associated ambient image. The surgical assistance system according to claim 1, characterized in that when updating the surrounding image at the stored registered position, the control unit determines that a misalignment exists if the superposition of the stored surrounding image and the newly detected surrounding image is misaligned by more than a predetermined threshold.
The surgical microscope is equipped with at least one actuator to actively move the movable microscope arm and the movable microscope head. The surgical assistance system according to claim 1, further characterized in that the control unit is further adapted to control the actuator to actively move the microscope head to a predetermined position and/or orientation.
The auxiliary system includes an input unit for selecting a focus point in the surrounding image. The surgical assistance system according to claim 4, characterized in that the control unit is adapted to actively control the microscope arm and the microscope head via at least one actuator based on the selected focus point, such that the optical microscope axis is aligned with the selected focus point in the patient at a predetermined associated distance and/or a predetermined image angle.
The control unit is configured to detect, via image analysis of the surrounding image and/or a sensor provided on the microscope head, an object within the area of the microscope head and the distance of the movable microscope head to the object, and to emit an alarm signal and/or to limit at least the degrees of freedom of movement of the microscope head (4) if there is a possibility of collision with the object or if the distance falls below a predetermined distance, the control unit is configured to do so. This is the surgical assistance system according to claim 1.
The surgical assistance system according to claim 1, characterized in that the control unit is adapted to correctly detect and continuously track the pose of the medical instrument and/or the microscope head using at least the ambient image and markers and/or patient characteristics of the camera system.
Data containing the geometric relationships of at least one medical device, along with associated usage instructions, are stored in the memory unit. The surgical assistance system according to claim 1, characterized in that the control unit is adapted to detect surgical instruments in the surrounding image based on the geometric relationships of the stored data, and to output the relevant usage instructions to the user via the display device.
The surgical assistance system according to claim 1, characterized in that at least one of the display devices is an OR monitor and/or a head-mounted display and/or a binocular system having superimposed data.
The surgical assistance system according to claim 1, characterized in that the magnification of the optical system is at least 5x, the optical system has a zoom function, and/or the magnification of the ambient camera is up to 5x.
When executed by a computer, the computer will A step of targeting a region through an optical system located on a movable microscope head, A step of generating a microscope image at the magnification provided by the optical system using a microscope image unit located on the microscope head, A step of generating a surrounding image using a surrounding camera positioned on the microscope head, wherein the field of view of the surrounding camera includes the field of view of the optical system. A step of generating a composite view including the microscope image and the surrounding image, The steps of outputting the composite view using a display device, and The three-dimensional structure of the patient is spatially detected by the surrounding camera, and the detected three-dimensional structure is associated with the patient's three-dimensional image data in order to register the patient. A computer-readable storage medium equipped with instructions to perform the following steps.
An operating room , characterized by comprising the surgical support system described in claim 1.
The surrounding camera of the aforementioned camera system comprises an optical camera axis and a camera optical system, The surgical assistance system according to claim 1, characterized in that the microscope head is adapted such that the field of view of the peripheral camera includes the field of view of the optical system, via the orientation of the optical camera axis and the camera optical system.
The surgical assistance system according to claim 1, characterized in that the assistance system is adapted to spatially detect the patient's face as the patient's 3D structure.
The surgical assistance system according to claim 1, characterized in that, in order to register the patient, the detected 3D structure is associated with the 3D image data using only the surrounding camera.
The surgical assistance system according to claim 1, characterized in that the surrounding camera is securely attached to the housing.
The surgical assistance system according to claim 2, characterized in that the camera system is comprised of the surrounding camera as the sole 2D or 3D camera in the camera system.
The auxiliary system is configured to detect the pose of the microscope head and provide it to the control unit. The control unit stores the pose of the microscope head in the storage unit as a registered pose and the associated surrounding image in the form of the detected 3D structure. The surgical assistance system according to claim 3, characterized in that when updating the surrounding image with the stored registered pose, the control unit determines that a misalignment exists if the overlapping portion of the stored surrounding image and the newly detected surrounding image is misaligned by more than a predetermined threshold.

Description

This disclosure relates to a surgical assistance system comprising a surgical microscope for use in surgical interventions/procedures on patients. The surgical microscope includes a housing and a movable microscope head, particularly housed within the housing, and comprising an optical system adapted to provide optical magnification of a target area (intervention area) in the direction of the optical microscope axis, particularly at a distance between 5 cm and 80 cm, particularly preferably between 20 cm and 60 cm, and to create a (digital) microscope image by a microscope image unit particularly equipped with a sensor, and to provide the microscope image in a digitally/computer-readable format. Furthermore, the surgical microscope includes an articulated/movable microscope arm at its base, on which the movable microscope head is positioned and particularly supported, and the microscope arm is adapted to adjust the position and/or orientation of the microscope head in space. The surgical assistance system further comprises at least one display device for displaying visual content and a control unit adapted to process the microscope image and control the display device for corresponding display. Furthermore, this disclosure relates to an image display method, storage medium, and sterile space relating to the preamble of the claims. Surgical microscopes are standard instruments used in surgical interventions, particularly in neurosurgery, spinal surgery, and microsurgery. These surgical microscopes typically feature binocular optics, allowing surgeons to directly view optically magnified microscopic images/photographs, or to view magnified images/photographs via display devices/external notification devices such as OR monitors for visually displaying (digitally) recorded microscopic images. However, in either case, surgeons view only the microscopic image of the intervention area at very high magnification through binoculars or an OR monitor. As a result, surgeons cannot, or at least find it difficult, see the surrounding surgical area/field/intervention area (of the microscopic image) to confirm the position of their hands or the medical instruments used during the surgical intervention. Only the tips of surgical instruments, when properly positioned, are within the field of view of a surgical microscope. However, viewing the area around the microscope image is crucial to avoiding damage outside the actual intervention area. To view the area around the microscope, surgeons must quickly visually survey the area being treated by the surgical microscope before returning to the microscope image, which is a significant burden during intervention. Constantly switching between different field-of-view directions of two field-of-view modalities and maintaining a constant correlation in the surgeon's mind is not only tiring but often leads to unexpected errors. Furthermore, surgical microscopes must be positioned in the operating room so that, on the one hand, the surgeon in particular can easily handle and look through the microscope, and can view the OR monitor with a good field of view, and on the other hand, can clearly see the surgical field/operational area/intervention area. Therefore, the OR monitor must be positioned so that the surgeon has a good field of view of both modalities, but this limits the flexibility of the operating room's layout. Also, because the volume of the microscope head usually prevents direct viewing of the patient, it is difficult for the surgeon to directly view the intervention area. Moreover, this position and the corresponding good field of view are only available to the surgeon; other surgical staff in other positions in the operating room have difficulty seeing the microscope image and intervention area displayed on the OR monitor. Therefore, medical staff are constantly forced to change their head position and viewing direction when they are in an unfavorable position. For example, US 201902901 A1 discloses an auxiliary system comprising a surgical microscope used in combination with a portable visualization system mounted on the surgeon's head. This visualization system allows the surgeon to view either a microscopic image or a type of magnified image/magnification display, depending on the field of view of their eye. While this allows the surgeon to switch between two modalities, such visualization systems have limitations and difficulties. Because the optics are integrated, the visualization system is heavy, expensive to manufacture, and prone to errors because the beam path to the surgeon's eye must always be precise. Furthermore, the surgeon loses their sense of their surroundings due to the visualization system. For example, US 9,936,863 B2 discloses a system comprising a surgical microscope with an additional camera mounted on a wound dilator on the patient, specifically to provide the surgeon with a central microscope image from the surgical microscope and a peripheral im