US-12616552-B2 - Marker unit for use in AR aided surgery

US12616552B2US 12616552 B2US12616552 B2US 12616552B2US-12616552-B2

Abstract

Disclosed is a marker unit for use in augmented reality aided surgery, and in particular laparoscopic liver surgery. The marker unit comprises a flat body with two main surfaces. One of the main surfaces is provided with a set of optically detectable markers, and the opposite main surface is arranged to be connected to a bodily organ, and preferably provided with an adhesive. The marker unit further comprises a set of radiopaque markers that are visible by a medical imaging system. Both the optically detectable markers and the radiopaque markers are provided in a geometrical pattern not having rotational symmetry, thereby allowing the rotational position of the marker unit to be determinable. The geometrical patterns of the optically detectable markers and the radiopaque markers have a fixed, predetermined correlation.

Inventors

Axel BLOMÈ
Carl BODIN
Christian AL-MALEH
Klas MODIN
Lisa MÅNSSON
Madeleine Gustavsson
Mårten FALKENBERG
Niclas KVARNSTRÖM
Torbjörn Lundh

Assignees

NAVARI SURGICAL AB

Dates

Publication Date: 20260505
Application Date: 20230222
Priority Date: 20220225

Claims (14)

1 . A marker unit for use in augmented reality aided laparoscopic surgery of a bodily organ, the marker unit being made of a rigid material and comprising: a flat body with two main surfaces, one of the main surfaces being provided with a set of optically detectable markers and the opposite main surface being provided with an adhesive and arranged to be connected to the bodily organ; and a set of radiopaque markers visible by a medical imaging system; wherein both the optically detectable markers and the radiopaque markers are provided in a geometrical pattern not having rotational symmetry, thereby allowing the rotational position of the marker unit to be determinable; wherein the geometrical patterns of the optically detectable markers and the radiopaque markers have a fixed, predetermined correlation; wherein the flat body comprises two layers fixedly connected to each other; wherein the set of radiopaque markers is arranged sandwiched between said two layers; wherein the main surface comprising the set of optically detectable markers is hydrophobic and non-reflecting; and wherein the main surfaces have dimensions in two orthogonal direction in the range of 5-20 mm.
2 . The marker unit of claim 1 , wherein the flat body has a polygonal shape.
3 . Ther marker unit of claim 2 , wherein the flat body has a generally rectangular shape or square shape.
4 . The marker unit of claim 1 , wherein the main surface has dimensions in two orthogonal directions in the range of 5-15 mm or 7-12 mm.
5 . The marker unit of claim 1 , wherein the set of optically detectable markers comprises a plurality of geometrical figures distributed over the main surface.
6 . The marker unit of claim 5 , wherein the plurality of geometrical figures comprises a plurality of QR codes.
7 . The marker unit of claim 1 , wherein a maximal extension in any direction in the marker unit is less than 12 mm or less than 10 mm.
8 . The marker unit of claim 1 , wherein the marker unit is rigid.
9 . The marker unit of claim 1 , further comprising a handle for a laparoscopic tool.
10 . The marker unit of claim 9 , wherein the handle is arranged on a part of the flat body being free of the adhesive and/or the optically detectable markers.
11 . The marker unit of claim 9 , wherein the handle comprises one or more protruding portions that protrude in a lateral and/or transverse direction in relation to the main surfaces of the flat body.
12 . A system for augmented reality aided surgery comprising a marker unit in accordance with claim 1 , and further comprising a display and a controller, the controller being arranged to: receive medical imaging data showing the marker unit at a position intended for surgery; receive optically received image data showing the marker at the position intended for the surgery; correlate the medical imaging data and the optically received image data based on the position of the set of radiopaque markers in the medical imaging data and the set of optically detectable markers in the optically received image data; and providing an augmented reality image on the display, the augmented reality image comprising at least in part image data from the medical imaging data and the optically received image data.
13 . The system of claim 12 , wherein the medical imaging data is received form at least one of: computed tomography scan, cone beam computed tomography, magnetic resonance imaging, and ultrasound imaging.
14 . The system of claim 12 , wherein the optically received image data is received from a camera, and preferably a laparoscopic camera.

Description

This patent application is a U.S. national stage filing under 35 U.S.C. § 371 of PCT International Application No. PCT/EP2023/05443 filed Feb. 22, 2023 (published as WO2023/161286 on Aug. 31, 2023), which claims priority to and the benefit of Swedish Patent Application No. SE 2250262-9 filed Feb. 25, 2022. The entire contents of these applications are incorporated herein by reference. TECHNICAL FIELD OF THE INVENTION The present invention relates to a marker unit for use in augmented reality (AR) aided surgery. The invention also relates to a system for augmented reality (AR) aided surgery comprising such a marker unit. BACKGROUND OF THE INVENTION Surgical removal of a cancer tumor in the liver was traditionally done by open surgery. Today there is a big change towards laparoscopic, minimally invasive surgery. However, laparoscopic surgery is a relatively complicated procedure, and requires great skill and much training. In particular, it is often difficult for the surgeon to find his or her way and orient based on the limit visual input available from the laparoscopic camera. Liver surgery is particularly challenging, due to the uniformity of the liver's shape and surface. Furthermore, tumors in the liver are usually located deep inside the liver and not visible with the thin laparoscopic light-camera. In laparoscopic surgery, a number of ports are provided, for access to the operation site with instruments, camera and the like. The surgeons orient themselves from the outside, creating a perception of the inside through the 2D-images from the camera and a preoperative 3D-image on the side as a map. Although laparoscopic surgery has many advantages, it is a difficult technique to learn that needs a lot of practice. It is especially complicated with e.g. liver tumors, since the liver is an unusually homogeneous organ with a smooth surface, making it very hard for the surgeons to orient themselves in the 2D-camera view. In recent years, many attempts have been made to overcome these problems. “Radiopaque fiducials guiding laparoscopic resection of liver tumours” by M. Falkenberg et al, Surg. Laparosc. Endosc. Percutan Tech, Sep. 28, 2021 investigates how radiopaque fiducials and fluoroscopy can complement ultrasound during tumor resection in the liver and provide augmented 3D-information of the liver combined with the information from the camera view. The overall aim of the study was to develop a workflow where the position of the tumor can be traced during the procedure as a complement to the use of ultrasound. “The effect of intraoperative imaging on surgical navigation for laparoscopic liver resection surgery” by A. Teatini et al, Scientific Reports (2019) 9:18687 relates to the so-called CAScination AR solution. It is based on cameras that detects markers outside the body. For this to work, the body and the liver have to be unmoved. Using that, and the information from the earlier taken CT-scan, gives the opportunity (in theory) to display hidden parts. However, in practice it is difficult to remain full stability between all the parts during surgery. In addition, US20200005473 discloses alignment system for liver surgery, WO 2016/170372 discloses an apparatus and method for registering pre-operative image data with intraoperative laparoscopic ultrasound images, and WO 2016/012556 discloses an image generating apparatus method with combination of functional imaging and ultrasound image. Despite these efforts, there is still a need for a more efficient system for guiding a surgeon during laparoscopic surgery, especially for liver surgery. In particular, there is a need for a system which facilitates orientation at the operation site, inside the body, and which can be implemented and used in a relatively fast and cost-efficient manner. SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide a marker unit for use in augmented reality aided surgery, which alleviates all or at least some of the drawbacks of presently known systems. Another object of the invention is to provide a system for AR aided surgery comprising such a marker unit. This object is achieved by means of a marker unit and a system for AR aided surgery as defined in the appended claims. According to a first aspect of the invention, there is provided a marker unit for use in augmented reality aided surgery, the marker unit comprising a flat body with two main surfaces, one of the main surfaces being provided with a set of optically detectable markers and the opposite main surface being arranged to be connected to a bodily organ, and preferably provided with an adhesive, and further comprising a set of radiopaque markers, visible by a medical imaging system, wherein both the optically detectable markers and the radiopaque markers are provided in a geometrical pattern not having rotational symmetry, thereby allowing the rotational position of the marker unit to be determinable, and wherein the geometrical patterns o