Search

US-12622751-B2 - Determining relative 3D positions and orientations between objects in 2D medical images

US12622751B2US 12622751 B2US12622751 B2US 12622751B2US-12622751-B2

Abstract

Systems and methods are provided for processing X-ray images, wherein the methods are implemented as a software program product executable on a processing unit of the systems. Generally, an X-ray image is received by the system, the X-ray image being a projection image of a first object and a second object. The first and second objects are classified, and a respective 3D model of the objects is received. At the first object, a geometrical aspect like an axis or a line is determined, and at the second object, another geometrical aspect like a point is determined. Finally, a spatial relation between the first object and the second object is determined based on a 3D model of the first object, a 3D model of the second object, and the information that the point of the second object is located on the geometrical aspect of the first object.

Inventors

  • Arno Blau

Assignees

  • METAMORPHOSIS GMBH

Dates

Publication Date
20260512
Application Date
20210426
Priority Date
20191217

Claims (11)

  1. 1 . A system for processing X-ray images, the system comprising a processing unit and a software program product, wherein the software program product when executed by the processing unit causes the system to: receive an X-ray image, the X-ray image being a projection image of a first object and a second object; classify the first object and receive a 3D model of the first object, determine a geometrical aspect of the first object and identify the geometrical aspect in relation to the 3D model of the first object; classify the second object and receive a 3D model of the second object, select a point of the second object and identify the point at the 3D model of the second object; determine a relative 3D position and 3D orientation between the first object and the second object, with the first object being a bone nail inserted in a bone, based on: (i) the 3D model of the first object; (ii) the 3D model of the second object; (iii) an at least partial 3D reconstruction of a bone surface of the bone with known spatial relation to the geometrical aspect of the first object; and (iv) with the point of the second object being positioned at the bone surface, wherein the software program product when executed by the processing unit further causes the system to determine the at least partial 3D reconstruction of a bone surface of the bone with known spatial relation to the geometrical aspect of the bone nail based on (i) a registration of a further X-ray image with the X-ray image, wherein the further X-ray image shows the bone nail inserted in the bone, wherein an imaging direction of the further X-ray image differs from an imaging direction of the X-ray image.
  2. 2 . The system of claim 1 , wherein the further X-ray image further shows the second object and wherein the software program product when executed by the processing unit further causes the system to determine the at least partial 3D reconstruction of the bone surface of the bone with known spatial relation to the geometrical aspect of the bone nail based on (ii) with that the point of the second object being at the same 3D position relative to the first object in both X-ray images.
  3. 3 . The system of claim 1 , wherein the software program product when executed by the processing unit further causes the system to determine a deviation of the 3D position and 3D orientation of the second object from an intended spatial relation of the second object relative to the geometrical aspect of the first object.
  4. 4 . The system of claim 1 , wherein the second object is a tool or a second implant.
  5. 5 . The system of claim 1 , wherein the point of the second object is a tip of the object, the tip being in contact with the bone surface.
  6. 6 . The system of claim 1 , the system further comprising a C-arm based X-ray imaging device for generating the X-ray image.
  7. 7 . A system for processing X-ray images, the system comprising a processing unit and a software program product, wherein the software program product when executed by the processing unit causes the system to: receive a first X-ray image, the first X-ray image being a projection image of a first object and a second object; classify the first object and receive a 3D model of the first object, determine a geometrical aspect of the first object and identify the geometrical aspect in relation to the 3D model of the first object; classify the second object and receive a 3D model of the second object, select a point of the second object and identify the point at the 3D model of the second object; and determine a relative 3D position and 3D orientation between the first object and the second object, with the first object being a bone nail inserted in a bone, based on; (i) the 3D model of the first object; (ii) the 3D model of the second object; (iii) registration of a second X-ray image with the first X-ray image, wherein both X-ray images show both the first and the second object; and (iv) with the point of the second object being at the same 3D position relative to the first object in both X-ray images, wherein the software program product when executed by the processing unit further causes the system to determine a position of an at least partial 3D reconstruction of the bone surface relative to the first object based on (i) the registration of the second X-ray image with the first X-ray image, (ii) a detection of bone edges in both X-ray images, and (iii) with the point of the second object being positioned at the bone surface.
  8. 8 . The system of claim 7 , wherein the software program product when executed by the processing unit further causes the system to determine a deviation of the 3D position and 3D orientation of the second object from an intended spatial relation of the second object relative to the first object.
  9. 9 . The system of claim 7 , wherein the second object is a tool or a second implant.
  10. 10 . The system of claim 7 , wherein the point of the second object is a tip of the object and wherein the information of the 3D location of said tip is a point of contact of the tip with the bone surface.
  11. 11 . The system of claim 7 , the system further comprising a C-arm based X-ray imaging device for generating the X-ray image.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation of International Application No. PCT/EP2020/086503 filed on Dec. 16, 2020, which claims the benefit and priority of EP 19217245.0 filed on Dec. 17, 2019. The entire disclosure of each of the above applications are incorporated herein by reference. TECHNICAL FIELD The invention relates to the fields of artificial intelligence and computer assisted surgery. In particular, the invention relates to a device and a method for determining 3D representations and relative 3D positions and relative 3D orientations between objects based on an X-ray projection image. The method may be implemented as a computer program executable on a processing unit of the device. BACKGROUND In orthopedics or orthopedic trauma surgery or spinal surgery, it is a common task to aim for a target object or target structure (as part of a target object) with a relatively thin instrument. Target structures may be anatomical (e.g., a pedicle) or parts of other instruments or implants (e.g., a distal locking hole of a long antegrade intramedullary nail). In general, the goal may be to determine the 3D relative position and 3D relative orientation between instrument and target object. Based on available intraoperative 2D imaging techniques, this may be challenging. It is particularly difficult if the precise geometry of the target object is unknown, and/or if the instrument is known, but not uniquely localizable in 3D space based on the 2D X-ray image. In orthopedics or orthopedic trauma surgery or spinal surgery, it is a common task to aim for a target object or target structure (as part of a target object) with a relatively thin instrument. Target structures may be anatomical (e.g., a pedicle) or parts of other instruments or implants (e.g., a distal locking hole of a long antegrade intramedullary nail). In general, the goal may be to determine the 3D relative position and 3D relative orientation between instrument and target object. Based on available intraoperative 2D imaging techniques, this may be challenging. It is particularly difficult if the precise geometry of the target object is unknown, and/or if the instrument is known, but not uniquely localizable in 3D space based on the 2D X-ray image. For surgical procedures, preoperative CT scans may be performed, which allow a more precise planning of the procedure. This is the case, for instance, when operating within a complex 3D structure, or when drilling or placing screws within narrow anatomical structures or in the vicinity of critical structures (e.g., spinal cord, nerves, aorta). Typical examples of such procedures are the placements of sacroiliac or pedicle screws. When the target structure is a tool or an implant, its 3D geometry is typically known: An example is the distal locking procedure, where a 3D model of, or 3D information about the target object (nail), and in particular the target structure “distal locking hole” (a cylinder) is available. However, for the surgeon to utilize this 3D information and to apply it to intraoperative 2D X-ray images requires a high level of spatial perception and imagination. In some cases and for some procedures, it may be possible to determine, for instance, the direction of drilling by aligning it with a particular viewing direction of the imaging equipment (e.g., for the distal locking procedure, a true medial-lateral view). Yet ensuring that the drilling indeed proceeds precisely along this direction is not generally possible. This will now be illustrated for the example of distal locking of a long antegrade intramedullary nail. In the conventional distal locking procedure of an antegrade nail, the surgeon moves the C-arm into a true lateral position, which means that the hole to be locked appears perfectly round in the X-ray image. This positioning is tedious and time-consuming, possibly taking several minutes, because it is done iteratively: it typically requires the acquisition of 5 to 20 X-ray images with corresponding re-adjustments of the C-arm. A faster way of achieving this positioning is to use the fluoroscopic mode of the C-arm (producing a continuous X-ray video stream), but this leads to a higher X-ray dose. Moreover, in order to ensure high accuracy for distal locking, not only must the hole appear round, but it also must be close to the center of the X-ray image. However, in practice, once the hole appears round enough in the X-ray image, this C-arm position is typically used for distal locking even if the hole is not close to the center of the X-ray image. Due to a cone shape of the X-ray beam fan, the direction of X-ray beams is inclined the further the beams are away from the center of the X-ray image. Thus, drilling through a hole should be in the direction of the focal point of the X-ray source and not parallel to a center line between X-ray source and detector. In a next step, the tip of the drill may be placed on the intended drill location and