CN-121987346-A - Ankle arthroscope noninvasive navigation method based on anatomical constraint and binocular vision

Abstract

The invention discloses an ankle arthroscope noninvasive navigation method based on anatomical constraint and binocular vision, which mainly comprises the steps of reconstructing a three-dimensional skeleton triangular grid model, extracting talus pulley joint shafts, defining patient body surface mark points and deploying binocular vision environments, detecting the mark points on the skin surface of a patient's operation side ankle in real time, performing three-dimensional mapping, calculating a physiological effective rotation angle based on the real-time posture of the ankle of the anatomical constraint, and finally realizing ankle arthroscope noninvasive navigation through digital twin synchronization and auxiliary decision making. The method is completely noninvasive and non-radiative, does not need to be subjected to perspective or implantation of a marker in operation, and effectively reduces the complexity of the operation. Noise data generated by skin sliding can be identified and stripped by introducing a manifold projection algorithm and a biomechanical constraint mechanism, and effective suppression of the STA is realized in noninvasive tracking. In addition, the lightweight gesture recognition network ensures the real-time performance of the system, does not need expensive optical tracking equipment, reduces the deployment cost and has clinical popularization value.

Inventors

• XING YUAN
• LI YULONG
• Shi Zitan
• ZHAO JIANCHANG

Assignees

• 天津大学

Dates

Publication Date

20260508

Application Date

20260319

Claims (1)

1. 1. An ankle arthroscope noninvasive navigation method based on anatomical constraint and binocular vision is characterized by comprising the following steps of: step 1) reconstructing a three-dimensional skeleton triangular mesh model: step 1-1) acquiring CT image data of a patient pre-operation ankle joint, and extracting tibia, talus, fibula, calcaneus, navicular bone, cuboid, metatarsal and cuneiform from the CT image data of the ankle joint through a threshold segmentation and region growing algorithm by using open source software 3D slicer; Step 1-2) reconstructing the extracted bone segmentation result by using open source software 3D slicer to generate a three-dimensional bone triangular grid model, wherein the three-dimensional bone triangular grid model comprises a talus three-dimensional bone triangular grid model, a three-dimensional bone triangular grid model formed by tibia and fibula and a three-dimensional bone triangular grid model formed by talus, calcaneal, navicular, cuboid, metatarsal and cuneiform, and a three-dimensional bone triangular grid model formed by talus, calcaneal, navicular, cuboid, metatarsal and cuneiform and a cuneiform; Presetting virtual operation planning information in one or two models of the leg model and the foot model, wherein the information comprises information such as preset operation access points, key anatomical points, anatomical structure early warning areas and the like; Step 2) extracting the talus pulley joint axis (TJA): Step 2-1) performing point cloud sampling on the talus three-dimensional skeleton triangular mesh model, and automatically dividing a talus pulley joint surface point cloud by utilizing normal filtering and curvature filtering based on an anatomical principle; Step 2-2) constructing an objective function to perform best cone fitting calculation according to the anatomical characteristics of the talus joint surface approximate cone based on the talus joint surface point cloud, Wherein, the In order to be a radial distance from each other, For the axial distance it is the axial distance, Is a conical half cone angle; The cone parameters include cone half cone angle Determining the opening amplitude of the cone, the axial distance Is a point in a point cloud To the cone top In the axial direction Projection distance on, i.e. The radial distance Is a dot To the end by And A defined vertical distance of the central axis; Step 2-3) solving the optimal cone parameters by using a BFGS quasi-Newton optimization algorithm, thereby extracting a cone central axis vector The central axis vector of the cone As fitted talar joint axial quantity Extracting the talus pulley joint shaft; Step 3) defining body surface mark points of a patient and deploying a binocular vision environment: Step 3-1) predefining a mark point group on the ankle of the patient on the operation side, wherein the mark point group comprises 6 mark points, namely a first tibia point (0), a second tibia point (1), an inner ankle (2), a navicular bone (3), a first metatarsal bone (4) and a calcaneus bone (5); step 3-2) performing visual environment deployment, respectively drawing self-defined geometric patterns on 6 marker points on the skin surface of a patient by using a medical sterile pen; Step 4) detecting mark points on the skin surface of the ankle on the operation side of the patient in real time, and performing three-dimensional mapping: Step 4-1) capturing RGB video stream and synchronous depth image in real time by the binocular camera in the deployed visual environment, utilizing a pre-trained light gesture recognition neural network, and extracting two-dimensional pixel coordinates of 6 mark points from an original RGB color image; Step 4-2) combining the information of the synchronous depth image, mapping the two-dimensional point set corresponding to the two-dimensional pixel coordinates of the 6 mark points into an original three-dimensional key point sequence under a camera coordinate system ; Step 5) calculating a physiological effective rotation angle based on the real-time ankle posture of anatomical constraint: Step 5-1) for the original three-dimensional key point sequence Performing sliding window moving average filtering to eliminate high-frequency noise existing in the binocular camera and obtain a smoothed three-dimensional key point sequence ; Step 5-2) based on double rigid body kinematics, smoothing the three-dimensional key point sequence Dividing into a calf group and a foot group, wherein the calf group comprises a mark point which is provided with a first tibia point (0), a second tibia point (1) and a medial malleolus (2), the foot group comprises a mark point which is provided with a navicular bone (3), a calcaneus bone (4) and a first metatarsal bone (5), and the preliminary rotation matrix of the calf group and the foot group relative to the calf model and the foot model is calculated by using a Kabsch algorithm And Further to determine relative motion including soft tissue artifacts , wherein, Representing a real-time relative rotation matrix of the foot model relative to the calf model containing soft tissue slip errors; Step 5-3) Using lie algebra Projection properties, relative motion including soft tissue artifacts as described above Mapping to the fitted talar joint axial amount Calculating the physiological effective rotation angle between the calf model and the foot model on the defined single-degree-of-freedom rotation manifold ; Step 6) digital twin synchronization and decision-making aid Step 6-1) constructing a digital twin ankle model in open source three-dimensional software Blender according to the calf model and the foot model, wherein the digital twin ankle model receives the effective rotation angle The real-time script interface deployed in the Blender receives data and drives the motion of a foot model relative to a lower leg model in the Blender scene in real time by utilizing the corrected angle parameters, so that synchronous high-fidelity mapping of the real-time ankle posture of a patient on the Blender digital twin ankle model is realized; Step 6-2) carrying out real-time dynamic superposition on the real ankle real-time image of the patient in the operation site and the digital twin ankle model real-time motion state obtained in step 6-1) by combining with the virtual operation planning information preset in step 1-2) through a computer, and displaying the real-time dynamic superposition in the visual field of the AR glasses of a doctor, thereby realizing noninvasive navigation of an ankle arthroscope.

Description

Ankle arthroscope noninvasive navigation method based on anatomical constraint and binocular vision Technical Field The invention relates to the field of computer-aided surgery (CAS) and medical image processing, in particular to a noninvasive computer vision positioning, gesture sensing and Augmented Reality (AR) navigation method applied to ankle arthroscope minimally invasive surgery. Background The ankle arthroscopy operation has the advantages of small wound, quick recovery and the like. However, the ankle gap is extremely narrow, and the operation relies on two-dimensional endoscopic images, lacks depth perception, and is difficult for a doctor to establish a precise operation approach, which is prone to cartilage or neurovascular injury. Under the condition of lacking the assistance of a navigation system, the accuracy and the safety of the operation depend on the clinical experience of doctors completely, the learning curve is long, and the problems of access deviation or incomplete lesion cleaning and the like can occur in high-difficulty cases. The existing radiographic image auxiliary and optical positioning navigation technology in the ankle department field mainly has the following defects: (1) Radiation and risk of invasiveness. Traditional perspective navigation has accumulated radiation exposure, and bone needle-based optical navigation requires drilling markers into the diaphysis, adding additional trauma and risk of infection. (2) Soft Tissue Artifacts (STA) cause failure of accuracy. The existing noninvasive body surface tracking scheme is limited by relative sliding (namely STA) between skin and deep bones, so that tracking errors under static and dynamic conditions often exceed a clinical allowable range, and the surgical accuracy requirement is difficult to meet. (3) The system deployment is costly. Traditional navigation relies on expensive infrared optical tracking equipment, is large in size and complex to operate, and is difficult to popularize in primary hospitals. Disclosure of Invention Aiming at the prior art, the invention provides the ankle arthroscope noninvasive navigation system and the ankle arthroscope noninvasive navigation method based on anatomical constraint and binocular vision, which can effectively filter STA through an algorithm on the premise of not nailing bone (noninvasive) and not using complex tracking equipment, so that high-precision dynamic tracking of the ankle of a patient is realized. In order to solve the technical problems, the invention provides an ankle arthroscope noninvasive navigation method based on anatomical constraint and binocular vision, which mainly comprises the following steps: step 1), reconstructing a three-dimensional skeleton triangular mesh model; Step 2) extracting a talus pulley joint shaft; step 3), defining body surface mark points of a patient and deploying binocular vision environments; step 4) detecting mark points on the skin surface of the ankle on the operation side of the patient in real time, and performing three-dimensional mapping; Step 5) calculating a physiological effective rotation angle based on the real-time ankle posture of anatomical constraint; And 6) digital twin synchronization and auxiliary decision making, and finally, realizing noninvasive navigation of the ankle arthroscope. In the invention, the specific content of the step 1) is as follows: step 1-1) acquiring CT image data of a patient pre-operation ankle joint, and extracting tibia, talus, fibula, calcaneus, navicular bone, cuboid, metatarsal and cuneiform from the CT image data of the ankle joint through a threshold segmentation and region growing algorithm by using open source software 3D slicer; Step 1-2) reconstructing the extracted bone segmentation result by using open source software 3D slicer to generate a three-dimensional bone triangular grid model, wherein the three-dimensional bone triangular grid model comprises a talus three-dimensional bone triangular grid model, a three-dimensional bone triangular grid model formed by tibia and fibula and a three-dimensional bone triangular grid model formed by talus, calcaneal, navicular, cuboid, metatarsal and cuneiform, and a three-dimensional bone triangular grid model formed by talus, calcaneal, navicular, cuboid, metatarsal and cuneiform and a cuneiform; Presetting virtual operation planning information in one or two models of the leg model and the foot model, wherein the information comprises information such as preset operation access points, key anatomical points, anatomical structure early warning areas and the like; in the invention, the specific content of the step 2) is as follows: Step 2-1) performing point cloud sampling on the talus three-dimensional skeleton triangular mesh model, and automatically dividing a talus pulley joint surface point cloud by utilizing normal filtering and curvature filtering based on an anatomical principle; Step 2-2) constructing an objective function to pe