CN-122005089-A - Collaborative navigation method and system for craniotomy robot

CN122005089ACN 122005089 ACN122005089 ACN 122005089ACN-122005089-A

Abstract

The invention discloses a collaborative navigation method and a collaborative navigation system for a surgical robot, which comprise the steps of S1, carrying out three-dimensional reconstruction on a surgical part of a patient to obtain a planned surgical path, S2, predicting that any marking ball on a first positioning tool is blocked when a surgical instrument moves on the planned surgical path and can not pass through a blocking track point on the surgical path when an optical tracking system is used for tracking and positioning, S3, blocking track points are planned to obtain an optimal collaborative navigation track of the optical tracking system, wherein the collaborative navigation system for the surgical robot comprises a surgical robot module, a navigation robot module and a collaborative navigation control module formed by a preoperative preparation module, a preoperative blocking prediction module and a collaborative navigation path planning module, and the collaborative navigation method and the collaborative navigation system for the surgical robot can enable the planned path to have optimality and smoothness and a short adjustment time in a self-adaptive complex surgical environment.

Inventors

SUN QIYUAN
YUAN SHUQIANG
ZHANG GUOBIN
LIU ZHENZHONG
PAN YUNLONG

Assignees

天津理工大学

Dates

Publication Date: 20260512
Application Date: 20260128

Claims (10)

1. The collaborative navigation method for the surgical robot is characterized by comprising the following steps: S1, performing three-dimensional reconstruction on an operation part of a patient to obtain a planned operation path; S2, when the surgical instrument is predicted to move on a planned surgical path, any marking ball on the first positioning tool is blocked and cannot be tracked and positioned by the optical tracking system, a blocking track point of the surgical instrument on the surgical path is predicted, so that a blocking path section on the surgical path is determined; S3, planning to obtain the optimal collaborative navigation track of the optical tracking system based on the shielding track points obtained in the step S2, wherein the method comprises the steps of 1) constructing a path planning network based on the multi-task network deep reinforcement learning, 2) defining a reward function formed by step rewards, joint rewards and first constraint rewards Initializing and generating a high-dimensional state vector based on the pose of the first positioning tool under the end point track point of the shielding path section And a bonus function To input it into the path planning network for first stage training to obtain the optimal observation point of optical tracking system, 3) defining the path planning reward composed of target approaching reward, motion smooth reward and second constraint reward And in a high-dimensional state vector Added with joint velocity and acceleration to form a high-dimensional state vector Initializing to generate a high-dimensional state vector And trajectory planning rewards And inputting the optical tracking system into a path planning network for second-stage training to obtain the optimal collaborative navigation track of the optical tracking system.
2. The collaborative navigation method of a surgical robot according to claim 1, wherein in step S1, when a surgical instrument is moved to each track point on a surgical path, the specific implementation step of determining whether a marker ball on a first positioning tool is blocked is as follows: S201, based on a planned operation path, the pose of four marking balls on a first positioning tool on a current track point is obtained through a positive kinematic model of an operation mechanical arm; s202, obtaining a space point set describing the position of the clamp and the surgical instrument on the current track point (J=1, 2,., n), n being a set of spatial points The number of mid-space points; S203, respectively calculating space point sets Vector lengths to the left and right lens centers, respectively And And each marking ball Vector lengths from center point to left and right lens centers, respectively And J is the set of spatial points The serial number of the space point in the middle, i is the marking ball Is a sequence number of (2); s204, respectively calculating difference angles between marking balls based on left lens Right lens-based difference angle between marker balls , Respectively calculating the difference angle between the marking ball and the clamp based on the left lens Difference angle between marking ball and clamp based on right lens ; S205, for each marking ball in turn (I=1, 2,3, 4) to make the following condition judgment, when any one of the marker balls When any one of the conditions 1 to 4 is judged to be satisfied, judging the current track point as a shielding track point; condition 1: And (2) and Or (b) ; Condition 2: And (2) and ; Condition 3: And (2) and ; Condition 4: And (2) and 。
3. The collaborative navigation method of claim 2, wherein in step S204, any two marker balls on the first positioning tool are based on the left or right lens line of sight And The calculation formula of the difference angle between the marking balls is as follows: , , In the formula, To mark the inter-ball difference angle based on the left lens, For the left lens and two marking balls 、 The included angle between the two parts is that, To the marking ball for the left lens Is arranged at the half angle of the viewing cone, To the marking ball for the left lens Is a viewing cone half angle; to mark the inter-ball difference angle based on the right lens, For right lens and two marking balls 、 The included angle between the two parts is that, To the marking ball for right lens Is arranged at the half angle of the viewing cone, To the marking ball for right lens Is a viewing cone half angle; Based on the left lens or the right lens sight, any marking ball and space point set The calculation formula for forming the difference angle between the marking ball and the clamp between any space points is as follows: , , In the formula, To be based on the difference angle between the marking ball of the left lens and the clamp, For the center of the left lens to the space point and marking the ball The included angle between the center points is defined by the two, Left lens center to marker ball Is arranged at the half angle of the viewing cone, Is based on the difference angle between the marking ball of the right lens and the clamp, For right lens center to space point and marking ball The included angle between the center points is defined by the two, To mark ball for right lens center Is a viewing cone half angle of (c).
4. The collaborative navigation method of a surgical robot according to claim 1, wherein the path planning network for deep reinforcement learning of a multi-task network includes an Actor network and a Critic network in step S3, wherein the Actor network and the Critic network are each composed of a connected shared network and task-specific branch network, the task-specific branch networks of the two are each composed of N task-specific branch modules to output an action average value and a Q value corresponding to each task respectively by inputting a shared feature extracted from the shared network, the path planning network further includes a multi-task deep Q network composed of N Q sub-networks each including two independent Q networks to output two Q values based on the shared feature of the Critic network and take a smaller value for calculating a loss function of a corresponding task branch in the Critic network, The loss function expression of each task branching module in the Actor network is as follows: , In the formula, As a function of the temperature parameter(s), As a function of the policy of task k, As a logarithmic approximation of the motion, The Q value of the task k; the loss function expression of each task branch module in the Critic network is as follows: , , In the formula, For the next state The act of down-sampling the current policy, For the output of the Critic network, Is a parameter of the temperature of the liquid, Is a discount factor that is used to determine the discount, For the instant rewards of task k, The target Q value of the task k output by the multi-task deep Q network is i, and i represents the i-th specific parameter of the task k; The loss function expression of the shared network is: , wherein N is the number of tasks; the loss function expression of each Q sub-network in the multitasking deep Q network is as follows: , In the formula, For the instant rewards of task k, As a discount factor, the number of times the discount is calculated, The Q value of task k output for the Critic network, Is a regular term of the entropy of the light, For the next state of task k, Is the next action of task k.
5. The collaborative navigation method of a surgical robot of claim 1, wherein, during a first stage of training on a path planning network, Reward function The expression of (2) is: Wherein, the method comprises the steps of, Step size rewards Set as the pose of the optical tracking system at the moment t And initial pose Is used for the distance of euclidean distance, ; Joint rewards Is arranged as a joint limit for the navigation mechanical arm, , The minimum joint angle and the maximum joint angle of the motion allowed by the joints of the navigation mechanical arm, wherein i represents the number of the joints; constrained rewards The method comprises the following steps: , As the weighting coefficient of the observation angle, For all observation angles Is a reward function of (2) The sum of the two values, , Is a lens J-phase positioning tool Ti observation angle, j is L or R, ti represents positioning tool, i is 1 or 2; As the weight coefficient of the interference angle, For all interference angles Is a reward function of (2) The sum of the two values, , Is a lens An interference angle of j; , covering a bonus function for the corresponding field of view of d Ti The sum of the two values, D Ti is a positioning tool The distance of the minimum outer wrap sphere boundary to the field of view plane; the minimum package ball radius for the positioning tool Ti; As the weight coefficient of the inverse angle of rotation, Is the reverse angle A corresponding roll-over reward function is provided, ; High-dimensional state vector The expression of (2) is: , wherein, T is the state of the positioning tool, , And The position of the first positioning tool and the four element pose, And The position and four element gesture of the second positioning tool respectively; G is the state of the navigation mechanical arm, Θ is the current joint angle of the navigation mechanical arm, θ∈ 6, Pee and Qee are the position and quaternion gesture of the tail end of the navigation mechanical arm respectively, and Pee epsilon 3,Qee∈ 3;P L and Q L are respectively the central position and the gesture quaternion of the left lens, and P L E 6,Q L ∈ 6;P R and Q R are respectively the central position and the gesture quaternion of the right lens, P R E 6,Q R ∈ 6; In view of the environmental constraints of the present invention, , Wherein, the For the flip angle of the optical tracking system, Is the interference angle of the left lens, Is the interference angle of the right lens, For the left lens-first positioning tool observation angle, For the left lens-second positioning tool perspective, For the right lens-first positioning tool observation angle, For the right lens-second positioning tool perspective, For the field plane parameter, d T1 is the distance from the minimum outer wrap sphere boundary of the first positioning tool to the field plane, and d T2 is the distance from the minimum outer wrap sphere boundary of the second positioning tool to the field plane.
6. The collaborative navigation method of claim 5, wherein during the second stage of training of the path planning network, Track planning rewards The expression of (2) is: Wherein, the method comprises the steps of, Target proximity rewards The method comprises the following steps: , for a range prize to be awarded, , , The weight of the rewards are respectively given, For the euclidean distance of the current position of the optical tracking system from the optimal observation point, Delta pos and delta rot are respectively success thresholds; in order to approach the benefit of the prize, , For the time-based reward, , Is the time of step i; Motion smoothing rewards The method comprises the following steps: , wherein, To give a benefit to the joint angle, , And The joint angles at the time step t+1 and the time step t are respectively, and i is the serial number of the joint; To reward the angular velocity of the joint, , , Is the maximum angular velocity allowed by the ith joint, N is the total number of joints, and the angular velocity of the joints Is the angular velocity of the ith joint at the current time step, deltat is the time step, And The joint angles of the time step t and the time step t-1 are respectively; To reward the acceleration of the joint, , Angular acceleration of joint Is the angular acceleration of the ith joint at the current time step, Is the maximum angular acceleration allowed by the ith joint; second constraint reward The method comprises the following steps: , And Respectively are awarding functions A first constraint reward and a joint reward.
7. The surgical robot co-navigation method of claim 1, further comprising the step of fine tuning an optical positioning system: 1) In the motion process of the surgical instrument, the optical positioning system monitors the motion gesture of the surgical instrument in real time and acquires the left lens-first positioning tool observation angle and the right lens-first positioning tool observation angle in real time; 2) Calculating the difference value between the planned left lens-first positioning tool planning observation angle and the current left lens-first positioning tool observation angle and the difference value between the planned right lens-first positioning tool planning observation angle and the current right lens-first positioning tool observation angle in real time; 3) And setting an observation angle difference value threshold, and when the difference value of any observation angle is more than or equal to the observation angle difference value threshold, fine-tuning the optical tracking system from the planned pose to the corresponding pose of the surgical instrument after the deviation.
8. A surgical robot co-navigation system, comprising: The surgical robot module comprises a surgical mechanical arm (1), a first positioning tool (3) and a second positioning tool (4), wherein a clamp (2) is fixed at the tail end of the surgical mechanical arm (1), the first positioning tool (3) is fixed on the clamp (2), and the second positioning tool (4) is fixed on the adjacent side of a surgical part of a patient; the navigation robot module comprises a navigation mechanical arm (6), and an optical tracking system (5) for continuously tracking and positioning the first positioning tool (3) and the second positioning tool is fixed at the tail end of the navigation mechanical arm; The collaborative navigation control module comprises a preoperative preparation module, a preoperative shielding prediction module and a collaborative navigation path planning module, wherein the preoperative preparation module performs three-dimensional reconstruction on an operation part of a patient to plan an operation path, maps the planned operation path to the operation part of the patient under a real operation space, maps the tail end of an operation instrument to the planned operation path under the real operation space, the preoperative shielding prediction module is used for predicting a shielding track point corresponding to any marking ball on a first positioning tool (3) when the operation instrument moves in the planned operation path, and the collaborative navigation path planning module performs the planning to obtain the optimal collaborative navigation track of the optical tracking system (5) through constructing and completing a trained path planning network based on the multi-task network depth reinforcement learning.
9. The surgical robot co-navigation system of claim 8, wherein the pre-operative occlusion prediction module comprises: The first positioning tool pose acquisition module is used for calculating the pose of the clamp on each track point and converting the pose of the four marking balls on the first positioning tool on the track points; the space point set generating module is used for generating a space point set capable of describing the positions of the clamp and the surgical instrument on each track point; The parameter acquisition module is used for acquiring vector lengths from each space point in the space point set to the center of the left lens and the center of the right lens respectively when the surgical instrument is positioned on each track point of the planned surgical path, and vector lengths from each marking ball center point to the center of the left lens and the center of the right lens respectively; And the prediction module is used for determining whether each track point is a shielding track point according to the parameters acquired by the parameter acquisition module.
10. The surgical robot co-navigation system of claim 8, wherein the co-navigation path planning module comprises: a network construction module for constructing a multi-task network deep reinforcement learning path planning network; A first parameter generation module for defining a first stage training reward function And generating a high-dimensional state vector for the first-stage training by initializing through obtaining the shielding track points predicted by the prediction module And a bonus function ; A second parameter generation module for defining a second stage training trajectory planning reward And is based on a high-dimensional state vector Initializing to generate a high-dimensional state vector for second-stage training And trajectory planning rewards ; The training module is used for sequentially calling the first parameter generation module and the second parameter generation module and performing two-stage training on the multi-task network deep reinforcement learning path planning network so as to obtain the optimal collaborative navigation track of the optical tracking system.

Description

Collaborative navigation method and system for craniotomy robot Technical Field The invention relates to the technical field of medical navigation robots, in particular to a collaborative navigation system of a craniotomy robot. Background The robot assisted surgery can significantly improve the accuracy and safety of the surgery, wherein the optical positioning system is the core for achieving high-precision navigation. However, the optical positioning system (OTS) in the prior art mostly adopts a passive tracking mode for fixing the pose. Taking skull as an example, the pose of the end effector (such as a milling cutter) of the surgical robot is dynamically changeable, so that visual occlusion is easily caused between a positioning mark point on the end effector and OTS at a fixed position, real-time position information of the instrument is lost, navigation continuity is destroyed, the surgical time is increased, bone window boundary drifting cutting is more likely to be caused, dura mater and brain tissues are damaged, and the safety of a patient is seriously endangered. In order to solve the problem of vision shielding, the prior art proposes a scheme of multi-camera fusion or multi-sensor fusion, but the method increases the complexity and the cost of the system, and cannot realize active and dead-angle-free tracking fundamentally. Although few researches are attempted to mount OTS on a mechanical arm to realize active navigation, the method is mostly aimed at joint orthopedic operations, and lacks a targeted occlusion prediction and collaborative track planning strategy for controlling operations with the characteristics of complex paths, changeable instrument posture conversion, small operation area range and the like, such as craniotomy. Therefore, it is necessary to develop a robot collaborative navigation system capable of realizing active prediction and avoiding shielding and ensuring continuous navigation in the whole course of craniotomy, and the robot collaborative navigation system has important significance. Disclosure of Invention The invention aims to provide a craniotomy robot collaborative navigation system for solving the problem of vision occlusion caused by changeable appliance pose. For this purpose, the technical scheme of the invention is as follows. In one aspect, the invention provides a collaborative navigation method for a surgical robot, which comprises the following steps: S1, performing three-dimensional reconstruction on an operation part of a patient to obtain a planned operation path; S2, when the surgical instrument is predicted to move on a planned surgical path, any marking ball on the first positioning tool is blocked and cannot be tracked and positioned by the optical tracking system, a blocking track point of the surgical instrument on the surgical path is predicted, so that a blocking path section on the surgical path is determined; S3, planning to obtain the optimal collaborative navigation track of the optical tracking system based on the shielding track points obtained in the step S2, wherein the method comprises the steps of 1) constructing a path planning network based on the multi-task network deep reinforcement learning, 2) defining a reward function formed by step rewards, joint rewards and first constraint rewards Initializing and generating a high-dimensional state vector based on the pose of the first positioning tool under the end point track point of the shielding path sectionAnd a bonus functionTo input it into the path planning network for first stage training to obtain the optimal observation point of optical tracking system, 3) defining the path planning reward composed of target approaching reward, motion smooth reward and second constraint rewardAnd in a high-dimensional state vectorAdded with joint velocity and acceleration∈6,∈6, Forming a high-dimensional state vectorInitializing to generate a high-dimensional state vectorAnd trajectory planning rewardsAnd inputting the optical tracking system into a path planning network for second-stage training to obtain the optimal collaborative navigation track of the optical tracking system. Furthermore, the collaborative navigation method of the surgical robot further comprises a fine adjustment step of the optical positioning system, so that when the motion track of the surgical instrument is offset by a small amplitude, the optical tracking system is correspondingly adjusted in time. On the other hand, the invention also provides a surgical robot collaborative navigation system, which comprises: The surgical robot module comprises a surgical mechanical arm, a first positioning tool and a second positioning tool, wherein a clamp is fixed at the tail end of the surgical mechanical arm, the first positioning tool is fixed on the clamp, and the second positioning tool is fixed on the adjacent side of a surgical part of a patient; the navigation robot module comprises a navigation mechanical arm, and an op