CN-116725668-B - Instrument positioning navigation method, device, system and storage medium

CN116725668BCN 116725668 BCN116725668 BCN 116725668BCN-116725668-B

Abstract

The application belongs to the technical field of navigation, and particularly relates to an instrument positioning navigation method, device and system and a storage medium. The method comprises the steps of obtaining an endoscope real-time image and a preoperative image of an identification object, determining a first position conversion model of the endoscope relative to the identification object based on the endoscope real-time image and the preoperative image, obtaining a second position conversion model of a fixed base point in an instrument arm relative to the endoscope, determining a third position conversion model of the fixed base point relative to the identification object based on the first position conversion model and the second position conversion model, and obtaining the position of the mobile node relative to the identification object based on the position relation between the mobile node in the instrument arm and the fixed base point and the third position conversion model. The application improves the navigation precision of the robot and avoids the safety risk of the robot operation by determining the position of the instrument arm relative to the target based on the endoscope image and the robot kinematic model and accurately navigating based on the determined position.

Inventors

Request for anonymity
WANG ZHEN

Assignees

上海视微医疗科技有限公司

Dates

Publication Date: 20260505
Application Date: 20230523

Claims (14)

1. An instrument positioning navigation device, characterized by being applied to a flexible robot including an endoscope and an instrument arm, comprising: the acquisition module is used for acquiring the real-time image of the endoscope and the preoperative image of the identification object; A first conversion module for determining a first position conversion model of the endoscope relative to the recognition object based on the endoscope real-time image and the preoperative image; a second conversion module for acquiring a second position conversion model of a fixed base point in the instrument arm relative to the endoscope; a third transformation module for determining a third position transformation model of the fixed base point relative to the recognition object based on the first position transformation model and the second position transformation model; and the position module is used for obtaining the position of the mobile node relative to the identification object based on the position relation between the mobile node in the instrument arm and the fixed base point and the third position conversion model.
2. An instrument positioning and navigation device according to claim 1, wherein the number of mobile nodes is plural; The obtaining the position of the mobile node relative to the identification object based on the position relation between the mobile node and the fixed base point in the instrument arm and the third position conversion model includes: determining a positional relationship between each mobile node in the instrument arm and the fixed base point; and determining the position of each mobile node relative to the identification object based on the position relation between each mobile node and the fixed base point and the third position conversion model.
3. An instrument positioning navigation device according to claim 2, said instrument arm comprising a rigid rod and a flexible tube connected to each other, wherein said determining the positional relationship between each mobile node in said instrument arm and said fixed base point comprises: the method comprises the steps of obtaining the position of a connecting point relative to the fixed base point, wherein the connecting point is a connecting point between the rigid rod and the flexible pipe; determining a bending angle and a rotation angle of the flexible pipe by driving a sensor; the positional relationship between the each mobile node in the instrument arm and the fixed base point is determined based on the positional relationship of the connection point with respect to the fixed base point, the length, the bending angle, and the rotation angle of the flexible tube.
4. An instrument positioning and navigation device according to claim 1, wherein said device is further adapted to, after said obtaining a position of said mobile node with respect to said recognition object based on a positional relationship between said mobile node and said fixed base point in said instrument arm and said third position conversion model, And controlling an end node in the mobile nodes to move to a preset position of the identification object based on the position of the mobile nodes relative to the identification object, wherein the end node is the mobile node positioned at the tail end of the instrument arm in the mobile nodes.
5. The instrument positioning and navigation device according to claim 4, wherein the preset position is a position corresponding to a target object in the identified objects; The controlling the end node in the mobile node to move to the preset position of the identification object based on the position of the mobile node relative to the identification object comprises the following steps: Determining a first distance between the end node and the preset position based on the position of the end node relative to the identification object; and when the first distance is greater than a first preset distance threshold, controlling the instrument arm to move along a linear path until the first distance is smaller than or equal to the first preset distance threshold so as to enable the tail end node to move to a preset position of the identification object, wherein the linear path is a connecting line between the tail end node and the preset position.
6. An instrument positioning and navigation device according to claim 5, wherein said device is further adapted to, after said determining a first distance between said end node and said predetermined location based on a location of said end node relative to said identification object, Determining a second distance between the instrument arm and a non-target object based on the position relation of each mobile node relative to the identification object, wherein the second distance represents a preset distance between the mobile node and the non-target object on the instrument arm; when the first distance is greater than a first preset distance threshold and the second distance is less than a second preset distance threshold, controlling the instrument arm to move along an obstacle avoidance path; And when the instrument arm moves to the end point of the obstacle avoidance path and the first distance is larger than the first preset distance threshold, controlling the instrument arm to move along the linear path until the first distance is smaller than or equal to the first preset distance threshold, so that the end node moves to the preset position of the identification object.
7. The instrument positioning and navigation device of claim 6, wherein the controlling the instrument arm to move along the obstacle avoidance path when the first distance is greater than a first preset distance threshold and the second distance is less than a second preset distance threshold comprises: Controlling the instrument arm to stop moving when the first distance is greater than a first preset distance threshold and the second distance is less than a second preset distance threshold; and responding to the control instruction, and controlling the instrument arm to move along the obstacle avoidance path.
8. The instrument positioning and navigation device of claim 6, wherein said controlling movement of said instrument arm along an obstacle avoidance path comprises: Determining the obstacle avoidance path based on the outer contour of the non-target object, wherein the distance between the obstacle avoidance path and the outer contour of the non-target object is a preset obstacle avoidance distance; and controlling the instrument arm to move along the obstacle avoidance path.
9. The instrument positioning navigation device of claim 8, wherein the determining the obstacle avoidance path based on the outer contour of the non-target object comprises: determining a first intersection point and a second intersection point of the straight line path and the outer contour of the non-target object, wherein the distance between the first intersection point and the end node is smaller than the distance between the second intersection point and the end node; Determining a starting point of the obstacle avoidance path on the linear path based on the first intersection point, wherein the distance between the starting point and the first intersection point is the preset obstacle avoidance distance; Determining an end point of the obstacle avoidance path on the linear path based on the second intersection point, wherein the distance between the end point and the second intersection point is the preset obstacle avoidance distance; And determining the obstacle avoidance path based on the starting point of the obstacle avoidance path and the ending point of the obstacle avoidance path.
10. An instrument positioning navigation device according to claim 3, wherein the device is further adapted to: determining an instrument activity boundary based on an edge of the recognition object; Determining a positional relationship of each mobile node relative to the instrument activity boundary based on the positional relationship of each mobile node relative to the recognition object; Determining a third distance between the instrument arm and the instrument active boundary based on the positional relationship of each mobile node relative to the instrument active boundary, the third distance characterizing a shortest distance between the mobile node and the instrument active boundary on the instrument arm; And when the third distance is smaller than a third preset distance threshold value, controlling the instrument arm to stop moving.
11. An instrument positioning and navigation device according to claim 10, wherein said device is further adapted to, after said determining a third distance between said instrument arm and said instrument activity boundary based on said positional relationship of each mobile node with respect to said instrument activity boundary, And when the third distance is smaller than a fourth preset distance threshold, applying repulsive force to the handle of the flexible robot, wherein the fourth preset distance threshold is larger than the third preset distance threshold.
12. The instrument positioning and navigation device of claim 11, wherein said applying a repulsive force to the handle of the flexible robot when the third distance is less than a fourth preset distance threshold comprises: when the third distance is smaller than a fourth preset distance threshold value, determining the mobile node with the distance between the mobile node and the instrument activity boundary being the third distance as an alarm node; determining the repulsive force based on the speed of the alert node, the third distance, and a preset safety threshold; The repulsive force is applied to a handle of the flexible robot.
13. An instrument positioning navigation system, comprising: An instrument arm; the instrument arm includes a mobile node and a fixed base point; An endoscope for acquiring a real-time image of an identification object; The device comprises a processor and a memory, wherein at least one instruction, at least one section of program, a code set or an instruction set is stored in the memory, and the at least one instruction, the at least one section of program, the code set or the instruction set is loaded and executed by the processor to realize the operation process corresponding to the device positioning navigation device of any one of claims 1-12.
14. A computer readable storage medium having stored therein at least one instruction, at least one program, code set, or instruction set that is loaded and executed by a processor to implement the corresponding operational procedure of the instrument positioning navigation system of claim 13.

Description

Instrument positioning navigation method, device, system and storage medium Technical Field The invention relates to the technical field of surgical robots, in particular to an instrument positioning navigation method, device and system and a storage medium. Background In the medical field, the flexible robot can be used as a surgical robot, is particularly suitable for a complex and narrow working space, and provides more accurate control and operation for surgery. Unlike conventional rigid robots, the instrument arm of flexible robots is typically made up of a plurality of flexible continuum, where the instrument arm as a whole may have various configurations during a surgical procedure, and where the position of points on the instrument arm relative to the distal end of the instrument arm may change over time. Due to the high flexibility and super-redundancy degree of freedom of the flexible robot structure, the path navigation method of the traditional rigid robot is not suitable for the flexible robot any more. In addition, the existing navigation methods of the flexible robots have the problems of inaccurate and unclear planning paths, so that the safety risk and the operation difficulty of the robot operation are improved. Disclosure of Invention In order to solve the technical problems, the application provides an instrument positioning navigation method, device and system and a storage medium. The application can improve the navigation precision of the robot and avoid the safety risk of the robot operation by determining the position of the instrument arm relative to the target based on the endoscope image and the robot kinematic model and accurately navigating based on the determined position. In one aspect, the present application provides a method of instrument positioning navigation for a flexible robot including an endoscope and an instrument arm, the method comprising: Acquiring an endoscope real-time image and a preoperative image of an identification object; Determining a first position conversion model of the endoscope relative to the recognition object based on the real-time image and the preoperative image of the endoscope; acquiring a second position conversion model of a fixed base point in the instrument arm relative to the endoscope; determining a third position conversion model of the fixed base point relative to the recognition object based on the first position conversion model and the second position conversion model; The position of the mobile node relative to the recognition object is obtained based on the positional relationship between the mobile node and the fixed base point in the instrument arm and the third position conversion model. The method comprises the steps of determining the position relation between each mobile node in the instrument arm and a fixed base point, determining the position of each mobile node relative to the identification object based on the position relation between each mobile node in the instrument arm and the fixed base point and a third position conversion model. Optionally, the instrument arm comprises a rigid rod and a flexible tube which are connected with each other, and the position relation between each movable node and the fixed base point in the instrument arm is determined, wherein the position relation between the connecting point and the fixed base point is acquired, the connecting point is the connecting point between the rigid rod and the flexible tube of the instrument arm, each movable node is positioned on the flexible tube, the bending angle and the rotating angle of the flexible tube are determined through a driving sensor, and the position relation between each movable node and the fixed base point in the instrument arm is determined based on the position relation between the connecting point and the fixed base point, the length of the flexible tube, the bending angle and the rotating angle. Optionally, after the position of the mobile node relative to the identification object is obtained based on the position relation between the mobile node and the fixed base point in the instrument arm and the third position conversion model, the method further comprises controlling an end node in the mobile node to move to a preset position of the identification object based on the position of the mobile node relative to the identification object, wherein the end node is the mobile node located at the tail end of the instrument arm in the mobile node. The method comprises the steps of determining a first distance between an end node and a preset position based on the position of the end node relative to an identification object, controlling an instrument arm to move along a straight path until the first distance is smaller than or equal to a first preset distance threshold value when the first distance is larger than the first preset distance threshold value, so that the end node moves to the preset position of the identification object,