CN-115517765-B - Navigation method, system, computer device, storage medium and computer program product

Abstract

The present application relates to a navigation method, a system, a computer device, a storage medium and a computer program product. The method comprises the steps of obtaining a target medical model, generating operation planning information based on an initial image scanned by a scanning device, obtaining a real-time image scanned by the scanning device and registering the real-time image and the target medical model to obtain a first registration relation, obtaining a first coordinate relation between a coordinate system corresponding to an executing tool and an image coordinate system of the real-time image, and navigating the executing tool based on the first coordinate relation, the first registration relation and the operation planning information. By adopting the method, modeling can be performed in the operation process, preoperative CT is not needed, real-time registration is realized, and additional marks are not needed to be added.

Inventors

• Request for anonymity
• Request for anonymity
• Request for anonymity
• LI TAO

Assignees

• 苏州微创畅行机器人有限公司

Dates

Publication Date

20260508

Application Date

20221028

Claims (20)

1. 1. A navigation system, the system comprising: the scanning device is used for scanning to obtain an initial image and a real-time image; The execution tool is used for operating according to the operation planning information; a controller for performing the steps of: Acquiring a target medical model, wherein the target medical model is obtained based on the initial image reconstruction of the scanning device; Generating operational planning information based on the target medical model; acquiring a real-time image obtained by scanning by the scanning device, and registering the real-time image with the target medical model to obtain a first registration relationship; acquiring a first coordinate relation between a coordinate system corresponding to an execution tool and an image coordinate system of the real-time image; Navigating the execution tool based on the first coordinate relationship, the first registration relationship, and the operational planning information; the acquiring a target medical model includes: Acquiring a current initial image scanned by a scanning device; The method comprises the steps of extracting target information from a current initial image, calculating a histogram of the current initial image, extracting a target area based on the histogram, calculating the distance from a voxel to the surface of an object based on the depth value of the target area and the coordinate of the voxel under a camera coordinate system, and generating target information based on the distance from the voxel to the surface of the object; and obtaining a target medical model based on the extracted target information.
2. 2. The system according to claim 1, wherein after the acquiring the real-time image scanned by the scanning device and registering the real-time image with the target medical model to obtain the first registration relationship, the method further comprises: Updating the position information of the target medical model based on the position of the target in the real-time image; The navigation of the execution tool based on the first coordinate relationship, the first registration relationship, and the operation planning information includes: updating the operation planning information based on the updated position information of the target medical model; and navigating the execution tool based on the first coordinate relationship, the first registration relationship and the updated operation planning information.
3. 3. The system of claim 1, wherein the deriving the target medical model based on the extracted target information comprises: Generating a current target three-dimensional point cloud based on the extracted target information; Registering the current target three-dimensional point cloud with the initial image of the previous frame; Updating voxel values of the corresponding three-dimensional point clouds based on the registration result; And carrying out target reconstruction on the voxel values of the updated three-dimensional point cloud to obtain a target medical model.
4. 4. A system according to claim 3, wherein updating voxel values of the corresponding three-dimensional point cloud based on the registration result comprises: Acquiring voxel values and weights of voxels corresponding to the current target three-dimensional point cloud and the previous frame initial image; and updating the corresponding voxel value based on the voxel values of the voxels corresponding to the current target three-dimensional point cloud and the previous frame initial image and the weight, and retaining the voxel values of the voxels of which the current target three-dimensional point cloud and the previous frame initial image are not registered.
5. 5. The system of claim 4, wherein prior to the obtaining the voxel values and weights of the voxels of the current target three-dimensional point cloud and the previous frame initial image, further comprising: And calculating the weight of the current target three-dimensional point cloud based on the weight of the voxel corresponding to the initial image of the previous frame and the target weight.
6. 6. The system of any one of claims 1 to 5, wherein after generating operational planning information based on the target medical model, further comprising: Generating an operation boundary based on the operation planning information; The navigation of the execution tool based on the first coordinate relationship, the first registration relationship, and the operation planning information includes: Determining a real-time position of the execution tool based on the first coordinate relationship, the first registration relationship, and the operation planning information; And when the real-time position exceeds the operation boundary, controlling the execution tool to stop moving, and/or alarming.
7. 7. The system of any one of claims 1 to 5, wherein the first coordinate relationship between the coordinate system corresponding to the acquisition execution tool and the image coordinate system of the real-time image comprises at least one of: Based on the position of the executing tool in the mechanical arm coordinate system, the initial coordinate of the mechanical arm base, the self-posture of the mechanical arm, the terminal coordinate system of the mechanical arm, the rigid connection posture of the terminal of the mechanical arm and the scanning device and the image coordinate system of the real-time image, a first coordinate relation between the coordinate system corresponding to the executing tool and the image coordinate system of the real-time image is obtained, or And obtaining a first coordinate relation between a coordinate system corresponding to the execution tool and the image coordinate system of the real-time image based on the terminal coordinate system of the handheld device, the rigid connection gesture of the handheld device and the scanning device and the image coordinate system of the real-time image.
8. 8. The system of claim 7, wherein the scanning device comprises at least two, a portion of the scanning device being configured to scan an operating region and another of the scanning devices being configured to scan an environmental region, the controller being further configured to perform the steps of: Generating environmental information based on an environmental image scanned by the scanning device scanning an environmental area; acquiring a second coordinate relation between a coordinate system corresponding to the execution tool and an image coordinate system of the environment image; and performing obstacle avoidance navigation on the execution tool based on the second coordinate relation and the environment information.
9. 9. The system of claim 1, wherein the navigating the execution tool based on the first coordinate relationship, the first registration relationship, and the operational planning information comprises: Navigation information of the execution tool is generated and displayed based on the first coordinate relationship, the first registration relationship and the operation planning information.
10. 10. The system of claim 1, wherein the scanning device is mounted to a robotic arm or the scanning device is mounted to a handheld device.
11. 11. The system of claim 10, wherein the scanning device comprises at least two, a portion of the scanning device being configured to scan an operating region and another of the scanning devices being configured to scan an environmental region.
12. 12. A computer readable storage medium having stored thereon a computer program, characterized in that the computer program when executed by a processor realizes the steps of: Acquiring a target medical model, wherein the target medical model is obtained based on the initial image reconstruction of the scanning device; Generating operational planning information based on the target medical model; acquiring a real-time image obtained by scanning by the scanning device, and registering the real-time image with the target medical model to obtain a first registration relationship; acquiring a first coordinate relation between a coordinate system corresponding to an execution tool and an image coordinate system of the real-time image; Navigating the execution tool based on the first coordinate relationship, the first registration relationship, and the operational planning information; the acquiring a target medical model includes: Acquiring a current initial image scanned by a scanning device; The method comprises the steps of extracting target information from a current initial image, calculating a histogram of the current initial image, extracting a target area based on the histogram, calculating the distance from a voxel to the surface of an object based on the depth value of the target area and the coordinate of the voxel under a camera coordinate system, and generating target information based on the distance from the voxel to the surface of the object; and obtaining a target medical model based on the extracted target information.
13. 13. The computer readable storage medium according to claim 12, wherein the computer program when executed by the processor is further configured to, after the acquiring the real-time image scanned by the scanning device and registering the real-time image with the target medical model to obtain the first registration relationship,: Updating the position information of the target medical model based on the position of the target in the real-time image; The navigation of the execution tool based on the first coordinate relationship, the first registration relationship, and the operation planning information includes: updating the operation planning information based on the updated position information of the target medical model; and navigating the execution tool based on the first coordinate relationship, the first registration relationship and the updated operation planning information.
14. 14. The computer readable storage medium according to claim 12, wherein the obtaining a target medical model based on the extracted target information, which is implemented when the computer program is executed by a processor, comprises: Generating a current target three-dimensional point cloud based on the extracted target information; Registering the current target three-dimensional point cloud with the initial image of the previous frame; Updating voxel values of the corresponding three-dimensional point clouds based on the registration result; And carrying out target reconstruction on the voxel values of the updated three-dimensional point cloud to obtain a target medical model.
15. 15. The computer readable storage medium of claim 14, wherein the updating of voxel values of the corresponding three-dimensional point cloud based on registration results, as implemented by the computer program when executed by a processor, comprises: Acquiring voxel values and weights of voxels corresponding to the current target three-dimensional point cloud and the previous frame initial image; and updating the corresponding voxel value based on the voxel values of the voxels corresponding to the current target three-dimensional point cloud and the previous frame initial image and the weight, and retaining the voxel values of the voxels of which the current target three-dimensional point cloud and the previous frame initial image are not registered.
16. 16. The computer readable storage medium of claim 15, wherein the computer program, when executed by the processor, before the obtaining voxel values and weights of voxels of the current target three-dimensional point cloud and the previous frame initial image, further comprises: And calculating the weight of the current target three-dimensional point cloud based on the weight of the voxel corresponding to the initial image of the previous frame and the target weight.
17. 17. The computer readable storage medium according to any one of claims 12 to 16, wherein the computer program, when executed by a processor, after the generating operation planning information based on the target medical model, further comprises: Generating an operation boundary based on the operation planning information; The navigation of the execution tool based on the first coordinate relationship, the first registration relationship, and the operation planning information includes: Determining a real-time position of the execution tool based on the first coordinate relationship, the first registration relationship, and the operation planning information; And when the real-time position exceeds the operation boundary, controlling the execution tool to stop moving, and/or alarming.
18. 18. The computer readable storage medium according to any one of claims 12 to 16, wherein the first coordinate relationship between the coordinate system corresponding to the acquisition execution tool and the image coordinate system of the real-time image, which is realized when the computer program is executed by a processor, comprises at least one of: Based on the position of the executing tool in the mechanical arm coordinate system, the initial coordinate of the mechanical arm base, the self-posture of the mechanical arm, the terminal coordinate system of the mechanical arm, the rigid connection posture of the terminal of the mechanical arm and the scanning device and the image coordinate system of the real-time image, a first coordinate relation between the coordinate system corresponding to the executing tool and the image coordinate system of the real-time image is obtained, or And obtaining a first coordinate relation between a coordinate system corresponding to the execution tool and the image coordinate system of the real-time image based on the terminal coordinate system of the handheld device, the rigid connection gesture of the handheld device and the scanning device and the image coordinate system of the real-time image.
19. 19. The computer readable storage medium of claim 18, wherein the scanning means comprises at least two, a portion of the scanning means being for scanning an operating region and another of the scanning means being for scanning an environmental region, the computer program when executed by the processor further performing the steps of: Generating environmental information based on an environmental image scanned by the scanning device scanning an environmental area; acquiring a second coordinate relation between a coordinate system corresponding to the execution tool and an image coordinate system of the environment image; and performing obstacle avoidance navigation on the execution tool based on the second coordinate relation and the environment information.
20. 20. The computer readable storage medium of claim 12, wherein the navigation of the execution tool based on the first coordinate relationship, the first registration relationship, and the operation planning information, as implemented when the computer program is executed by a processor, comprises: Navigation information of the execution tool is generated and displayed based on the first coordinate relationship, the first registration relationship and the operation planning information.

Description

Navigation method, system, computer device, storage medium and computer program product Technical Field The present application relates to the field of intelligent medical technology, and in particular, to a navigation method, a navigation system, a computer device, a storage medium, and a computer program product. Background With the development of computer technology, a surgical navigation system appears, and the appearance of the surgical navigation system accords with the development trend of accurate surgery. The operation navigation system provides richer reference information and more accurate guidance for operation through analysis of medical images of patients and application of various sensors in operation, and becomes a powerful tool for assisting doctors in completing operations. In the traditional technology, the preoperative surgical scheme is that data such as patient CT and the like are acquired before operation and are completed in a computer of an operation navigation system, and then a doctor performs operation according to the preoperative surgical scheme. However, current surgical navigation systems require preoperative CT imaging, which results in long surgical preparation time and additional trauma due to the need to add positioning markers to the target area in order to complete bone to CT registration. Disclosure of Invention Based on this, it is necessary to provide a navigation method, system, computer device, computer readable storage medium and computer program product that is capable of modeling during operation, does not require preoperative CT, and does not require additional markers for real-time registration. In a first aspect, the present application provides a navigation method, the method comprising: Acquiring a target medical model, wherein the target medical model is obtained based on the initial image reconstruction of the scanning device; Generating operational planning information based on the target medical model; acquiring a real-time image obtained by scanning by the scanning device, and registering the real-time image with the target medical model to obtain a first registration relationship; acquiring a first coordinate relation between a coordinate system corresponding to an execution tool and an image coordinate system of the real-time image; And navigating the execution tool based on the first coordinate relationship, the first registration relationship and the operation planning information. In one embodiment, after the acquiring the real-time image scanned by the scanning device and registering the real-time image with the target medical model to obtain the first registration relationship, the method further includes: Updating the position information of the target medical model based on the position of the target in the real-time image; The navigation of the execution tool based on the first coordinate relationship, the first registration relationship, and the operation planning information includes: updating the operation planning information based on the updated position information of the target medical model; and navigating the execution tool based on the first coordinate relationship, the first registration relationship and the updated operation planning information. In one embodiment, the acquiring the target medical model includes: Acquiring a current initial image scanned by a scanning device; extracting target information from the current initial image, and generating a current target three-dimensional point cloud based on the extracted target information; Registering the current target three-dimensional point cloud with the initial image of the previous frame; Updating voxel values of the corresponding three-dimensional point clouds based on the registration result; And carrying out target reconstruction on the voxel values of the updated three-dimensional point cloud to obtain a target medical model. In one embodiment, the extracting the target information from the current initial image includes: calculating a histogram of the current initial image, and extracting a target area based on the histogram; calculating to obtain the distance from the voxel to the object surface based on the depth value of the target area and the coordinate of the voxel under a camera coordinate system; target information is generated based on the distance of the voxel to the object surface. In one embodiment, updating the voxel values of the corresponding three-dimensional point cloud based on the registration result includes: Acquiring voxel values and weights of voxels corresponding to the current target three-dimensional point cloud and the previous frame initial image; and updating the corresponding voxel value based on the voxel values of the voxels corresponding to the current target three-dimensional point cloud and the previous frame initial image and the weight, and retaining the voxel values of the voxels of which the current target three-dimensional point c