CN-122023736-A - Magnetic resonance image reality scene enhancement method and device and electronic equipment

CN122023736ACN 122023736 ACN122023736 ACN 122023736ACN-122023736-A

Abstract

The application discloses a realistic magnetic resonance image realistic scene enhancement method, a realistic magnetic resonance image realistic scene enhancement device and electronic equipment, and belongs to the technical field of augmented reality. The method comprises the steps of obtaining a real-time visual image and a real-time magnetic resonance image of a target object in a medical scene, obtaining a real three-dimensional model of the target object based on the real-time visual image, obtaining a magnetic resonance three-dimensional model of the target object based on the real-time magnetic resonance image, and carrying out three-dimensional fusion operation on the real three-dimensional model and the magnetic resonance three-dimensional model to obtain an augmented reality view of the target object. The application discloses a magnetic resonance image reality scene enhancement method, which aims to solve the problem of poor observation effect on real-time conditions of objects in a medical scene in the related technology.

Inventors

JIANG JIANYANG
HUANG XIAOYAN

Assignees

中国科学技术大学先进技术研究院

Dates

Publication Date: 20260512
Application Date: 20260119

Claims (10)

1. A method for enhancing a magnetic resonance image reality scene, comprising: acquiring a real-time visual image and a real-time magnetic resonance image of a target object in a medical scene; acquiring a real three-dimensional model of the target object based on the real-time visual image, and acquiring a magnetic resonance three-dimensional model of the target object based on the real-time magnetic resonance image; and carrying out three-dimensional fusion operation on the three-dimensional reality model and the three-dimensional magnetic resonance model to obtain an augmented reality view of the target object.
2. The method of claim 1, wherein the real-time magnetic resonance image includes a plurality of two-dimensional magnetic resonance slice data, wherein the acquiring a real three-dimensional model of the target object based on the real-time visual image and acquiring a magnetic resonance three-dimensional model of the target object based on the real-time magnetic resonance image comprises: and carrying out three-dimensional reconstruction on the two-dimensional magnetic resonance slice data based on the medical image interaction tool pack and the visualization tool pack to obtain a magnetic resonance three-dimensional model.
3. The method of claim 1, wherein the acquiring a real three-dimensional model of the target object based on the real-time visual image and acquiring a magnetic resonance three-dimensional model of the target object based on the real-time magnetic resonance image comprises: performing conversion operation on the real-time visual image based on a target data processing library to obtain a three-dimensional point cloud model; And acquiring a real three-dimensional model based on the three-dimensional point cloud model.
4. The method of claim 1, wherein performing a three-dimensional fusion operation on the real three-dimensional model and the magnetic resonance three-dimensional model to obtain an augmented reality view of the target object comprises: And based on a target transformation matrix, performing three-dimensional fusion operation on the three-dimensional reality model and the three-dimensional magnetic resonance model, and obtaining an augmented reality view of the target object.
5. The method of claim 1, wherein after the three-dimensional fusion operation of the real three-dimensional model and the magnetic resonance three-dimensional model to obtain the augmented reality view of the target object, the method comprises: in the case of detecting a target instrument mark, resolving position information and posture information of the target instrument based on the target instrument mark; Rendering the target instrument in the augmented reality view based on the position information and the pose information of the target instrument.
6. The method of claim 1-5, wherein after the three-dimensional fusion operation of the real three-dimensional model and the magnetic resonance three-dimensional model to obtain the augmented reality view of the target object, the method comprises: And rendering and displaying the augmented reality view based on a first window, rendering and displaying the real-time visual image based on a second window, rendering and displaying the real-time magnetic resonance image based on a third window, and rendering and displaying the magnetic resonance three-dimensional model based on a fourth window.
7. The magnetic resonance image reality scene enhancement method of claim 6, further comprising: acquiring a focus mark and a planned intervention path based on the augmented reality view; rendering and displaying the planned intervention path and a predicted intervention path, wherein the predicted intervention path is generated based on the focus position, the position information and the posture information of the target instrument.
8. A magnetic resonance image reality scene enhancement device, comprising: The first acquisition module is used for acquiring a real-time visual image and a real-time magnetic resonance image of a target object in a medical scene; The second acquisition module is used for acquiring a real three-dimensional model of the target object based on the real-time visual image and acquiring a magnetic resonance three-dimensional model of the target object based on the real-time magnetic resonance image; And the third acquisition module is used for carrying out three-dimensional fusion operation on the three-dimensional reality model and the magnetic resonance three-dimensional model to acquire the augmented reality view of the target object.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the magnetic resonance image reality scenario augmentation method of any one of claims 1-7 when the program is executed by the processor.
10. A computer readable storage medium having stored thereon a computer program, which when executed by a processor implements the magnetic resonance image reality scenario augmentation method of any one of claims 1-7.

Description

Magnetic resonance image reality scene enhancement method and device and electronic equipment Technical Field The application belongs to the technical field of augmented reality, and particularly relates to a magnetic resonance image reality scene augmentation method, a device and electronic equipment. Background With the development of medical image technology, in a medical surgical scene, a real-time magnetic resonance image generated by a magnetic resonance imaging (magnetic resonance imaging) device is acquired to perform real-time observation of an intraoperative condition. However, in general, only real-time magnetic resonance images can be acquired alone or images of the intraoperative scene can be acquired alone, which results in poor observation effects on real-time conditions of the intraoperative object and instrument. Disclosure of Invention The present application aims to solve at least one of the technical problems existing in the prior art. Therefore, the application provides a method, a device and electronic equipment for enhancing a magnetic resonance image reality scene, which are used for solving the problem of poor observation effect on real-time conditions of objects in a medical scene in the related technology. In a first aspect, the present application provides a magnetic resonance image reality scene enhancement method, the method comprising: acquiring a real-time visual image and a real-time magnetic resonance image of a target object in a medical scene; Acquiring a real three-dimensional model of the target object based on the real-time visual image, and acquiring a magnetic resonance three-dimensional model of the target object based on the real-time magnetic resonance image; And carrying out three-dimensional fusion operation on the real three-dimensional model and the magnetic resonance three-dimensional model to obtain an augmented reality view of the target object. According to the magnetic resonance image real scene enhancement method, a real-time visual image and a real-time magnetic resonance image of a target object in a medical scene are acquired, a real three-dimensional model of the target object is acquired based on the real-time visual image, a magnetic resonance three-dimensional model of the target object is acquired based on the real-time magnetic resonance image, three-dimensional fusion operation is carried out on the real three-dimensional model and the magnetic resonance three-dimensional model, an augmented reality view of the target object is acquired, the real-time visual image and the real-time magnetic resonance image are fused, the augmented reality view of the target object is acquired, the surface layer condition of the target object in the medical scene can be observed through the augmented reality view, the internal condition of the target object can be observed, and the observation effect in the medical scene is improved. According to one embodiment of the application, the real-time magnetic resonance image comprises a plurality of two-dimensional magnetic resonance slice data, the real-time visual image is based on the real-time visual image to acquire a real three-dimensional model of the target object, and the real-time magnetic resonance image is based on the real-time magnetic resonance image to acquire the magnetic resonance three-dimensional model of the target object, and the real-time visual image comprises the following steps: based on the medical image interaction tool pack and the visualization tool pack, three-dimensional reconstruction is carried out on the plurality of two-dimensional magnetic resonance slice data, and a magnetic resonance three-dimensional model is obtained. According to one embodiment of the present application, acquiring a real-time three-dimensional model of a target object based on a real-time visual image and acquiring a magnetic resonance three-dimensional model of the target object based on a real-time magnetic resonance image includes: Converting the real-time visual image based on the target data processing library to obtain a three-dimensional point cloud model; Based on the three-dimensional point cloud model, a real three-dimensional model is obtained. According to one embodiment of the present application, performing a three-dimensional fusion operation on a real three-dimensional model and a magnetic resonance three-dimensional model to obtain an augmented reality view of a target object includes: and based on the target transformation matrix, performing three-dimensional fusion operation on the real three-dimensional model and the magnetic resonance three-dimensional model, and obtaining an augmented reality view of the target object. According to one embodiment of the present application, after performing a three-dimensional fusion operation on a real three-dimensional model and a magnetic resonance three-dimensional model to obtain an augmented reality view of a target object, the method includes: In