CN-122023441-A - Rendering processing method, system, equipment and medium based on intelligent segmentation of medical images

CN122023441ACN 122023441 ACN122023441 ACN 122023441ACN-122023441-A

Abstract

The invention provides a rendering processing method, a rendering processing system, rendering processing equipment and a rendering processing medium based on intelligent segmentation of medical images. The rendering processing method comprises the steps of obtaining semantic confidence distribution of each voxel in a target image by utilizing a semantic interaction model, and carrying out semantic interaction on each voxel Defining a multi-modal input signal Wherein Gray scale or intensity information from different image modalities is represented; determining the semantic weight of each voxel according to the semantic interaction model According to each voxel Multimodal input signal And semantic weights And determining the corresponding illumination intensity and transparency to render the target image.

Inventors

CHEN DUANDUAN
Qin Linyu
MAO PENGZHI
LI SHILONG
LI HAOZHENG
LIANG SHICHAO
ZHANG XUEHUAN
ZHANG SHUAITONG
GAO GE

Assignees

北京理工大学

Dates

Publication Date: 20260512
Application Date: 20260126

Claims (10)

1. A rendering processing method based on medical image segmentation, comprising: correlating semantic feature vectors obtained from natural language task instructions Image feature vector in target image To generate joint feature vectors , wherein, Representing voxel coordinate points of the image; Based on semantic feature vectors with highest confidence Generating initial values of a configuration parameter set by the corresponding joint feature vectors, wherein the configuration parameter set comprises one or more segmentation parameters; according to the consistency between the values of the segmentation parameters in the configuration parameter set and the image characteristic reference values, a clarification question is initiated to a user so as to obtain updated values of the segmentation parameters in the configuration parameter set; Performing segmentation processing on the target image according to the determined updated values of each segmentation parameter in the configuration parameter set to obtain a blood vessel segmentation result, wherein the blood vessel segmentation result comprises a blood vessel three-dimensional voxel structure; Determining a key region in the three-dimensional voxel structure of the blood vessel according to the natural language task instruction and the feedback of the user on the clarification question, rendering the three-dimensional voxel structure of the blood vessel based on a physical volume rendering method, and performing enhanced rendering on the key region, and And determining a hemodynamic parameter based on the blood vessel segmentation result, so as to render the three-dimensional voxel structure of the blood vessel according to the hemodynamic parameter and a cardiac cycle, wherein the hemodynamic parameter at least comprises streamline data and space coordinates of a blood flow streamline node.
2. The method of claim 1, wherein the determining key regions in the vessel three-dimensional voxel structure from the natural language task instructions and user feedback on the clarification question comprises: Determining semantic nodes of the user for the rendered key requirements according to the natural language task instruction and feedback of the user for the clarification question; Associating semantic nodes of the critical demand with the vessel three-dimensional voxel structures through a graph attention network (GAT) in which attention coefficients between each vessel three-dimensional voxel node in a vessel three-dimensional voxel structure and semantic nodes in its anatomy As shown in the following formula: Wherein the method comprises the steps of And Feature vectors representing three-dimensional voxel nodes and semantic nodes respectively, In order for the weight matrix to be learnable, Is an attention parameter vector, and And determining a key region in the three-dimensional vascular voxel structure according to the attention coefficient corresponding to each three-dimensional vascular voxel node in the three-dimensional vascular voxel structure.
3. The method of claim 1 or 2, wherein the performing a segmentation process on the target image according to the determined updated values of the respective segmentation parameters in the set of configuration parameters to obtain a vessel segmentation result comprises: Performing a pre-segmentation model on the target image according to the determined updated values of the segmentation parameters in the configuration parameter set to obtain a pre-segmentation result, and A complete segmentation result for the target image is obtained based on the shown pre-segmentation result using the first computational network, and a straightened vessel model is obtained based on the shown complete segmentation result using the second computational network to obtain a simplified segmentation result.
4. A method according to claim 3, wherein The simplified segmentation results appear as a three-dimensional point cloud, Extracting hemodynamic parameters based on the simplified segmentation result includes: Inputting an internal point cloud of the reduced segmented result three-dimensional point cloud to a first channel of a dynamic graph rolling network (DGCNN), and inputting a surface point cloud of the reduced segmented result three-dimensional point cloud to a second channel of DGCNN, wherein the first channel is independent of the second channel; The first channel and the second channel respectively perform feature coding on the input point clouds, wherein the surface point clouds in the second channel are subjected to feature aggregation to form global feature tensors; And carrying out feature fusion on feature coding results of the first channel and the second channel, wherein the global features are spliced with point-by-point features of an internal point cloud through a broadcasting mechanism to form a comprehensive feature representation comprising local geometric details and global structure perception.
5. The method of claim 1 or 2, wherein the determining a hemodynamic parameter based on the vessel segmentation result, to render the vessel three-dimensional voxel structure in accordance with the hemodynamic parameter in accordance with a cardiac cycle, comprises: drawing the blood flow streamline nodes on a three-dimensional voxel structure according to the topological sequence of the blood flow streamline nodes according to the space coordinates of the blood flow streamline nodes; a separate data structure representing a set of lines is generated for each frame in the cardiac cycle, wherein each data structure representing a set of lines is formed by a respective flow streamline node in the three-dimensional voxel structure.
6. The method of claim 1, wherein Obtaining the semantic feature vectors according to natural language task instructions through a large language model interface (API), The method further comprises the steps of: comparing one or more of a first rendering result obtained by rendering the three-dimensional voxel structure of the blood vessel, a second rendering result obtained by performing enhanced drawing on the key region, and a third rendering result obtained by rendering the three-dimensional voxel structure of the blood vessel according to the hemodynamic parameter and the cardiac cycle with corresponding reference rendering results to obtain a rendering loss parameter, and Training the large language model interface according to the rendering loss parameters.
7. A method according to claim 3, the method further comprising: comparing one or more of a first rendering result obtained by rendering the three-dimensional voxel structure of the blood vessel, a second rendering result obtained by performing enhanced drawing on the key region, and a third rendering result obtained by rendering the three-dimensional voxel structure of the blood vessel according to the hemodynamic parameter and the cardiac cycle with corresponding reference rendering results to obtain a rendering loss parameter, and Training the first network and the second network according to the rendering loss parameters.
8. A rendering processing system based on medical image segmentation, comprising: The data management module is configured to be connected with the image database so as to call a corresponding target image from the image database according to an input instruction; An interactive reasoning big language model module configured to correlate semantic feature vectors obtained according to natural language task instructions Image feature vector in target image To generate joint feature vectors According to the semantic feature vector with highest confidence Generating initial values of a configuration parameter set by corresponding joint feature vectors, initiating a clarification question to a user according to consistency between values of all segmentation parameters in the configuration parameter set and image feature reference values to obtain updated values of all segmentation parameters in the configuration parameter set, Representing image voxel coordinate points, wherein the configuration parameter set comprises one or more segmentation parameters; an image segmentation module configured to perform segmentation processing on a target image according to the determined updated values of each segmentation parameter in the configuration parameter set to obtain a blood vessel segmentation result, wherein the blood vessel segmentation result comprises a blood vessel three-dimensional voxel structure; An image rendering module configured to determine a critical region in the three-dimensional voxel structure of the blood vessel based on the natural language task instruction and the clarification question by a user, render the three-dimensional voxel structure of the blood vessel based on a physical volume rendering method, and enhance the critical region, and A parameter extraction module configured to determine a hemodynamic parameter based on the vessel segmentation result, wherein the hemodynamic parameter comprises at least streamline data and spatial coordinates of a blood flow streamline node, The image rendering module is further configured to render the three-dimensional voxel structure of the blood vessel according to the cardiac cycle according to the hemodynamic parameters determined by the parameter extraction module.
9. An electronic device comprising a processor, a storage medium and a bus, the storage medium storing machine-readable instructions executable by the processor, the processor in communication with the storage medium via the bus when the electronic device is in operation, the processor executing the machine-readable instructions to perform the steps of the medical image segmentation based rendering method of any one of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, performs the steps of the rendering processing method based on medical image segmentation as claimed in any one of claims 1 to 7.

Description

Rendering processing method, system, equipment and medium based on intelligent segmentation of medical images Technical Field The present disclosure relates to the field of data processing, and more particularly, to a rendering processing method, system, device, and medium based on medical image segmentation. Background Diagnosis and treatment of cardiovascular and cerebrovascular diseases generally requires precise localization of lesion details. Taking aortic dissection as an example, the incision position and the false cavity range directly determine the operation scheme, are consistent with the influence logic of the width and the position of the neck of the cerebral aneurysm on the operation, and all need the image technology to clearly show the relation between lesions and surrounding structures. The image reconstruction and the fine rendering are the data basis of the fine rendering of the cardiovascular and cerebrovascular images, the structuring degree of the data base directly determines the clinical value of the rendering, and the data base runs through the whole diagnosis and treatment process and interpretation habit of doctors. For example, in the emergency stage, the two-dimensional image needs to be converted into a three-dimensional structured model for image reconstruction, and lesion targets (such as aortic orifice and cerebral aneurysm) are clearly marked so as to help emergency doctors to quickly locate key lesions. For another example, during the planning phase of the procedure, the reconstruction needs to preserve details of vessel wall thickness, branch opening locations, etc. to support the transparency rendering. For example, coronary artery CTA reconstruction can show the relationship between plaque and vessel lumen by transluciding the vessel wall, and aortic reconstruction can translucide the true lumen, judging the distance between stent and branch vessel. In addition, in the follow-up evaluation stage, standardized morphological parameters (such as aortic diameter and cerebral infarction focus volume) need to be generated for reconstruction, so that the comparison of rendering results and historical data is facilitated. In clinical practice, doctors need to quickly identify critical information of pathological changes, such as positioning aortic orifice and cerebral infarction occlusion blood vessel in emergency, and the existing image post-processing mode is difficult to meet the interpretation requirement of 'quick and accurate'. Therefore, a new rendering processing method based on medical image segmentation is needed to improve the above technical problems. Disclosure of Invention In view of the above, the present disclosure provides an image processing method, system, device, and medium based on a semantic interaction model. According to one aspect of the present disclosure, there is provided a rendering processing method based on medical image segmentation, including correlating semantic feature vectors obtained according to natural language task instructionsImage feature vector in target imageTo generate joint feature vectors, wherein,Representing voxel coordinate points of the image according to the semantic feature vector with highest confidence coefficientThe method comprises the steps of generating initial values of a configuration parameter set, wherein the configuration parameter set comprises one or more segmentation parameters, initiating a clarification question to a user according to consistency between values of all segmentation parameters in the configuration parameter set and image characteristic reference values to obtain updated values of all segmentation parameters in the configuration parameter set, executing segmentation processing on the target image according to the determined updated values of all segmentation parameters in the configuration parameter set to obtain a blood vessel segmentation result, wherein the blood vessel segmentation result comprises a blood vessel three-dimensional voxel structure, determining a key region in the blood vessel three-dimensional voxel structure according to a natural language task instruction and feedback of the user on the clarification question, rendering the blood vessel three-dimensional voxel structure based on a physical volume rendering method, and performing enhanced rendering on the key region, and determining a blood flow dynamics parameter based on the blood vessel segmentation result to render the blood vessel three-dimensional voxel structure according to a cardiac cycle according to the blood flow dynamics parameter, wherein the blood flow dynamics parameter at least comprises streamline data and space coordinates of a blood flow streamline node. According to one aspect of the disclosure, determining critical areas in the three-dimensional voxel structure of a vessel based on the natural language task instructions and user feedback to the clarification question includes determining semantic nodes of