Search

CN-121981999-A - Image processing method, system, equipment and medium based on semantic interaction model

CN121981999ACN 121981999 ACN121981999 ACN 121981999ACN-121981999-A

Abstract

The disclosure provides an image processing method, system, equipment and medium based on a semantic interaction reasoning big model. The method comprises the steps of associating semantic feature vectors obtained according to natural language task instructions with image feature vectors in a target image to generate joint feature vectors, generating initial values of a configuration parameter set according to the joint feature vectors corresponding to the semantic feature vectors with highest confidence, wherein the configuration parameter set comprises one or more segmentation parameters, initiating a clarification question to a user according to consistency between values of the segmentation parameters and image feature reference values to obtain updated values of the segmentation parameters, executing a pre-segmentation model on the target image according to the determined updated values of the segmentation parameters to obtain a pre-segmentation result, obtaining a complete segmentation result for the target image based on the pre-segmentation result, and obtaining a straightened blood vessel model based on the complete segmentation result to obtain a simplified segmentation result.

Inventors

  • CHEN DUANDUAN

Assignees

  • 北京理工大学

Dates

Publication Date
20260505
Application Date
20260126

Claims (10)

  1. 1. An image processing method based on a semantic interaction model, comprising the following steps: correlating semantic feature vectors obtained from natural language task instructions Image feature vector in target image To generate joint feature vectors , wherein, Representing voxel coordinate points of the image; Based on semantic feature vectors with highest confidence Generating initial values of a configuration parameter set by the corresponding joint feature vectors, wherein the configuration parameter set comprises one or more segmentation parameters; according to the consistency between the values of the segmentation parameters in the configuration parameter set and the image characteristic reference values, a clarification question is initiated to a user so as to obtain updated values of the segmentation parameters in the configuration parameter set; Performing a pre-segmentation model on the target image according to the determined updated values of the segmentation parameters in the configuration parameter set to obtain a pre-segmentation result, and A complete segmentation result for the target image is obtained based on the shown pre-segmentation result using the first computational network, and a straightened vessel model is obtained based on the shown complete segmentation result using the second computational network to obtain a simplified segmentation result.
  2. 2. The method of claim 1, wherein the initiating a clarification question to a user to obtain updated values of each segmentation parameter in the set of configuration parameters based on consistency between values of each segmentation parameter in the set of configuration parameters and image feature reference values comprises: determining whether the consistency between the initial value of each segmentation parameter in the configuration parameter set and the image characteristic reference value meets a preset threshold value; when the consistency between the values of the partition parameters in the configuration parameter set and the image characteristic reference values does not meet a preset threshold value, a clarification question is initiated to a user; Updating the values of parameters in the configuration parameter set according to feedback of a user for the clarification question until the consistency between the values of at least a part of the segmentation parameters in the configuration parameter set and the image characteristic reference value meets a preset threshold.
  3. 3. The method of claim 2, wherein the determining whether a correspondence between initial values of respective segmentation parameters in the set of configuration parameters and image feature reference values meets a predetermined threshold comprises: according to the set of configuration parameters Each of the dividing parameters of Is added to the corresponding image characteristic reference value, based on the following consistency detection function And (3) carrying out consistency detection: Wherein, the As the weight coefficient of each parameter, In order to divide the parameters of the object, Image feature reference values recommended by an image automatic detection module or a knowledge graph, and When (when) When a clarification question is initiated to the user.
  4. 4. The method of claim 1, wherein the initiating a clarification question to a user to obtain updated values of each segmentation parameter in the set of configuration parameters based on consistency between values of each segmentation parameter in the set of configuration parameters and image feature reference values comprises: Determining whether a value of at least a portion of the segmentation parameters in the set of configuration parameters exceeds a confidence threshold, wherein the confidence threshold is a confidence threshold for semantic understanding; initiating a clarification question to the user when the value of at least a portion of the segmentation parameters in the set of configuration parameters does not exceed the confidence threshold value, and And updating the configuration parameter set according to feedback of the user for the clarification question until the value of at least one part of the segmentation parameters in the configuration parameter set exceeds a confidence threshold.
  5. 5. The method of any of claims 1-4, wherein using the second computing network to obtain a straightened vessel model based on the complete segmentation result to obtain a simplified segmentation result comprises: performing skeleton line calculation of the blood vessel on the voxel structure generated by the first calculation network segmentation to form a discrete skeleton line of the step size of the blood vessel; Performing skeleton smoothing on discrete skeleton lines of the blood vessel based on the loss function; Extracting a cross section at the center point of each smoothed skeleton line of the blood vessel, and randomly establishing a 2D coordinate system on the cross section; combining all cross sections and rotating a randomly generated 2D coordinate system over the cross sections; And carrying out straightening mapping on the blood vessels according to the combined and rotated cross sections to generate straightened images, and carrying out segmentation and inverse transformation processing on the straightened images to obtain simplified segmentation results.
  6. 6. The method of claim 5, wherein the rotating the randomly generated 2D coordinate system over the cross-section comprises: The randomly generated 2D coordinate system over the cross section is rotated using the following loss function: Wherein the method comprises the steps of Represents the loss value based on the cross section, Is a set of parameterized discrete cross-sectional point sequences, The rotation parameter is indicated as such, Representing an offset or step size parameter for selecting a spacing point in the sequence of points, calculating a second order difference at different step sizes, Representing each discrete cross-section point sequence I.e. the number of points in the sequence. A sequence of points representing a single discrete cross section, belonging to One element in the collection.
  7. 7. The method of claim 5, further comprising: The sensitivity of the simplified segmentation result to different orders of magnitude is calculated to evaluate the accuracy of the calculated simplified segmentation result and when the accuracy of the simplified segmentation result meets a predetermined value, it is output as a segmentation result.
  8. 8. An image processing system based on a semantic interaction model, comprising: The data management module is configured to be connected with the image database so as to call a corresponding target image from the image database according to an input instruction; an interactive reasoning big language model module configured to correlate semantic feature vectors obtained according to natural language task instructions Image feature vector in target image To generate joint feature vectors According to the semantic feature vector with highest confidence Generating initial values of a configuration parameter set by corresponding joint feature vectors, initiating a clarification question to a user according to consistency between values of all segmentation parameters in the configuration parameter set and image feature reference values to obtain updated values of all segmentation parameters in the configuration parameter set, Representing image voxel coordinate points, wherein the configuration parameter set comprises one or more segmentation parameters; An image pre-segmentation module configured to perform a pre-segmentation model on the target image according to the determined updated values of the respective segmentation parameters in the set of configuration parameters to obtain a pre-segmentation result, and The image segmentation module is configured to obtain a complete segmentation result for the target image based on the pre-segmentation result using the first computing network, and to obtain a straightened vessel model based on the complete segmentation result using the second computing network to obtain a simplified segmentation result.
  9. 9. An electronic device comprising a processor, a storage medium and a bus, the storage medium storing machine-readable instructions executable by the processor, the processor and the storage medium communicating over the bus when the electronic device is operating, the processor executing the machine-readable instructions to perform the steps of the method of any one of claims 1 to 7 relating to a semantic interaction reasoning based large model.
  10. 10. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a computer program which, when executed by a processor, performs the steps of the intelligent image processing method according to any of claims 1 to 7 involving a semantic interaction reasoning based large model.

Description

Image processing method, system, equipment and medium based on semantic interaction model Technical Field The present disclosure relates to the field of data processing, and more particularly, to an image processing method, system, device, and medium based on a semantic interaction inference large model. Background Image reconstruction is a core support of the whole process of diagnosis and treatment-research of cardiovascular and cerebrovascular diseases. In the aspect of diagnosis, through image reconstruction such as CT angiography (Computed Tomography Angiography, CTA), magnetic resonance angiography (Magnetic Resonance Angiography, MRA) and the like, a two-dimensional image can be converted into a three-dimensional vascular model, focus (such as a cerebral aneurysm position and an aortic dissection breach) can be accurately positioned, and missed diagnosis caused by the limitation of a two-dimensional image view angle is avoided. For example, intracranial arterial CTA reconstruction can clearly show the extent of arterial stenosis in the brain, and aortic CTA reconstruction can quantify the true lumen volume. In terms of treatment, the reconstruction model provides an anatomical basis for surgery planning, for example, cerebral aneurysm occlusion needs to be determined based on reconstruction, and a pulse thoracic aortic endoluminal prosthesis (TEVAR) surgery needs to rely on reconstruction to select stent diameters and anchoring regions. In the follow-up aspect, the curative effect can be quantified through the comparison of the pre-operation reconstruction model and the post-operation reconstruction model. In addition, in the scientific research aspect, the standardized image reconstruction can unify vessel morphological parameters of a large number of patients, and a unified data base is provided for intelligent algorithm training. On the other hand, with the development of information industry, smart medical treatment using artificial intelligence technology is becoming more and more popular. The large language model is an artificial intelligence model, which is intended to understand human language and execute corresponding instructions and output feedback. In general, large language models can perform a wide range of tasks, including text summarization, reasoning, and the like, by training on a large amount of text data. Thereby relieving pressure on the healthcare worker. However, current large language models of medicine are disjoint from image reconstruction techniques. For example, current large language models can only passively respond to user questions. For example, when a user asks what is a thoracic aortic endoluminal repair (TEVAR) procedure, the current large language model can only answer the definition of TEVAR procedure based on the question, and cannot actively ask for information that causes a critical risk, such as aortic arch angle, in conjunction with image reconstruction techniques. In addition, due to the fact that the data linkage of the existing medical large language model and the image reconstruction technology is insufficient, the model cannot read the quantization parameters generated by reconstruction, and only manual input of a user can be relied on. This not only increases the cost of operation, but also is prone to distortion of the interactive information due to input errors. Therefore, a new image processing method based on semantic interaction model is needed to improve the above technical problems. Disclosure of Invention In view of the above, the present disclosure provides an image processing method, system, device, and medium based on a semantic interaction model. According to the image processing method, system, equipment and medium based on the semantic interaction model, the large model can be queried through multiple rounds of semantic understanding and multiple rounds of interaction, corresponding cases, images with corresponding dimensions and modes can be quickly and accurately invoked, and the interested areas of related images can be segmented more accurately and intelligently. According to one aspect of the present disclosure, there is provided an image processing method based on a semantic interaction model, including correlating semantic feature vectors obtained according to natural language task instructionsImage feature vector in target imageTo generate joint feature vectors, wherein,Representing voxel coordinate points of the image according to the semantic feature vector with highest confidence coefficientThe method comprises the steps of generating initial values of a configuration parameter set, wherein the configuration parameter set comprises one or more segmentation parameters, initiating a clarification question to a user according to consistency between values of all segmentation parameters in the configuration parameter set and image characteristic reference values to obtain updated values of all segmentation parameters in the configur