CN-122023273-A - CMR heart image intelligent analysis device based on multiple modes

CN122023273ACN 122023273 ACN122023273 ACN 122023273ACN-122023273-A

Abstract

The invention provides a CMR (complementary metal-oxide-semiconductor) heart image intelligent analysis device based on multiple modes, which relates to the technical field of medical treatment, and comprises a coordination module, an execution module and a report generation module, wherein the coordination module is used for analyzing task execution instructions input by a user and CMR heart images of multiple modes through a visual language big model to obtain analysis results, generating an execution plan based on the analysis results, calling each target expert model based on the execution plan, the execution module is used for obtaining the analysis results based on corresponding input information through the target expert model, and the report generation module is used for generating a diagnosis report based on the analysis results of each target expert model. According to the invention, the task execution instruction and CMR heart images of a plurality of modes can be analyzed by the VLM, an execution plan is generated based on the analysis result, and finally, a diagnosis report is comprehensively generated based on the analysis result of the target expert model corresponding to the CMR product line, so that multi-task and cross-mode analysis is realized, and the accuracy of intelligent diagnosis is improved.

Inventors

ZHANG HENGGUI
XU LEI
QU TAIPING
ZHANG HONGKAI
LIU HUAN
ZHAO PENGFEI
ZHANG NAN
Bo Kairui
JIANG KUN

Assignees

北京智源人工智能研究院
首都医科大学附属北京安贞医院

Dates

Publication Date: 20260512
Application Date: 20251231

Claims (10)

1. CMR heart image intelligent analysis device based on multimode, characterized by comprising: The interaction module is used for receiving task execution instructions input by a user and CMR heart images of a plurality of modes; the coordination module is used for analyzing the task execution instruction and the CMR heart images of the multiple modes through the visual language large model VLM to obtain an analysis result, wherein the analysis result comprises a task type of at least one target task, a target expert model corresponding to the target task in a CMR product line, an execution sequence and a dependency relationship of each target expert model; The coordination module is further used for generating an execution plan based on the task types, the CMR product line, the target expert models and the execution sequence and the dependency relationship of the target expert models; the coordination module is further used for calling each target expert model based on the execution plan and determining input information required by each target expert model, wherein the input information comprises CMR heart images of at least one mode and/or analysis results output by other target expert models; the execution module is used for obtaining an analysis result based on corresponding input information through the target expert model aiming at each target expert model; And the report generation module is used for generating a diagnosis report based on the analysis result of each target expert model.
2. The intelligent multi-modality-based CMR cardiac image analysis device of claim 1, wherein the coordination module is specifically configured to: Identifying the task execution instruction and the CMR heart images of the multiple modes through the VLM to obtain key entities, wherein the key entities comprise image types, analysis requirements and diagnosis targets; determining a user intent based on the image type, analysis requirements, and diagnostic targets; Determining a task type of at least one target task, a target expert model corresponding to the target task in the CMR product line, an execution sequence and a dependency relationship of each target expert model based on the image type, the analysis requirement, the diagnosis target and the user intention; And determining the CMR product line, the target expert model, the execution sequence and the dependency relationship of each target expert model as the analysis result.
3. The intelligent multi-modality-based CMR cardiac image analysis device of claim 2, wherein the coordination module is specifically configured to: Determining the priority of the target task corresponding to each task type, the execution sequence of each target expert model and the dependency relationship based on the execution plan; and calling a target expert model corresponding to each target task based on each priority, each execution sequence and the dependency relationship.
4. The intelligent multi-modality-based CMR cardiac image analysis device of claim 1, wherein the target expert model comprises a cardiac segmentation expert model comprising a region detection module, a multi-scale segmentation module, a motion feature enhanced attention module, an anatomical structure adaptive attention module, and a calculation module, wherein the input information corresponding to the cardiac segmentation expert model comprises a first cardiac image comprising CMR cardiac images of a plurality of modalities; the execution module is specifically configured to: Inputting the first heart image into the region detection module to obtain a full heart region of interest output by the region detection module; inputting the whole heart region of interest into the multi-scale segmentation module to obtain a plurality of heart segmentation regions output by the multi-scale segmentation module; inputting the first heart image into the motion feature attention-enhancing module to obtain a motion abnormal region output by the motion feature attention-enhancing module; Inputting the first heart image into the anatomic structure self-adaptive attention module to obtain an abnormal anatomic region output by the anatomic structure self-adaptive attention module; inputting the plurality of heart segmentation areas into the calculation module to obtain the corresponding function parameter values of the heart segmentation areas output by the calculation module; and determining the plurality of heart segmentation areas, the motion abnormality area, the abnormal anatomical area and each of the functional parameter values as an analysis result of the heart segmentation expert model.
5. The intelligent multi-modal-based CMR cardiac image analysis device of claim 4, wherein the target expert model comprises a diagnostic classification model comprising at least one modal feature extraction module, a cross-modal contrast learning module, a decision fusion module, and a classification diagnostic module, wherein the input information corresponding to the diagnostic classification model comprises at least two modal CMR cardiac images and the analysis result of the cardiac segmentation expert model; the execution module is specifically configured to: inputting the CMR heart images of the modes into a characteristic extraction module of the modes aiming at the CMR heart images of the modes to obtain image characteristics output by the characteristic extraction module of the modes; inputting CMR image features of all the modes into the cross-mode contrast learning module, and calibrating and complementing the CMR image features of all the modes through the cross-mode contrast learning module to obtain fusion features after calibration and complementation; Inputting the calibrated and complementary fusion characteristics into the decision fusion module to obtain total fusion characteristics output by the decision fusion module; Inputting the analysis results of the total fusion features and the heart segmentation expert model into the classification diagnosis module to obtain classification diagnosis results output by the classification diagnosis module; and determining the classified diagnosis result as an analysis result of the diagnosis classification model.
6. The intelligent multi-modality-based CMR cardiac image analysis device of claim 1, wherein the report generation module is specifically configured to: Carrying out standardized integration on the analysis results of each target expert model to obtain analysis results after each integration, wherein the standardized integration comprises at least one of data format standardized integration, unit standardized integration and term standardized integration; Inputting each integrated analysis result into the VLM, obtaining a logic relationship between each integrated analysis result through the VLM, and generating the diagnosis report based on the logic relationship according to a preset report template, wherein the preset report template comprises patient information, examination information, diagnosis conclusion and diagnosis suggestion.
7. The intelligent CMR heart image analysis device based on the multiple modes according to claim 1, further comprising a question-answering module, wherein the question-answering module is used for: receiving problem information input by a user; inputting the problem information into a reinforcement learning model, and calling an expert model related to the user problem and the target CMR heart image when the reinforcement learning model determines that the problem information comprises the user problem and the target CMR heart image, and inputting the target CMR heart image into the expert model related to the user problem and the target CMR heart image to obtain a target analysis result; Inputting the user questions and the target analysis results into the VLM, searching at least one first knowledge segment related to the user questions and the target analysis results in a medical knowledge base through the VLM, and generating answers corresponding to the question information based on the first knowledge segments.
8. The intelligent multi-modality based CMR cardiac image analysis device of claim 7, wherein the question answering module is further configured to: Inputting the question information into a reinforcement learning model, inputting the user question into the VLM when the reinforcement learning model determines that the question information comprises the user question, searching at least one second knowledge segment related to the user question in a medical knowledge base through the VLM, and generating an answer corresponding to the question information based on each second knowledge segment.
9. The intelligent multi-modality based CMR cardiac image analysis device of any of claims 1-8, wherein the CMR product line comprises a CMR product line that performs various functions of cardiac nuclear magnetism.
10. The intelligent multi-modality based CMR cardiac image analysis device of any of claims 1-8, wherein the CMR product line comprises at least one task, each task comprising a corresponding expert model.

Description

CMR heart image intelligent analysis device based on multiple modes Technical Field The invention relates to the technical field of medical treatment, in particular to a CMR heart image intelligent analysis device based on multiple modes. Background In medical application scenarios, cardiac image analysis requires high accuracy and stability. The doctor needs to make clinical decisions based on accurate measurements of image segmentation, lesion detection, and key parameters, such as ejection fraction (Ejection Fraction, EF), end-diastole Volume (EDV), left ventricular myocardial mass (Left Ventricular Mass, LVM), etc. at the time of diagnosis. Although the traditional artificial intelligence model can obtain higher precision in a single task, the traditional artificial intelligence model has defects in multi-mode and multi-task collaborative analysis. Disclosure of Invention The invention provides a CMR (complementary metal-oxide-semiconductor) heart image intelligent analysis device based on multiple modes, which is used for solving the defect that the prior art has defects in the process of multi-mode and multi-task collaborative analysis. The invention provides a CMR heart image intelligent analysis device based on multiple modes, which comprises: The interaction module is used for receiving task execution instructions input by a user and CMR heart images of a plurality of modes; the coordination module is used for analyzing the task execution instruction and the CMR heart images of the multiple modes through a Visual Language Model (VLM) to obtain an analysis result, wherein the analysis result comprises a task type of at least one target task, a target expert Model corresponding to the target task in a CMR product line, and an execution sequence and a dependency relationship of each target expert Model; The coordination module is further used for generating an execution plan based on the task types, the CMR product line, the target expert models and the execution sequence and the dependency relationship of the target expert models; the coordination module is further used for calling each target expert model based on the execution plan and determining input information required by each target expert model, wherein the input information comprises CMR heart images of at least one mode and/or analysis results output by other target expert models; the execution module is used for obtaining an analysis result based on corresponding input information through the target expert model aiming at each target expert model; And the report generation module is used for generating a diagnosis report based on the analysis result of each target expert model. According to the CMR heart image intelligent analysis device based on the multiple modes, the coordination module is specifically used for: Identifying the task execution instruction and the CMR heart images of the multiple modes through the VLM to obtain key entities, wherein the key entities comprise image types, analysis requirements and diagnosis targets; determining a user intent based on the image type, analysis requirements, and diagnostic targets; Determining a task type of at least one target task, a target expert model corresponding to the target task in the CMR product line, an execution sequence and a dependency relationship of each target expert model based on the image type, the analysis requirement, the diagnosis target and the user intention; And determining the CMR product line, the target expert model, the execution sequence and the dependency relationship of each target expert model as the analysis result. According to the CMR heart image intelligent analysis device based on the multiple modes, the coordination module is specifically used for: Determining the priority of the target task corresponding to each task type, the execution sequence of each target expert model and the dependency relationship based on the execution plan; and calling a target expert model corresponding to each target task based on each priority, each execution sequence and the dependency relationship. According to the intelligent CMR heart image analysis device based on the multiple modes, the target expert model comprises a heart segmentation expert model, wherein the heart segmentation expert model comprises a region detection module, a multiple-scale segmentation module, a motion characteristic enhancement attention module, an anatomical structure self-adaptation attention module and a calculation module, input information corresponding to the heart segmentation expert model comprises a first heart image, and the first heart image comprises CMR heart images of multiple modes; the execution module is specifically configured to: Inputting the first heart image into the region detection module to obtain a full heart region of interest output by the region detection module; inputting the whole heart region of interest into the multi-scale segmentation module to obtain a plu