CN-122025164-A - Heart oversensing method based on multi-mode large model workflow

CN122025164ACN 122025164 ACN122025164 ACN 122025164ACN-122025164-A

Abstract

The invention discloses a heart super-survey method based on a multi-mode large model workflow, and belongs to the technical field of medical image processing. The method solves the problems of low accuracy of measured values and low report generation efficiency in the traditional heart ultrasonic examination. The technical scheme includes that an echocardiographic video stream is read to obtain a sequence to be identified, data preprocessing is conducted, section type information comprising a four-cavity section of the apex of the heart, a two-cavity section of the apex of the heart and a long-axis section beside a sternum is obtained through section classification and quality control models, section type information and the past medical history of a patient are input into a large language model to generate a calling instruction, a corresponding vision parameter measurement model is called according to the instruction to finish heart structure parameter measurement, measured value information is input into the large language model after being summarized, and a structured heart ultrasound examination report is automatically generated by combining measured value parameters and clinical information. According to the method, multi-mode collaborative processing is realized through multi-mode information fusion and intelligent scheduling, and the accuracy of the cardiac superconductivity value and the normalization of the diagnosis report are improved.

Inventors

Shan Chunjie
Qian Sunnan
LUO SHOUHUA
ZHANG KUI
XU FANG
CHENG HANLIN
CHEN YIDI
Weng Hexiang
GONG XULONG
YAO JING

Assignees

苏州海斯菲德信息科技有限公司

Dates

Publication Date: 20260512
Application Date: 20251231

Claims (7)

1. A heart supersensing method based on a multi-mode large model workflow is characterized by comprising the following steps: Reading an ultrasonic cardiogram video stream to obtain an ultrasonic cardiogram sequence to be identified; Inputting the sequence to be identified into a data preprocessing module for data preprocessing; inputting the preprocessed sequence into a section classification and quality control model to obtain section type information; inputting the section type information and the past medical history of the patient into a large language model, and generating an instruction for calling a visual measurement model; Calling a corresponding visual parameter measurement model according to the instruction information, so that measurement of heart structural parameters can be dynamically completed; And summarizing the measured value information, inputting the summarized measured value information into a large language model, and automatically generating a structured heart ultrasonic examination report which accords with medical specifications based on measured value parameters and clinical information.
2. The method for cardiac supersurvey based on multi-mode large model workflow of claim 1, wherein the section classification firstly preprocesses the acquired cardiac supervideo, and automatically identifies different cardiac section types and imaging quality in the video by using a trained multi-task section classification and quality assessment model, thereby laying a foundation for targeted invocation of a subsequent survey model.
3. The method of claim 2, wherein the cardiac surface types include a four-chamber heart surface, a two-chamber heart surface, and a parasternal long axis surface.
4. The multi-modal large model workflow based cardiac supercondensation method of claim 1, wherein: In addition to the section type and the image quality score, multi-mode information including past medical history, clinical indexes and electrocardiogram scanning result information of the patient is synchronously acquired and fused, and the acquired information is unified as a prompt word and is input into a large-scale language model for arrangement and analysis.
5. The multi-modal large model workflow based cardiac supercondensation method of claim 4, wherein: The large language model dynamically judges the combination and the calling sequence of the applicable measured value models in the cases based on the input multi-mode information, and the system calls a plurality of vision measurement models aiming at specific sections and parameters through intelligent scheduling to realize the cooperative processing of the multiple models.
6. The multi-modal large model workflow based cardiac supercondensation method of claim 1, wherein: and the parameter results obtained by the vision measurement models are subjected to unified formatting treatment, summarized and checked by a system, and the results output by different models are comprehensively considered.
7. The multi-modal large model workflow based cardiac supercondensation method of claim 6, wherein: And (3) after the summarized measurement results are combined with clinical information to be arranged, inputting the measurement results into a large language model to be used as prompt words, and automatically writing a structured echocardiographic diagnosis report which accords with medical specifications.

Description

Heart oversensing method based on multi-mode large model workflow Technical Field The invention relates to a medical image processing method, in particular to a method for automatically measuring and generating reports of echocardiographic parameters based on a multi-mode large model workflow. Background Heart echocardiography is used as an important imaging examination means for clinical diagnosis of cardiovascular diseases, and can observe the structure and function of the heart in real time and noninvasively. With the rapid development of artificial intelligence technology, an echocardiography automatic analysis system based on deep learning gradually becomes a research hot spot, and aims to improve diagnosis efficiency and accuracy and reduce the workload of doctors. In the prior art, a plurality of technical schemes for automatic analysis of an ultrasonic cardiogram exist. Chinese patent CN117562583B discloses an artificial intelligence aided heart function detection system and method, the system acquires an echocardiogram and an electrocardiogram of a patient, and finally determines the heart function health level by extracting image features and performing cross-modal interaction processing. Chinese patent CN119055278B discloses a left ventricular ejection fraction prediction method based on echocardiography, which obtains a section classification result through a classification model, and then invokes a corresponding segmentation model and prediction model to complete LVEF calculation. Chinese patent CN120410967A proposes an automatic quantitative evaluation method of ultrasonic images based on artificial intelligence, which obtains the mode type and the section type of the ultrasonic images through a depth image classification model and a target detection model, and provides corresponding quantitative analysis modes for different sections. In the aspect of section identification and parameter measurement, chinese patent CN111012377B discloses an ultrasonic cardiogram heart parameter calculation and myocardial strain measurement method, and the trained neural network is used for carrying out automatic section classification processing and image segmentation on a heart ultrasonic video so as to obtain a heart parameter and myocardial strain measurement result. Chinese patent CN116452899a describes an echocardiographic standard section identification and scoring method based on deep learning, establishes a section classification dataset and an image quality scoring model, and realizes automatic identification and quality assessment of sections. However, existing automatic diagnostic systems for cardiac echocardiography still have significant drawbacks in practical applications. Firstly, the existing system mostly adopts fixed rules and preset flows when a measured value model is called, the scheduling strategy is relatively single, and the capability of dynamic adjustment according to the characteristics of specific cases is lacking. Secondly, the prior art mainly focuses on the processing of the images, and the comprehensive utilization of multi-mode information such as the prior medical history and clinical indexes of patients is insufficient, so that the advantages of multi-source information fusion can not be fully exerted. In addition, the intelligent degree of the measurement flow of the existing system is limited, and the measurement strategy is difficult to flexibly adjust according to factors such as image quality, section type, patient clinical information and the like, so that the adaptability is insufficient when facing complex and changeable clinical scenes, and the accuracy and efficiency of automatic cardiac ultrasonic diagnosis are affected. Disclosure of Invention The heart ultra-measurement method based on the multi-mode large model workflow aims at solving the technical problems that an existing heart ultrasonic cardiogram automatic diagnosis system mostly adopts fixed rules when a measured value model is called, a scheduling strategy is single, and the comprehensive utilization of multi-mode information such as image quality, past medical history of a patient and the like is lacked, so that scheduling is not flexible and intelligent enough, complex and changeable clinical scenes are difficult to adapt to, the measurement flow is not intelligent enough, the accuracy and efficiency of heart ultrasonic automatic diagnosis are affected, intelligent scheduling of the multi-mode model cluster based on a large language model is realized, the intelligence and the adaptability of model scheduling are remarkably improved, the stability and the personalized diagnosis capability of a measurement result are enhanced, and the accuracy and the efficiency of heart ultrasonic automatic diagnosis are improved. The technical scheme adopted for solving the technical problems is that a heart supersensing method based on a multi-mode large model workflow is provided, and comprises the following steps: R