CN-122023603-A - Disease evolution visualization method and system based on ophthalmic multi-mode image

CN122023603ACN 122023603 ACN122023603 ACN 122023603ACN-122023603-A

Abstract

The embodiment of the application provides a disease evolution visualization method and a disease evolution visualization system based on ophthalmic multi-mode images, wherein the method comprises the steps of obtaining a multi-mode ophthalmic image sequence of a detected person with known eye disease types in a history period; the method comprises the steps of inputting multimode ophthalmic images and the prefronous multimode ophthalmic images related to the multimode ophthalmic images into a feature extraction model for feature extraction aiming at each multimode ophthalmic image in a multimode ophthalmic image sequence to obtain mask feature images of the multimode ophthalmic images, integrating the mask feature images of each multimode ophthalmic image in the multimode ophthalmic image sequence according to time sequence to obtain a lesion evolution thermodynamic diagram of an eye disease type, and displaying the eye disease evolution rule of a subject by utilizing the lesion evolution thermodynamic diagram. The application can accurately reveal the personalized evolution rule of the eye diseases of the testee in the development process.

Inventors

YAN CHUNNI
LV WENCHAO

Assignees

西安市第一医院

Dates

Publication Date: 20260512
Application Date: 20260211

Claims (10)

1. A method of visualizing disease evolution based on ophthalmic multi-modality imaging, the method comprising: Acquiring a multi-mode ophthalmic image sequence of a subject of a known eye disease type in a history period, wherein the history period is a current time and a period of preset duration before the current time; Inputting the multimode ophthalmic image and the prefrontal multimode ophthalmic image associated with the multimode ophthalmic image into a feature extraction model for feature extraction to obtain a mask feature map of the multimode ophthalmic image, wherein the feature extraction model comprises a multi-level backbone feature extraction module and a calculation module carrying lesion evolution feature thresholds of different eye disease types; Integrating mask feature images of each multi-mode ophthalmic image in the multi-mode ophthalmic image sequence according to a time sequence to obtain a lesion evolution thermodynamic diagram of the eye disease type; And displaying the eye disease evolution rule of the tested person by utilizing the lesion evolution thermodynamic diagram.
2. The method of claim 1, wherein the acquiring a sequence of multimodal ophthalmic images of a subject of known ocular disease type over a historical period of time comprises: acquiring multi-mode ophthalmic image data corresponding to a plurality of time points of the subject in the history period; Carrying out standardization processing on the multi-mode ophthalmic image data corresponding to each time point to obtain a multi-mode ophthalmic image corresponding to each time point; And sequentially integrating the multi-mode ophthalmic images corresponding to each time point to obtain the multi-mode ophthalmic image sequence.
3. The method according to claim 1 or 2, wherein the calculation module comprises a calculation sub-module deployed at an output of each of the multi-level backbone feature extraction modules and carrying lesion evolution feature thresholds for the different eye disease types; Inputting the multi-modal ophthalmic image and the precursor multi-modal ophthalmic image associated with the multi-modal ophthalmic image into a feature extraction model for feature extraction for each multi-modal ophthalmic image in the multi-modal ophthalmic image sequence to obtain a mask feature map of the multi-modal ophthalmic image, including: Inputting the multi-mode ophthalmic image into the multi-level backbone feature extraction module for feature extraction aiming at each multi-mode ophthalmic image to obtain multi-scale original features of the multi-mode ophthalmic image; for each scale original feature in the multi-scale original features, inputting the scale original feature and a precursor original feature corresponding to the scale original feature, and simultaneously inputting a calculation submodule at the same level with the scale original feature for feature processing to obtain a feature subgraph containing a dynamic evolution constraint mask corresponding to the scale original feature, wherein the precursor original feature is a feature identical to the scale of the scale original feature in the precursor moment multi-scale original features of the precursor multi-mode ophthalmic image associated with the multi-mode ophthalmic image; and integrating feature subgraphs containing dynamic evolution constraint masks corresponding to each scale original feature in the multi-scale original features according to the scale sequence to obtain mask feature graphs of the multi-mode ophthalmic images.
4. A method according to claim 3, wherein the precursor raw features corresponding to the scale raw features comprise: and under the condition that the multi-mode ophthalmic image is the multi-mode ophthalmic image at the initial moment in the multi-mode ophthalmic image sequence, the precursor original feature corresponding to the scale original feature is an eye basic feature with the same scale as the scale original feature.
5. The method of claim 3, wherein the precursor raw signature comprises a last-time raw sub-signature and a last-time raw sub-signature; for each scale original feature in the multi-scale original features, inputting the scale original feature and the preamble original feature corresponding to the scale original feature into a calculation submodule at the same level with the scale original feature for feature processing to obtain a feature subgraph containing a dynamic evolution constraint mask corresponding to the scale original feature, wherein the method comprises the following steps: Quantizing the difference between the scale original feature and the original sub-feature at the previous moment by adopting a calculation sub-module which is in the same level with the scale original feature to obtain a first feature difference value, and quantizing the difference between the scale original feature and the original sub-feature at the previous moment to obtain a second feature difference value; Processing the first characteristic difference value and the second characteristic difference value based on lesion evolution characteristic threshold values of different eye disease types to obtain dynamic evolution constraint masks corresponding to the scale original characteristics; and generating a feature subgraph containing dynamic evolution constraint masks corresponding to the scale original features.
6. The method of claim 5, wherein the lesion evolution characteristic threshold for the different ocular disease types comprises a maximum lesion variance for an ocular disease, a characteristic threshold for the known ocular disease type; the processing the first feature difference value and the second feature difference value based on the lesion evolution feature threshold values of the different eye disease types to obtain a dynamic evolution constraint mask corresponding to the scale original feature comprises: Analyzing the first characteristic difference value and the second characteristic difference value by adopting the maximum value of the lesion difference of the eye diseases to obtain the normalized difference of the scale original characteristic; inputting the typical characteristic threshold value of the known eye disease type and the normalized difference into a mask calculation formula to calculate to obtain a dynamic evolution constraint mask corresponding to the scale original characteristic, wherein the mask calculation formula is as follows: ; Wherein, the Scale primitive features of (a) A corresponding dynamic evolution constraint mask; For the scale original feature Is a normalized difference of (2); is of the known type of ocular disease Is a typical feature threshold of (1).
7. The method of claim 6, wherein analyzing the first feature difference value and the second feature difference value using a lesion difference maximum value of the ocular disease to obtain the normalized difference of the scale raw feature comprises: taking the minimum value from the first characteristic difference value and the second characteristic difference value to obtain a key difference index; and carrying out normalization treatment on the key difference index by adopting the maximum value of the pathological change difference of the eye diseases to obtain the normalization difference.
8. The method of claim 1, wherein the process of constructing the feature extraction model comprises: acquiring an initial extraction model constructed by a YOLOv backbone network; Integrating the calculation module carrying the lesion evolution characteristic threshold values of different eye disease types in the initial extraction model to obtain an intermediate extraction model; And training the intermediate extraction model by taking historical lesion data of a plurality of historical testees of different eye diseases as a training set and obtaining the feature extraction model by minimizing a loss function.
9. The method of claim 1, wherein the characteristic threshold for lesion evolution of the different eye disease type comprises a maximum value of a lesion difference of an eye disease, a characteristic threshold for a different eye disease type, and wherein the mode of determining the maximum value of the lesion difference of the eye disease, the characteristic threshold for the different eye disease type comprises: Acquiring multi-mode ophthalmic historical image data of a plurality of historical subjects of different eye diseases at a plurality of historical time points; performing feature extraction on multi-mode ophthalmic historical image data of each historical subject at each historical time point to obtain a high-dimensional feature vector of each historical subject at each historical time point; Aiming at each historical subject, adopting an L2 norm formula to analyze high-dimensional feature vectors of each historical subject among a plurality of continuous time points to obtain a feature difference sequence of each historical subject; counting the maximum value in the characteristic difference sequences of all historical subjects as the maximum value of the lesion difference of the eye diseases; And (3) analyzing the characteristic difference sequences of all the historical testees by adopting a K-means clustering algorithm to obtain typical characteristic thresholds of different eye disease types.
10. A disease evolution visualization system based on ophthalmic multi-modality imaging, the system comprising: the system comprises an acquisition unit, a display unit and a display unit, wherein the acquisition unit is used for acquiring a multi-mode ophthalmic image sequence of a subject with known eye disease types in a history period, wherein the history period is a current time and a period of preset time before the current time; The feature extraction unit is used for inputting the multimode ophthalmic image and the preface multimode ophthalmic image related to the multimode ophthalmic image into a feature extraction model for feature extraction to obtain a mask feature map of the multimode ophthalmic image, wherein the feature extraction model comprises a multi-level backbone feature extraction module and a calculation module carrying lesion evolution feature thresholds of different eye disease types; the integration unit is used for integrating the mask feature images of each multi-mode ophthalmic image in the multi-mode ophthalmic image sequence according to time sequence to obtain a lesion evolution thermodynamic diagram of the eye disease type; and the display unit is used for displaying the eye disease evolution rule of the tested person by utilizing the lesion evolution thermodynamic diagram.

Description

Disease evolution visualization method and system based on ophthalmic multi-mode image Technical Field The application relates to the field of medical images, in particular to a disease evolution visualization method and system based on ophthalmic multi-mode images. Background Ophthalmic diseases are one of the leading causes of vision impairment or even blindness worldwide, and their course is often characterized by progressive, chronic and complex characteristics. In the prior art, diagnosis and treatment of ophthalmic diseases (such as diabetic retinopathy and glaucoma) are usually performed by qualitative or semi-quantitative analysis depending on static images obtained from single points at single time, which lacks deep mining of dynamic association between time-series images, although partial research attempts to analyze disease progress by using time-series images are usually limited to single modes, or only simple image difference or volume change calculation is performed, complementarity of multi-mode information in anatomy and function is not fully fused, and specific pathological evolution modes of different disease types are also less considered, that is, the prior art is difficult to comprehensively and dynamically capture the dynamic evolution rule of the diseases, so that clinicians cannot be accurately assisted in understanding the disease track of individual patients more accurately, and further deviation phenomena of subsequent evaluation, treatment opportunity selection and treatment effect monitoring can occur. Disclosure of Invention Based on the problems, the embodiment of the application provides a disease evolution visualization method and system based on ophthalmic multi-mode images, aiming at precisely revealing the personalized evolution rule of eye diseases of a subject in the development process. The technical scheme of the embodiment of the application is realized as follows: The embodiment of the application provides a disease evolution visualization method based on ophthalmic multi-mode images, which comprises the steps of obtaining a multi-mode ophthalmic image sequence of a subject with known eye disease types in a history period, wherein the history period is a period of time preset at the current moment and before the current moment, inputting each multi-mode ophthalmic image in the multi-mode ophthalmic image sequence, the multi-mode ophthalmic image and an anterior multi-mode ophthalmic image associated with the multi-mode ophthalmic image into a feature extraction model for feature extraction to obtain a mask feature map of the multi-mode ophthalmic image, wherein the feature extraction model comprises a multi-level backbone feature extraction module and a calculation module carrying lesion evolution feature thresholds of different eye disease types, the mask feature map is a feature map comprising dynamic evolution constraint masks corresponding to the multi-mode ophthalmic image, integrating the mask feature map of each multi-mode ophthalmic image in the multi-mode ophthalmic image sequence according to time sequence to obtain a lesion thermodynamic diagram of the eye disease types, and displaying the evolution law evolution thermodynamic diagram of the subject. In some embodiments, the acquiring the multi-modal ophthalmic image sequence of the subject with the known eye disease type in the history period comprises acquiring multi-modal ophthalmic image data corresponding to a plurality of time points of the subject in the history period, performing standardization processing on the multi-modal ophthalmic image data corresponding to each time point to obtain multi-modal ophthalmic images corresponding to each time point, and sequentially integrating the multi-modal ophthalmic images corresponding to each time point to obtain the multi-modal ophthalmic image sequence. In some embodiments, the computing module comprises computing submodules which are deployed at the output end of each hierarchical backbone feature extraction module in the multi-hierarchical backbone feature extraction module and carry lesion evolution feature thresholds of different eye disease types, wherein the computing submodules process feature extraction is carried out on each multi-modal ophthalmic image in the multi-modal ophthalmic image sequence and on a front multi-modal ophthalmic image associated with the multi-modal ophthalmic image in a feature extraction model to obtain a mask feature map of the multi-modal ophthalmic image, the computing submodules comprise feature extraction is carried out on each multi-modal ophthalmic image, the multi-modal ophthalmic image is input into the multi-hierarchical backbone feature extraction module to obtain multi-scale original features of the multi-modal ophthalmic image, the front original features of the multi-modal ophthalmic image are input into the multi-scale original features corresponding to the multi-scale original features, feature processin