CN-122024977-A - Glaucoma lesion feature screening system and method based on artificial intelligence, electronic equipment and storage medium

CN122024977ACN 122024977 ACN122024977 ACN 122024977ACN-122024977-A

Abstract

The invention discloses an artificial intelligence-based glaucoma lesion feature screening system, an artificial intelligence-based glaucoma lesion feature screening method, electronic equipment and a storage medium, wherein the lesion feature screening system comprises a multi-mode data acquisition and preprocessing module, a multi-mode feature extraction and fusion module and a screening and grading decision module; the multi-modal data acquisition and preprocessing module is used for acquiring clinical data and carrying out standardized processing on the acquired clinical data to generate unified specification input data, the multi-modal feature extraction and fusion module is used for respectively extracting high-dimensional features from the standardized multi-modal images by utilizing a pre-trained convolution neural network with different structures and generating information enhanced joint feature representation through a fusion network based on an attention mechanism, and the screening and grading decision module is used for screening and grading glaucoma lesion features through a classifier based on the joint feature representation. The invention can remarkably improve the sensitivity and the grading accuracy of the screening of the early lesion characteristics of glaucoma.

Inventors

Lu Heyang
ZHANG WEIXIANG
ZHANG HANG
FANG YONG

Assignees

上海镜影医疗科技有限公司

Dates

Publication Date: 20260512
Application Date: 20251230

Claims (10)

1. An artificial intelligence based glaucoma lesion characterization screening system, the lesion characterization screening system comprising: the multi-mode data acquisition and preprocessing module is used for acquiring clinical data and carrying out standardized processing on the acquired clinical data to generate unified input data, wherein the clinical data comprises fundus color images, optical coherence tomography images and visual field inspection images; The multi-modal feature extraction and fusion module is connected with the multi-modal data acquisition and preprocessing module and is used for respectively extracting high-dimensional features from the standardized multi-modal images by utilizing the pre-trained convolution neural networks with different structures, and generating information-enhanced joint feature representation through a fusion network based on an attention mechanism; And the screening and grading decision module is connected with the multi-mode feature extraction and fusion module and is used for screening and grading glaucoma lesion features through a classifier based on the joint feature representation.
2. The artificial intelligence based glaucoma lesion characterization screening system according to claim 1, wherein: the multi-modal feature extraction and fusion module comprises: the first convolutional neural network submodule is used for extracting morphological characteristics of fundus color illumination; The second convolutional neural network submodule is used for extracting the structural quantization characteristics of the OCT image; A third convolutional neural network sub-module for extracting visual field inspection functional features; the feature fusion unit is used for carrying out dimension alignment and cross-modal attention fusion on the features output by the submodules; The feature fusion unit is specifically configured to dimension-lift the feature vector output by the third convolutional neural network submodule through the full-connection layer, make the feature vector consistent with the dimensions of the other modal feature vectors, splice a plurality of feature vectors with aligned dimensions, input the spliced feature sequence to a transducer encoder, calculate the inter-modal correlation weight by using a self-attention mechanism of the transducer encoder, and output the joint feature representation.
3. The artificial intelligence based glaucoma lesion characterization screening system according to claim 1, wherein: the screening and grading decision module further comprises a fine classification task for grading the severity of glaucoma lesions feature; The screening and grading decision module adopts a multi-task learning framework, and the total loss function is the weighted sum of the screening task loss function and the grading task loss function, namely L total ＝α·L screen +β·L grade , wherein alpha and beta are adjustable super parameters; The optical coherence tomography image is a retinal nerve fiber layer thickness map and a macular area ganglion cell complex thickness map.
4. The artificial intelligence based glaucoma lesion characterization screening system according to claim 3, wherein: The severity classification is automatically judged by a deep learning model based on the joint characteristic representation F fused , multi-modal information of comprehensive structural injury, functional injury and morphological change is judged, and comprehensive severity assessment is output and divided into early stage, middle stage, late stage and final stage; The lesion feature screening system further comprises a data output module, wherein the data output module is used for outputting a structured diagnosis report, and the report at least comprises a screening result, a grading result and an abnormal region thermodynamic diagram for visualizing diagnosis basis; The data output module generates a diagnostic thermodynamic diagram identifying an abnormal region on the input fundus color illumination and OCT image by utilizing a gradient weighting type activation mapping technology.
5. An artificial intelligence-based glaucoma lesion feature screening method, which is characterized by comprising the following steps: Acquiring and preprocessing multi-mode data, namely acquiring and standardizing clinical data such as acquired fundus color images, optical coherence tomography images, visual field inspection images and the like to generate input data with uniform specification; The multi-modal feature extraction and fusion step is that a pre-trained convolution neural network with different structures is utilized to extract high-dimensional features from the standardized multi-modal images respectively, and an information enhanced joint feature representation is generated through a fusion network based on an attention mechanism; And a screening and grading decision step of screening and grading glaucoma lesion features by a classifier based on the joint feature representation.
6. The artificial intelligence based glaucoma lesion characterization screening method according to claim 5, wherein: the multi-modal feature extraction and fusion step includes: The first convolutional neural network sub-module extracts eyeground color illumination morphological characteristics; the second convolutional neural network sub-module extracts OCT image structure quantization characteristics; The third convolutional neural network sub-module extracts visual field inspection functional features; in the feature fusion step, feature vectors output by the third convolutional neural network submodule are subjected to dimension lifting through a full-connection layer to enable the feature vectors to be consistent with the dimensions of the rest mode feature vectors, a plurality of feature vectors with the aligned dimensions are spliced, the spliced feature sequences are input to a transducer encoder, the self-attention mechanism is utilized to calculate the inter-mode correlation weight, and the joint feature representation is output.
7. The artificial intelligence based glaucoma lesion characterization screening method according to claim 5, wherein: the screening and grading decision step further comprises a fine classification task for grading the severity of glaucoma lesions; In the screening and grading decision step, a multi-task learning framework is adopted, wherein the total loss function is the weighted sum of the screening task loss function and the grading task loss function, namely L total ＝α·L screen +β·L grade , and alpha and beta are adjustable super parameters; The optical coherence tomography image is preferably a retinal nerve fiber layer thickness map and a macular area ganglion cell complex thickness map.
8. The artificial intelligence based glaucoma lesion characterization screening method according to claim 7, wherein: The severity classification is automatically judged by the deep learning model based on the joint characteristic representation F fused , multi-modal information of comprehensive structural injury, functional injury and morphological change is judged, comprehensive severity assessment is output and divided into early stage, middle stage, late stage and final stage; Outputting a structured diagnosis report, wherein the report at least comprises a screening result, a grading result and an abnormal region thermodynamic diagram used for visual diagnosis basis; in the data output step, a gradient weighted activation mapping technique is utilized to generate a diagnostic thermodynamic diagram identifying an abnormal region on the input fundus color photograph and OCT image.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method of any of claims 5 to 8 when the computer program is executed by the processor.
10. A storage medium having stored thereon computer program instructions which, when executed by a processor, implement the steps of the method of any of claims 5 to 8.

Description

Glaucoma lesion feature screening system and method based on artificial intelligence, electronic equipment and storage medium Technical Field The invention belongs to the technical field of medical image processing and artificial intelligence, relates to a lesion feature screening system, and in particular relates to an artificial intelligence-based glaucoma lesion feature screening system, an artificial intelligence-based glaucoma feature screening method, electronic equipment and a storage medium. Background Glaucoma is a global first irreversible blinding eye disease characterized by progressive damage to the optic nerve and visual field defects. Because early glaucoma patients often have no subjective symptoms, when the vision function is obviously impaired, the disease condition enters middle and late stages, and the best treatment time is missed. Therefore, the early accurate screening and the accurate grading of the severity degree have important significance for delaying the disease development and preserving the vision function of the patient. Currently, clinical diagnosis of glaucoma relies on a series of multi-modal examinations, each of which has its advantages and inherent limitations: (1) Fundus color photography for visual assessment of optic disc morphology such as cup-to-disc Ratio (C/D Ratio), disc border notch, hemorrhage, retinal Nerve Fiber Layer (RNFL) defect, etc. However, this method has limited recognition ability for early minor changes, and the diagnosis result is highly dependent on the clinical experience of the doctor, and has strong subjectivity. (2) Optical Coherence Tomography (OCT) can measure the thickness of Retinal Nerve Fiber Layer (RNFL) and Ganglion Cell Complex (GCC) in macular area quantitatively with high resolution, and provides objective quantification basis for structural damage of glaucoma. Nevertheless, the interpretation of data requires expertise and as single modality information, there is a risk of false positives and false negatives. (3) Tonometry, a central risk factor for glaucoma, is an important part of routine screening. However, since the diagnostic specificity is insufficient, the normotensive patient may still suffer from glaucoma (normal tension glaucoma), and thus cannot be regarded as an independent exclusion criterion. (4) Visual field examination, currently regarded as the "gold standard" for judging glaucoma function impairment. However, this examination has a strong subjective dependence and functional defects often lag behind structural damage occurrences, which is detrimental to true early diagnosis. (5) OCT angiography (OCTA) is a noninvasive observation of optic disc and macular region blood flow density (VD), which provides diagnostic basis from a microcirculation perspective, however, correlation analysis between OCTA parameters and structural parameters is complex. The prior art has the following defects: (1) The diagnosis is carried out by relying on a single mode, namely only on a single image mode (such as only looking at fundus color photographs or only analyzing OCT), and the diagnosis is carried out by lacking an effective comprehensive research and judgment tool, is incomplete in information and is easy to cause missed diagnosis or misdiagnosis. For example, early glaucoma may appear normal on fundus illumination, but RNFL thinning has occurred on OCT, which can lead to missed diagnosis of early or atypical cases. (2) Highly dependent on expert experience, the accuracy of diagnosis is greatly affected by the clinical experience and subjective judgment of doctors, and diagnosis differences can exist among different doctors. (3) The diagnosis efficiency is low, manual quantitative measurement and comparison of massive image data are time-consuming and labor-consuming, and are difficult to popularize in primary hospitals and large-scale screening. (4) The lack of accurate grading, some existing computer-aided diagnosis systems focus on "screening" (i.e., two categories: yes or no), lack the ability to automate, refine and grade the severity of glaucoma (pre-, early-, mid-, late-, and end-stages), and grade is critical to the choice of treatment regimen. In summary, the existing glaucoma diagnosis mode faces the core challenges of isolating information of each examination mode, subjective integration of diagnosis process depending on experience of doctors, insufficient sensitivity of early screening, and lack of efficient and objective quantitative classification means. In view of this, there is an urgent need to devise a new way of assisting diagnosis of glaucoma lesions in order to overcome at least some of the above-mentioned drawbacks of the existing ways. Disclosure of Invention The invention provides an artificial intelligence-based glaucoma lesion feature screening system, an artificial intelligence-based glaucoma lesion feature screening method, electronic equipment and a storage medium, which can remarkably improve sensit