CN-114792381-B - Macular region fundus multi-disorder classification method, device and storage medium

CN114792381BCN 114792381 BCN114792381 BCN 114792381BCN-114792381-B

Abstract

The invention discloses a method and a device for classifying multiple diseases of a fundus of a macular region and a storage medium, and belongs to the field of image processing. The method comprises the steps of classifying a macula area image of an eye bottom photo through a multi-disease classification task of transfer learning to obtain a classification result, carrying out picture pretreatment on the macula image to obtain a segmented picture, generating a disease feature description on the segmented picture based on a picture interpretable algorithm to obtain a pixel level clue, carrying out picture enhancement based on a Gan network according to the pixel level clue to obtain a synthesized picture, training according to the segmented picture and the synthesized picture to obtain a macula area multi-disease classification system based on a pixel level, carrying out macula recognition on the eye bottom photo through the macula area multi-disease classification system based on the pixel level to obtain a macula recognition result, and generating a disease feature heat map through the picture interpretable algorithm based on the macula recognition result. The invention can help the ophthalmologist to better understand the principle and basis of the prediction model.

Inventors

WANG HAN
PAN YI

Assignees

珠海中科先进技术研究院有限公司
中国科学院深圳先进技术研究院

Dates

Publication Date: 20260508
Application Date: 20220422

Claims (8)

1. A method for classifying multiple diseases of a fundus in a macular region, comprising the steps of: Detecting the center of a video disc and the fovea of the fundus photo based on an image segmentation algorithm to obtain a predicted value and cross entropy; Obtaining a detection result according to the predicted value, wherein the detection result comprises macula lutea fovea; wherein the image segmentation algorithm comprises but is not limited to FCN, unet or image segmentation algorithm based on the center position relationship of the video disc, and the predicted value is The cross entropy loss function is Wherein The true value is represented by a value that is true, The number of the pictures is shown in the figure, Representing the number of classifications; classifying the macula area image of the fundus oculi photo through a multi-disease classification task of transfer learning to obtain a classification result, wherein the fundus oculi photo comprises an OCT fundus photo and a UWF fundus photo; Performing picture preprocessing on the disease spot picture to obtain a segmented picture; Generating a symptom characteristic description for the segmented picture based on a picture interpretable algorithm to obtain a pixel-level clue; according to the pixel level clues, carrying out picture enhancement based on a Gan network to obtain a synthesized picture; The multi-disease classification system for the macula area based on the pixel level is obtained by training the segmented pictures and the synthesized pictures, and comprises pixel marks for carrying out pixel quantification on the macula segmented pictures, wherein the segmented pictures and the synthesized pictures are marked as the macula segmented pictures, the macula segmented pictures are subjected to migration learning based on a classification trunk model algorithm to obtain pixel level classification recognition clues, wherein all thawed layers of the macula segmented pictures with the type of OCT fundus pictures are learned, the training weight is equal to the initial weight, the thawed classification layers of the macula segmented pictures with the type of OCT fundus pictures are learned, and a picture level classification model is obtained based on the pixel level classification recognition clues, wherein the picture level classification model is as follows: If the symptoms are classified into two categories Activating a function for sigmoid, otherwise A loss function is squared for mse, and a threshold is set, wherein, Represent the first The number of the disease spots is equal to the number of the disease spots, The number of the lesions is represented by the number, The initial value is the weight of the classification model neurons, And Is a positive integer; Performing disease spot identification on the fundus photo through the macula area multi-disease classification system based on the pixel level to obtain a disease spot identification result; based on the disease spot identification result, a disease characteristic heat map is generated through a picture interpretable algorithm.
2. The method of claim 1, wherein the performing image preprocessing on the lesion image to obtain a segmented image includes at least one of the following: Correcting left and right eyes, correcting inverted pictures, normalizing pictures, roughly dividing based on macula fovea and dividing the pictures based on an FOV dividing model; The macula fovea-based rough segmentation includes: Dividing boundary with OCT fundus picture centered on macular fovea and 10mm diameter, and The UWF fundus photo is divided by taking the macula fovea as a center and taking the connecting midpoint of the macula fovea and the center of the test disc as a boundary.
3. The method of classifying ocular fundus multiple diseases in macular region of claim 2, further comprising: and obtaining the roughly segmented background picture, carrying out pixel marking, and training based on a precise extraction FOV algorithm to obtain the FOV segmentation model.
4. A macular region fundus multiple disease classification method according to claim 3 wherein said precision extraction FOV algorithm includes but is not limited to FCN, U net、Yolo、Inception。
5. The method of classifying ocular fundus polynomiasis in macular regions according to claim 1, wherein said image enhancement based on Gan network comprises: Based on a Gan network, according to given pathological features and binary blood vessel segmentation, fundus images with controllable lesion positions and quantity are synthesized, and a synthesized picture is obtained.
6. The method of classifying a macular region fundus multiple disorder according to claim 1, wherein classifying the macular region image of the fundus picture by a multiple disorder classification task of transfer learning comprises: algorithms employing one of the following algorithms or variations of one of the algorithms ：AlexNet、VGG、GoogleNet 、EfficientNet、ResNet 、SENet 、NiN 、Wide ResNet 、ResNext 、DenseNet 、FractalNet 、MobileNets 、NASNet.
7. A macular region fundus multiple disease classification device, comprising: the first module is used for classifying the macula area image of the fundus photo through a multi-disease classification task of transfer learning to obtain a classification result, wherein the fundus photo comprises an OCT fundus photo and a UWF fundus photo; the second module is used for carrying out picture preprocessing on the disease spot picture to obtain a segmented picture; a third module for generating a symptom feature description for the segmented picture based on a picture interpretable algorithm, resulting in a pixel level cue; A fourth module, configured to perform picture enhancement based on a Gan network according to the pixel level cues, to obtain a synthesized picture; a fifth module, configured to perform training according to the divided picture and the synthesized picture, to obtain a multi-disease classification system of the macular area based on a pixel level; a sixth module, configured to perform patch recognition on the fundus photo by using the pixel-level-based macula area multi-disease classification system, to obtain a patch recognition result; A seventh module, configured to generate a disease feature heat map through a picture interpretable algorithm based on the disease spot recognition result; wherein the macular region fundus multi-disorder classification apparatus is for performing the macular region fundus multi-disorder classification method according to any one of claims 1 to 6.
8. A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of any of claims 1 to 6.

Description

Macular region fundus multi-disorder classification method, device and storage medium Technical Field The invention relates to the field of image processing, in particular to a method and a device for classifying multiple diseases of fundus oculi in a macular area and a storage medium. Background In recent years, with the continuous improvement of medical imaging acquisition equipment and the continuous development of subjects such as image processing, pattern recognition, machine learning and the like, the medical image processing and analysis fields with multiple subjects crossing each other have achieved great achievements. These achievements are of great significance in assisting the physician in making a quick and accurate diagnosis. Although the use of machine learning models can assist doctors in performing a differential diagnosis of conditions in the region of the macula at the bottom of the eye, there are still a number of problems. For example, the amount of data used for learning is small, less for rare ocular fundus diseases, and real world data is more difficult to obtain. And the difficulty of precisely marking a large amount of data is high. For marking in the medical field, especially for marking based on pixels, the difficulty is great and the requirement for marking personnel is high. And outputting the classification result only, wherein a doctor cannot learn the judgment basis of the model, so that the classification model cannot be evaluated and the classification result of the model is adopted. Disclosure of Invention The present invention aims to solve at least one of the technical problems existing in the prior art. Therefore, the invention provides a method, a device and a storage medium for classifying ocular fundus multiple diseases in a macular region, which can help an ophthalmologist to better understand the principle and the basis of a prediction model. An embodiment of the invention provides a multi-disease classification method for a macular region fundus, which comprises the steps of classifying a macular region image of an fundus photo through a multi-disease classification task of transfer learning to obtain a classification result, wherein the fundus photo comprises an OCT fundus photo and a UWF fundus photo, the classification result comprises a lesion picture, picture preprocessing is carried out on the lesion picture to obtain a segmented picture, a disease characteristic description is generated on the segmented picture based on a picture interpretable algorithm to obtain a pixel-level clue, picture enhancement is carried out on the basis of a Gan network according to the pixel-level clue to obtain a synthetic picture, training is carried out on the segmented picture and the synthetic picture to obtain a pixel-level-based multi-disease classification system for the macular region, lesion recognition is carried out on the fundus photo through the pixel-level-based multi-disease classification system to obtain a lesion recognition result, and a disease characteristic heat map is generated through the picture interpretable algorithm based on the lesion recognition result. According to some embodiments of the invention, the method further comprises the steps of detecting the center of the optic disc and the fovea of the fundus photo based on an image segmentation algorithm to obtain a predicted value and cross entropy, obtaining a detection result according to the predicted value, wherein the detection result comprises the fovea of the macula, and the image segmentation algorithm comprises but is not limited to FCN, unet or an image segmentation algorithm based on the center position relation of the optic disc. According to some embodiments of the invention, the preprocessing of the image of the lesion to obtain a segmented image at least comprises one of correcting left and right eyes, correcting an inverted image, normalizing the image, roughly segmenting based on the fovea and segmenting the image based on the FOV (field of view) segmentation model, wherein the roughly segmenting based on the fovea comprises segmenting an OCT fundus image with the fovea as a center and a10 mm diameter as a segmentation boundary, and segmenting a UWF fundus image with the fovea as a center and a connecting midpoint of the fovea and the center of the test disc as a boundary. According to some embodiments of the invention, the method further comprises obtaining the background picture after rough segmentation and performing pixel marking, training based on a precise extraction FOV algorithm to obtain the FOV segmentation model, wherein the precise extraction FOV algorithm comprises but is not limited to FCN and U-net, yolo, inception. According to some embodiments of the invention, the image enhancement based on the Gan network to obtain the synthesized image comprises the steps of synthesizing fundus images with controllable lesion positions and quantity based on the Gan network accordi