CN-122023812-A - Multispectral OCT retina segmentation method and multispectral OCT retina segmentation system

CN122023812ACN 122023812 ACN122023812 ACN 122023812ACN-122023812-A

Abstract

The invention belongs to the technical field of image processing. A multispectral OCT retina segmentation method and a multispectral OCT retina segmentation system are provided, a plurality of band images are preprocessed to obtain a normalized multispectral input image set, global feature values of all bands are calculated in a spectrum dimension through a two-dimensional attention coder, a usefulness score is generated, the band images are weighted according to the calculated value, a spectrum attention weighted image set is obtained, the image set is spliced into a multichannel feature map, candidate feature maps are extracted for each layer to be segmented in a retina layering dimension, the candidate feature maps are combined with a layering attention weight map to weight, enhancement feature maps of all layers are obtained, fusion weights are generated based on feature average values of the enhancement feature maps, the core feature maps are weighted and summed to form a core feature map, the core feature map is input into a lightweight segmentation model, and segmentation masks of all layers of the retina are output. The method realizes self-adaptive screening of multispectral information and enhancement of layer-specific characteristics, and improves segmentation accuracy and robustness.

Inventors

SONG WEIYE
QI MIN
LIU ENYU
NIE ZIHAN
YANG HAOHUA
GUAN YANXIN
Yan Bingcan
ZHAO YIXIANG
ZHOU LIBO
CUI YUAN

Assignees

山东大学

Dates

Publication Date: 20260512
Application Date: 20260413

Claims (16)

1. A multi-spectral OCT retinal segmentation method, comprising the steps of: Preprocessing a plurality of band images acquired and converted by the multispectral OCT equipment to obtain a normalized multispectral input image set; The multispectral input image set is input into a two-dimensional attention encoder, global characteristic values are calculated for all wave band images through a spectrum dimension attention sub-module, usefulness scores of all wave bands are generated based on the global characteristic values, and then the wave band images are weighted according to the usefulness scores to obtain a spectrum attention weighted image set; Splicing the spectrum attention weighted image sets along the channel dimension to obtain a multi-channel feature map, respectively extracting candidate feature maps from each retina layer to be segmented through a retina layering attention sub-module based on the multi-channel feature map, generating a corresponding layering attention weighted map from each candidate feature map, and weighting the corresponding candidate feature maps through the layering attention weighted map to obtain an enhanced feature map of each retina layer; calculating the characteristic average value of each layer based on the enhancement feature map of each retina layer, generating fusion weights of each layer according to the characteristic average value, and carrying out weighted summation on the enhancement feature map of each retina layer by utilizing the fusion weights to obtain a core feature map; And inputting the core feature map into a lightweight segmentation model, and outputting segmentation masks of each retina layer.
2. The multi-spectral OCT retinal segmentation method according to claim 1, wherein, Preprocessing a plurality of band images acquired and converted by a multispectral OCT device to obtain a normalized multispectral input image set, wherein the method comprises the following steps: Loading a plurality of band images generated by format conversion of multispectral OCT original data; uniformly adjusting the images of each wave band to the same size; and carrying out pixel value normalization on each band image independently, and mapping the pixel value to a [0,1] interval to obtain the normalized multispectral input image set.
3. The multi-spectral OCT retinal segmentation method according to claim 1, wherein, Inputting the multispectral input image set into a two-dimensional attention encoder, respectively calculating global characteristic values for each band image through a spectrum dimension attention sub-module, and generating usefulness scores of each band based on the global characteristic values, wherein the method comprises the following steps: Flattening each wave band image into a one-dimensional vector, and extracting the global characteristic value of the wave band through 1X1 convolution operation; and applying a Softmax function to the global eigenvalues of all the wave bands to generate a usefulness score of each wave band, wherein the sum of the usefulness scores of each wave band is 1.
4. The multi-spectral OCT retinal segmentation method according to claim 1, wherein, Weighting each band image according to the usefulness score to obtain a spectrum attention weighted image set, comprising: Expanding the usefulness score of each band to a weight map consistent with the band image size; Multiplying the weight map with the corresponding wave band image pixel by pixel to obtain a spectrum attention weighted image of the wave band; and executing the operation on all the wave bands to obtain the spectrum attention weighted image set.
5. The multi-spectral OCT retinal segmentation method according to claim 1, wherein, Extracting candidate feature graphs from each retina layer to be segmented respectively through a retina layering attention sub-module based on the multi-channel feature graphs, wherein the candidate feature graphs comprise: and carrying out convolution operation on the multi-channel feature map by using 3X 3 convolution aiming at each retina layer to be segmented to obtain a candidate feature map corresponding to the retina layer.
6. The multi-spectral OCT retinal segmentation method according to claim 1, wherein, Generating a corresponding hierarchical attention weighting map for each candidate feature map, comprising: And applying a Sigmoid activation function to each candidate feature map, and compressing the pixel value to a [0,1] interval to obtain a layered attention weight map corresponding to the candidate feature map.
7. The multi-spectral OCT retinal segmentation method according to claim 1, wherein, Weighting the corresponding candidate feature images by using the layered attention weight image to obtain an enhanced feature image of each retina layer, wherein the method comprises the following steps: And multiplying each layered attention weight graph with the corresponding candidate feature graph pixel by pixel to obtain an enhanced feature graph of the retina layer.
8. The multi-spectral OCT retinal segmentation method according to claim 1, wherein, Calculating the characteristic mean value of each layer based on the enhanced characteristic map of each retina layer, and generating the fusion weight of each layer according to the characteristic mean value, wherein the method comprises the following steps: calculating the average value of all pixels for the enhancement feature map of each retina layer to obtain the feature average value of the retina layer; and applying a Softmax function to the characteristic average value of all retina layers to generate fusion weights of all layers, wherein the sum of the fusion weights of all layers is 1.
9. The multi-spectral OCT retinal segmentation method according to claim 1, wherein, Inputting the core feature map into a lightweight segmentation model, and outputting segmentation masks of each retina layer, wherein the segmentation masks comprise: In the training stage, taking the weighted sum of the Dice loss and the cross entropy loss as a loss function, and adopting an Adam optimizer to perform joint training on the two-dimensional attention encoder and the lightweight segmentation model; In the reasoning stage, a new multispectral input image set is sequentially processed by the two-dimensional attention encoder and the lightweight segmentation model, and corresponding retina each layer segmentation masks are output.
10. The multi-spectral OCT retinal segmentation method according to any one of claims 1-9, wherein, The lightweight partition model is of a lightweight U-Net structure.
11. A multi-spectral OCT retinal segmentation system, comprising: The preprocessing unit is configured to preprocess the multiple wave band images acquired and converted by the multispectral OCT equipment to obtain a normalized multispectral input image set; The spectrum weighting unit is configured to input the multi-spectrum input image set into a two-dimensional attention encoder, respectively calculate global characteristic values for all wave band images through a spectrum dimension attention sub-module, generate usefulness scores of all wave bands based on the global characteristic values, and weight all wave band images according to the usefulness scores to obtain a spectrum attention weighted image set; The layering enhancement unit is configured to splice the spectrum attention weighted image sets along a channel dimension to obtain a multi-channel feature map, respectively extract candidate feature maps for each retina layer to be segmented through a retina layering attention sub-module based on the multi-channel feature map, generate a corresponding layering attention weighted map for each candidate feature map, and weight the corresponding candidate feature map by using the layering attention weighted map to obtain an enhancement feature map of each retina layer; The layering enhancement unit is configured to calculate the characteristic mean value of each layer based on the enhancement characteristic map of each retina layer, generate fusion weights of each layer according to the characteristic mean value, and carry out weighted summation on the enhancement characteristic map of each retina layer by utilizing the fusion weights to obtain a core characteristic map; And a segmentation output unit configured to input the core feature map into a lightweight segmentation model and output segmentation masks of each layer of retina.
12. The multi-spectral OCT retinal segmentation system according to claim 11, In the spectrum weighting unit, the multispectral input image set is input into a two-dimensional attention encoder, global feature values are calculated for each band of images through a spectrum dimension attention sub-module, and a usefulness score of each band is generated based on the global feature values, and the method comprises the following steps: Flattening each wave band image into a one-dimensional vector, and extracting the global characteristic value of the wave band through 1X1 convolution operation; and applying a Softmax function to the global eigenvalues of all the wave bands to generate a usefulness score of each wave band, wherein the sum of the usefulness scores of each wave band is 1.
13. The multi-spectral OCT retinal segmentation system according to claim 11, And the spectrum weighting unit is used for weighting the wave band images according to the usefulness score to obtain a spectrum attention weighted image set, and the spectrum attention weighted image set comprises the following components: Expanding the usefulness score of each band to a weight map consistent with the band image size; Multiplying the weight map with the corresponding wave band image pixel by pixel to obtain a spectrum attention weighted image of the wave band; and executing the operation on all the wave bands to obtain the spectrum attention weighted image set.
14. A computer device comprises a processor and a computer-readable storage medium; a processor adapted to execute a computer program; a computer readable storage medium having stored therein a computer program which, when executed by the processor, implements the multi-spectral OCT retinal segmentation method according to any one of claims 1 to 10.
15. A computer readable storage medium, characterized in that it stores a computer program adapted to be loaded by a processor and to perform the multi-spectral OCT retinal segmentation method according to any one of claims 1 to 10.
16. A computer program product, characterized in that the computer program product comprises a computer program which, when executed by a processor, implements the multi-spectral OCT retinal segmentation method according to any one of claims 1 to 10.

Description

Multispectral OCT retina segmentation method and multispectral OCT retina segmentation system Technical Field The invention relates to the technical field of image processing, in particular to a multispectral OCT retina segmentation method and system. Background The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art. The retina is used as the core tissue of human visual perception, the inside of the retina is composed of a plurality of layers of fine structures such as a nerve fiber layer, a ganglion cell layer, an inner plexiform layer, an inner core layer, an outer plexiform layer, an outer core layer, a retinal pigment epithelium layer, a choroid layer and the like, and the morphological integrity and the functional normality of each layer directly determine the visual quality. In clinical diagnosis and treatment, common blindness-causing diseases such as glaucoma, diabetic retinopathy and age-related macular degeneration are accompanied by characteristic retinal layer structure abnormalities. For example, glaucoma causes progressive atrophy and thinning of nerve fiber layers, diabetic retinopathy is liable to cause interlayer edema in macular areas and detachment of retinal pigment epithelium layers, and age-related macular degeneration is often accompanied by degeneration damage of outer nuclear layers and retinal pigment epithelium layers. Therefore, the method accurately and efficiently segments the boundaries of each retina layer, is not only a core basis for early screening of diseases, but also a key support for disease progress monitoring, treatment scheme formulation and curative effect evaluation, and has important clinical significance for reducing blindness rate. The optical coherence tomography (optical coherence tomography, OCT for short) technology has become a "gold standard" for clinical retinal structure detection by virtue of the advantages of non-invasiveness, micron-level high resolution and real-time imaging, and can more clearly present the fine lamellar structure of retina compared with the traditional techniques of fundus photography, fluorescein fundus angiography, and the like. The multispectral OCT technology is used as an upgrading form of the traditional OCT, and can capture the special spectral response characteristics of different layers of tissues of the retina to different wavelengths by synchronously collecting a plurality of groups of spectral signals from visible light to near infrared band. For example, the nerve fiber layer is more sensitive to red light wave band response, the contrast ratio of the retinal pigment epithelial layer to green light wave band is higher, and the multidimensional information supplement provides more sufficient data support for distinguishing adjacent layer structures with similar morphology and identifying early tiny lesions. In practical application, format conversion and preprocessing are required to be performed on original RAW data of the multispectral OCT through specialized tools such as MATLAB, and finally, multispectral images are generated. The images completely retain the layer structure information of each spectrum band, have the advantages of universal format, easy storage and easy loading, and become a core data base for training an automatic retina layer segmentation model. Along with the improvement of clinical requirements for screening large-scale retinal diseases and the rapid popularization of portable multispectral OCT equipment, development of a high-precision segmentation technology which is adaptive to multispectral image characteristics and can fully mine the advantages of multispectral data is needed to meet the dual requirements of clinical accurate diagnosis and engineering lightweight deployment. Although multispectral OCT provides richer layer structure characterization information for retina layer segmentation and multispectral images provide a convenient data carrier for model training, the existing retina layer segmentation technology based on multispectral OCT images still has a plurality of core bottlenecks, and the segmentation precision and the practical application value are severely restricted. Firstly, the traditional encoder adopts a globally unified multispectral signal processing strategy, and is not subjected to targeted optimization aiming at the band difference of multispectral images. The different spectral bands have significant differences in the value of the characterization of the layers of the retina. For example, the red light wave band is more suitable for presenting the boundaries of nerve fiber layers and ganglion cell layers, the green light wave band has higher differentiation degree on the retinal pigment epithelium layer and the choroid layer, and the traditional encoder regards images of all wave bands as being equally important, and unified feature extraction logic is adopted, so that t