CN-115457021-B - Dermatological image segmentation method and system based on joint attention convolutional neural network

CN115457021BCN 115457021 BCN115457021 BCN 115457021BCN-115457021-B

Abstract

The invention relates to a dermatological image segmentation method and a dermatological image segmentation system based on a joint attention convolutional neural network, which relate to the technical field of image processing, wherein the method comprises the steps of obtaining dermatological images to be segmented; the method comprises the steps of inputting a skin disease image to be segmented into a skin disease image segmentation model, outputting a skin disease image segmentation result, wherein the skin disease image segmentation model is a trained joint attention convolutional neural network, the joint attention convolutional neural network is a U-Net-based neural network, a migration learning method is adopted in the joint attention convolutional neural network, the trained ResNet-34 is used as an encoder in the U-Net, a spatial attention module is used as jump connection between a symmetrical encoder and a decoder in the U-Net, each feature decoding block in the decoder adopts a pyramid channel attention module, and a multi-scale fusion attention module is adopted at the output end of the decoder. The invention improves the accuracy and reliability of the division of the skin lesions of the skin disease image.

Inventors

ZENG PENG
LI HAIYAN
LI HAIJIANG
WANG ZHENGYU
GUO LEI

Assignees

云南大学

Dates

Publication Date: 20260505
Application Date: 20220930

Claims (9)

1. A dermatological image segmentation method based on a joint attention convolutional neural network, comprising: Acquiring a skin disease image to be segmented; Inputting the skin disease image to be segmented into a skin disease image segmentation model, and outputting a skin disease image segmentation result, wherein the skin disease image segmentation model is a trained joint attention convolutional neural network; The joint attention convolutional neural network is a U-Net-based neural network, a transfer learning method is adopted in the joint attention convolutional neural network, the trained ResNet-34 is used as an encoder in the U-Net, a spatial attention module is used as jump connection between a symmetrical encoder and a decoder in the U-Net, the joint attention convolutional neural network further comprises a pyramid channel attention module and a multi-scale fusion attention module, each feature decoding block in the decoder adopts the pyramid channel attention module, the input of the multi-scale fusion attention module is a feature graph after the output of each feature decoding block is up-sampled, and the output of the multi-scale fusion attention module is the segmentation result of the dermatological image; The encoder in the joint attention convolutional neural network comprises a1 st encoding layer, a N th encoding layer, a1 st decoding layer, a N-1 st decoding layer and a N positive integer, wherein the 1 st encoding layer is connected with the N th encoding layer in sequence; The N-th coding layer is connected with the 1-th decoding layer; the spatial attention module comprises an nth space attention module to an nth-2 spatial attention module, wherein the nth space attention module is an N-2 spatial attention module, a first input end of the nth space attention module is an output characteristic of an nth coding layer, a second input end of the nth space attention module is an output characteristic of an nth-N-1 decoding layer, an output end of the nth space attention module is connected with the nth decoding layer, and the value range of N is 1 to N-2.
2. The method for segmentation of dermatological images based on joint attention convolutional neural network of claim 1, wherein non-local operations are employed between the N-1 coding layer and the 1 decoding layer.
3. The method for segmenting dermatological images based on a joint attention convolutional neural network of claim 1, wherein each spatial attention module uses a formula Obtaining a space attention feature map; Wherein, the Representing the output characteristics of the n-th coding layer, Representing the output characteristics of the N-1 decoding layer, A spatial attention profile is represented, A first attention coefficient is represented and, Representing a second attention coefficient, reLU represents a ReLU activation function, A first attention profile is represented, A second attention profile is shown, Representing a channel connection; Normalization of 1×1 convolution and batch processing with C as output channel number, C fetch and hold The number of channels is the same.
4. The dermatological image segmentation method based on a joint attention convolutional neural network according to claim 1, wherein a decoding layer is used for adding a first feature map and a second feature map, and is further used for decoding an addition result by adopting the feature decoding block, the feature decoding block comprises an input layer, a first convolutional layer, a second convolutional layer and a pyramid channel attention module which are sequentially connected, the feature decoding block further comprises a third convolutional layer, an input end of the third convolutional layer is connected with the input layer, and an output of the third convolutional layer and an output of the pyramid channel attention module are accumulated and then output after passing through a ReLU activation function; The first convolution layer and the second convolution layer each include a convolution operation with a convolution kernel of 3 x 3, and the third convolution layer includes a convolution operation with a convolution kernel of 1 x 1.
5. The method for segmenting dermatological images based on a joint attention convolutional neural network according to claim 4, wherein the pyramid channel attention module comprises a pyramid-type multi-scale feature extraction block, a channel attention weight extraction unit and a multi-scale feature extraction unit; The pyramid-type multi-scale feature extraction block is used for dividing channels of an input feature map into four groups, carrying out convolution operation on each group of channels by adopting convolution kernels with different sizes, and splicing four groups of convolution results in the channel dimension to obtain a first multi-scale feature map; The channel attention weight extraction unit is used for embedding global space information of the first multi-scale feature map into a channel descriptor by adopting global average pooling to obtain an aggregation feature, carrying out one-dimensional convolution with the kernel size of 3 on the aggregation feature to obtain multi-scale channel attention weights, and calibrating the multi-scale channel attention weights by adopting an excitation function Sigmoid; The multi-scale feature extraction unit is used for performing element product operation on the calibrated multi-scale channel attention weight and the first multi-scale feature map to obtain a second multi-scale feature map.
6. The dermatological image segmentation method based on a joint attention convolutional neural network of claim 5, wherein the multi-scale fusion attention module comprises a convolutional unit, an upsampling unit, a channel attention unit and a pixel normalization unit, which are connected in sequence; The convolution unit is used for unifying the channel number of the feature map after up-sampling each second multi-scale feature map; The up-sampling unit is used for unifying the feature images output by the convolution unit into set sizes and performing channel splicing to obtain a channel spliced feature image; The channel attention unit is used for extracting channel attention features from the channel stitching feature map; The pixel normalization unit is used for extracting the spatial attention characteristic of the channel stitching characteristic map based on the channel attention characteristic, performing Softmax activation operation on the spatial attention characteristic to obtain an attention characteristic map, and performing convolution operation on the attention characteristic map to obtain a dermatological image segmentation result.
7. The method for segmenting dermatological images based on a joint attention convolutional neural network according to claim 1, wherein the training process of the joint attention convolutional neural network comprises: Acquiring a data set of skin lesion images; Adjusting each skin lesion image in the dataset to a first set size; Randomly cropping the first-sized skin lesion image to a second-sized image; performing data enhancement on the skin lesion image with the second set size to obtain a data set with enhanced data; And training the joint attention convolutional neural network by adopting the data set with the enhanced data, and taking the trained joint attention convolutional neural network as a dermatological image segmentation model.
8. The method for segmenting dermatological images based on a joint attention convolutional neural network according to claim 1, wherein the joint attention convolutional neural is trained with a mixed loss function, which is a loss function including three levels of map level loss, patch level loss and pixel level loss; the mixing loss function is expressed as: ; Wherein, the For the value of the mixing loss function, In order for the map level penalty to be high, In order for the patch level to be lost, For the pixel-level loss, The map level loss is soft dice coefficient loss, the patch level loss is structural similarity loss, and the pixel level loss is binary cross entropy loss.
9. A joint attention convolutional neural network-based dermatological image segmentation system, comprising: The dermatological image acquisition module to be segmented is used for acquiring dermatological images to be segmented; The dermatological image segmentation module is used for inputting the dermatological image to be segmented into a dermatological image segmentation model and outputting dermatological image segmentation results, wherein the dermatological image segmentation model is a trained joint attention convolutional neural network; The joint attention convolutional neural network is a U-Net-based neural network, a transfer learning method is adopted in the joint attention convolutional neural network, the trained ResNet-34 is used as an encoder in the U-Net, a spatial attention module is used as jump connection between a symmetrical encoder and a decoder in the U-Net, the joint attention convolutional neural network further comprises a pyramid channel attention module and a multi-scale fusion attention module, each feature decoding block in the decoder adopts the pyramid channel attention module, the input of the multi-scale fusion attention module is a feature graph after the output of each feature decoding block is up-sampled, and the output of the multi-scale fusion attention module is the segmentation result of the dermatological image; The encoder in the joint attention convolutional neural network comprises a1 st encoding layer, a N th encoding layer, a1 st decoding layer, a N-1 st decoding layer and a N positive integer, wherein the 1 st encoding layer is connected with the N th encoding layer in sequence; The N-th coding layer is connected with the 1-th decoding layer; the spatial attention module comprises an nth space attention module to an nth-2 spatial attention module, wherein the nth space attention module is an N-2 spatial attention module, a first input end of the nth space attention module is an output characteristic of an nth coding layer, a second input end of the nth space attention module is an output characteristic of an nth-N-1 decoding layer, an output end of the nth space attention module is connected with the nth decoding layer, and the value range of N is 1 to N-2.

Description

Dermatological image segmentation method and system based on joint attention convolutional neural network Technical Field The invention relates to the technical field of image processing, in particular to a skin image segmentation method and system based on a joint attention convolutional neural network. Background Conventional image segmentation methods are typically based on optimal thresholds, region growing, active contour methods, supervision methods, and edge detection algorithms. However, conventional image segmentation methods often require manual intervention or extensive superparameter fine tuning, resulting in poor usability in complex scenarios. In contrast, the deep learning algorithm can automatically extract the features, effectively overcomes the defects of the traditional dermatological segmentation algorithm, and can be rapidly expanded to different task scenes by means of transfer learning. With the development of deep Convolutional Neural Networks (CNNs), U-Net networks have been widely used in the field of medical image segmentation. Inspired by the structure of the U-Net network, the improved network of the U-Net has been widely applied to segmentation of different tissues and organs or lesions in various medical images, including AttU-Net, CE-Net, CA-Net, CPF-Net, MSU-Net and FAT-Net. AttU-Net based on the attention mechanism introduced by the U-Net network, the jump connection of the U-Net network is reconstructed, the extraction capability of the network to the spatial information characteristics is enhanced, and the learning of noise and irrelevant information is restrained. CE-Net combines a dense hole convolution (DAC) module and a residual multi-core pooling (RMP) module with the encoder-decoder structure, capturing more abstract features and preserving more spatial information to improve the performance of medical image segmentation. CA-Net based on U-Net network to introduce spatial attention, channel attention and scale attention mechanisms to raise the interpretability and segmentation performance of the network. CPF-Net combines a Global Pyramid Guidance (GPG) module and a Scale Aware Pyramid Fusion (SAPF) module to fuse global/multi-scale context information. MSU-Net combines a plurality of convolution sequences and convolution kernels of different receiving domains to construct a multi-scale block, extracts more semantic features, and captures detailed multi-scale space features to enable the features to be more diversified. FAT-Net integrating Convolutional Neural Network (CNN) and transducer branches with dual encoders to capture local features, remote dependencies, and global context information simultaneously. The existing skin image segmentation algorithm based on deep learning mainly has the following defects that (1) network feature extraction capability is limited, features are easy to lose, accuracy is low, interpretability is poor, and lesion segmentation effect is poor. (2) The lesion segmentation results of low contrast (insignificant contrast of foreground and background), occlusion by hair or artifacts, large pixel variation inside the lesion, blurred boundaries, large dimensional variation, and irregular shape have limitations. The above disadvantages are caused by (1) insufficient extraction of global context information, insufficient dense prediction of detail space information, neglecting scale feature fusion at different decoding stages, resulting in an inability to accurately segment irregularly shaped lesions. (2) The network structure is too shallow to extract complete local features, resulting in discontinuous edge profiles. (3) Omitting the scale features fused to the different decoding stages does not allow accurate prediction of irregularly shaped lesion areas. (4) Neglecting continuous pooling and blending, problems of inaccurate lesion boundaries due to limited context information and insufficient discrimination feature mapping inevitably occur. Disclosure of Invention The invention aims to provide a dermatological image segmentation method and a dermatological image segmentation system based on a joint attention convolutional neural network, which improve the accuracy and reliability of dermatological image dermatological lesion segmentation. In order to achieve the above object, the present invention provides the following solutions: A dermatological image segmentation method based on a joint attention convolutional neural network, comprising: Acquiring a skin disease image to be segmented; Inputting the skin disease image to be segmented into a skin disease image segmentation model, and outputting a skin disease image segmentation result, wherein the skin disease image segmentation model is a trained joint attention convolutional neural network; the joint attention convolutional neural network is a U-Net-based neural network, a transfer learning method is adopted in the joint attention convolutional neural network, the trained ResNet-34 is u