CN-121982322-A - Continuity constraint and difficulty guided passive domain adaptive three-dimensional medical image segmentation method

CN121982322ACN 121982322 ACN121982322 ACN 121982322ACN-121982322-A

Abstract

The invention provides a passive domain adaptive three-dimensional medical image segmentation method guided by continuity constraint and difficulty. The method comprises the steps of performing full supervision training on a segmentation model by utilizing source domain labeling data in a source domain pre-training stage, performing style migration on a target domain image by adopting a concept of combining coarse generation and fine generation in a pseudo source domain image generation stage, and performing Fourier transformation on the pseudo source domain image to remove artifacts and noise existing in the coarse generated image. The target domain adaptation stage utilizes a pre-training segmentation model, a pseudo source domain image and a target domain image to fuse continuity constraint among slices and a difficult sample mining mechanism to carry out an adaptation process from a source domain to a target domain. The invention effectively fuses the space context constraint and the difficult sample mining mechanism of the three-dimensional medical image under the condition of no need of accessing source domain data.

Inventors

LIN JIAWEN
LUO HAOLIN
WENG QIAN
Rao Zekai

Assignees

福州大学

Dates

Publication Date: 20260505
Application Date: 20260131

Claims (10)

1. A passive domain adaptive three-dimensional medical image segmentation method guided by continuity constraint and difficulty is characterized by comprising the following steps: The method comprises the steps of S1, pre-training a source domain segmentation neural network, constructing a segmentation neural network architecture and training, wherein the segmentation neural network architecture comprises an encoder and a segmentation decoder, the encoder is responsible for extracting general and high-level features from an input source domain 3D medical image, and the segmentation decoder is used for segmenting the source domain 3D medical image; Step S2, passive domain style migration, which comprises freezing the segmentation neural network in the step S1 as a source segmentation model, training a domain style converter by using the source segmentation model and a target domain image, generating an image through Fourier frequency domain transformation optimization, obtaining a pseudo source domain image with target domain semantics of the source domain style, and segmenting the pseudo source domain image by using the source model to obtain a pseudo tag; And step S3, domain adaptation fine tuning, namely fine tuning the split neural network model in the step S1 by using the split neural network in the step S1, the pseudo source domain image, the target domain image and the pseudo labels in the step S2 through pseudo label weighting, inter-slice continuity constraint and a difficult sample mining method, so that the split neural network model is adapted to target domain data distribution.
2. The method for segmenting the passive domain-adaptive three-dimensional medical image guided by continuity constraint and difficulty according to claim 1, wherein a semantic segmentation network DeepLab based on a cavity convolution and spatial pyramid pooling module is adopted as a segmentation backbone in the segmentation neural network architecture in the step S1, and the source domain image in the step S1, the pseudo source domain image in the step S2 and the target domain image are connected on a channel by a previous slice and a subsequent slice to form an image with a 2.5D structure.
3. The method for continuously constrained and difficulty guided passive domain adaptive three-dimensional medical image segmentation according to claim 1, wherein the pseudo source domain image generation in step S2 is performed in two stages, comprising the following: The style converter in the step S2 is optimized by minimizing style loss and content loss, wherein the style loss is calculated based on statistical information stored in a shallow batch normalization layer in a source segmentation model, and the content loss is calculated based on deep features of the target domain image in the source segmentation model; and (3) fine generation, namely performing Fourier transformation on the coarse generation image and the target domain image, separating high frequency from low frequency, exchanging low frequency amplitude components of the coarse generation image, and then performing inverse Fourier transformation so as to remove artifacts existing in the coarse generation image, thereby obtaining a high-quality fine generation image serving as the false source domain image in the step S2.
4. The method for segmenting a continuity-constrained and difficulty-guided passive domain adaptive three-dimensional medical image according to claim 1, wherein the method for weighting the pseudo tag in step S3 calculates uncertainty by using a monte carlo Dropout method, opens the Dropout layer in the model training process and performs forward propagation for a plurality of times, and assigns weights based on the mean value of variances among propagation results for the uncertainty, so that the pseudo tag with lower uncertainty has higher weight and the pseudo tag with higher uncertainty has lower weight.
5. The method for segmenting a continuity-constrained and difficulty-guided passive domain adaptive three-dimensional medical image according to claim 4, wherein the inter-slice continuity method of step S3 is constrained by using consistency of segmentation results of front and rear slices, wherein the model segments the front and rear slices respectively first, then calculates distances of features of adjacent slices as a weight of consistency of the segmentation results of the adjacent slices, and calculates consistency loss between a center slice and the front and rear slices by using the weight, thereby constraining inter-slice continuity of the image on the segmentation results.
6. The passive domain adaptive three-dimensional medical image segmentation method based on continuity constraint and difficulty guidance according to claim 5, wherein the difficult sample mining method in step S3 is based on a comparison learning structure, according to an uncertainty mask and a segmentation result, low-confidence refractory positive class pixels and high-confidence erroneous negative class pixels are selected as candidate anchor points under a low uncertainty condition, high-confidence foreground pixels are used as positive samples, high-confidence background pixels are used as negative samples, positive and negative sample centers are respectively calculated as class prototypes, anchor points are further screened according to the distances between the candidate anchor points and the positive and negative sample centers, and a difficult sample mining mechanism is realized in a feature space through construction comparison learning loss.
7. The continuity constraint and difficulty guided passive domain adaptive three-dimensional medical image segmentation method as set forth in claim 6, wherein the style loss Is defined by equation (1), including the following: ;(1); Wherein, the For the number of shallow blocks selected, And Is the characteristic mean and standard deviation of the noise image at the nth layer, And Is the running mean and running standard deviation of the n-th layer BN storage of the source model, and corresponds to the content loss Defined by equation (2), including the following: ;(2); Wherein, the For the number of deep-layer blocks selected, And Representing the characteristics of the generated image and the target image.
8. The method for continuously constrained and difficulty guided passive domain adaptive three-dimensional medical image segmentation according to claim 7, wherein the pseudo-label weighting method based on Monte Carlo Dropout computation in step S3 is based on variance computation, and in the reasoning stage, the model Dropout layer is kept in an activated state, a plurality of different prediction results are generated through T forward propagation, variance of each pixel prediction result is computed, and average value of all pixel variances is taken as image-level uncertainty The specific calculations are defined by equation (3), including the following: ;(3); Wherein the method comprises the steps of For the partition map of the t-th prediction, Then the average segmentation map after T predictions, And Representing the width and height of the segmentation map respectively, and the final weight calculation is shown in formula (4), and comprises the following contents: ;(4); Wherein the method comprises the steps of For the adjustment coefficient, the value is usually set to 1.
9. The method for segmenting the passive domain adaptive three-dimensional medical image guided by continuity constraint and difficulty according to claim 8, wherein the continuity constraint among the slices firstly calculates pixel level differences among features as continuity weights among the slices, and weights of a preceding slice and a subsequent slice are respectively defined by formulas (5) and (6), and the method comprises the following steps: ; (5); ;(6); Wherein the method comprises the steps of 、 And For the feature vectors of the current slice, the previous slice and the following slice, As the MSE of the pixel level is possibly smaller, the MSE is multiplied by the scaling factor, so that the deformation degree of the features between slices can be better measured; Then, the weighted distance between the adjacent slices is calculated, and the specific calculation is shown in a formula (7), and comprises the following steps: ;(7); Wherein, therein 、 And The method is characterized in that the method is a segmentation diagram of a current slice, a previous slice and a next slice, wherein at the position with larger characteristic deformation, the continuity constraint among the slices is weakened, and at the position with smaller characteristic deformation, the continuity constraint among the slices is strengthened.
10. The passive domain adaptive three-dimensional medical image segmentation method guided by continuity constraint and difficulty according to claim 9, wherein the step S3 further comprises calculating a class prototype based on high confidence foreground and background pixels as a selection basis of candidate anchor points, firstly, selecting positive, negative and anchor point samples according to confidence threshold values, wherein the positive, negative and anchor point samples are respectively shown by formulas (8), (9) and (10), and the method comprises the following steps: ;(8); ;(9); ;(10); Wherein the method comprises the steps of 、、 Respectively representing selected positive samples, negative samples and anchor point pixel masks; representing a high confidence foreground threshold value, A high confidence background threshold is indicated, Then a low uncertainty threshold is indicated; And Then the segmentation map and uncertainty map are represented respectively, and positive and negative class prototypes are calculated from the high confidence positive and negative sample masks, defined by equations (11), (12), respectively: ;(11); ;(12); Wherein, the A prototype of the positive class is represented, Representing a prototype of the negative class, Selecting anchor point pixels according to positive and negative prototype, and specifically calculating as shown in formulas (13), (14) and (15): ;(13); ;(14); ;(15); Wherein, the Representing the degree of cosine similarity, Calculating similarity between anchor points and positive and negative samples, wherein the similarity is defined by formulas (16) and (17), and the final contrast learning loss is defined by formula (18): ; ; ; ; ; ; Wherein, the Is the feature vector of the anchor pixel, Is the anchor point and the first The degree of similarity of the positive samples, Then it is the anchor point and the first The degree of similarity of the individual negative examples, And (3) with The number of positive and negative samples is represented, respectively.

Description

Continuity constraint and difficulty guided passive domain adaptive three-dimensional medical image segmentation method Technical Field The invention provides a continuity constraint and difficulty guided passive domain adaptive three-dimensional medical image segmentation method, and relates to the field of passive domain adaptive three-dimensional medical image segmentation. Background The semantic segmentation technology based on deep learning has great potential in 3D medical image analysis, the semantic segmentation model based on deep learning generally assumes that training data and test data follow the same distribution, however in practical clinical application, when a high-performance segmentation model obtained by training on specific source domain data (such as specific model CT machine data of an A hospital) is directly applied to a target domain with domain offset (such as different model CT machine data of a B hospital), the segmentation precision is generally significantly reduced. In addition, the medical image segmentation model driven by data is strongly dependent on large-scale and high-quality labeling data, huge time and manpower resource cost are consumed for acquiring the labeling data, and the phenomena severely restrict generalization and application of the medical image segmentation model in real and multi-center clinical scenes. To alleviate the above problems, unsupervised domain adaptation (Unsupervised domain adaptation, UDA) of medical image segmentation techniques has evolved. The UDA medical image segmentation method carries out domain adaptation based on source domain labeling data and target domain non-labeling data, and most current UDA medical image segmentation methods depend on joint training of a source domain and a target domain or require access to source domain data for distribution alignment. However, in many real situations, due to reasons of privacy protection of patients, data security regulations or business confidentiality, the source domain data cannot be shared or accessed, and meanwhile, the medical image has the characteristics of huge volume and high storage cost, and the simultaneous storage of the two domain data in a single server environment consumes high storage resources, so that the intolerable training cost is introduced. The research on the passive domain adaptation (Source-free domain adaptation, SFDA) medical image segmentation technology is accelerated, and the SFDA medical image segmentation method only utilizes a model pre-trained on a Source domain and unlabeled target domain data to complete a domain adaptation process, so that the dependence of the domain adaptation model on the Source domain data is eliminated, and the method has incomparable advantages in privacy protection, deployment convenience and clinical feasibility. In recent years, researchers have obtained extensive harvest in application of a two-dimensional SFDA medical image segmentation method, and with rapid development of imaging equipment and calculation efficiency and increasing clinical requirements and application prospects, the research center of gravity in the current medical image segmentation field is accelerating to develop three-dimensional medical images. Under this trend, three-dimensional SFDA medical image segmentation gradually becomes a research hotspot, and three-dimensional medical images (such as MR and CT) have a huge computational load, more difficult data labeling and more complex spatial context modeling than two-dimensional medical images, so a large number of researchers aim at solving the unique challenges brought from two dimensions to three dimensions and mining the huge potential of the three-dimensional SFDA medical image segmentation in practical clinical applications such as disease progression assessment, radiotherapy target region delineation, operation planning and the like. However, the existing passive domain adaptation method cannot go deep into insight and efficiently utilize multi-level information contained in the three-dimensional medical image data itself when processing the three-dimensional medical image. In order to simply apply the two-dimensional SFDA medical image segmentation method to the mainstream three-dimensional SFDA medical image segmentation method, continuous three-dimensional medical images are simply deconstructed into isolated two-dimensional slices to be processed, and inherent spatial anatomical continuity between adjacent slices is completely split. Although some spatial information can be captured by a method based in part on three-dimensional convolution, the huge computational complexity and parameter amount make adaptation in a resource-constrained environment in actual clinical practice difficult to bear. In addition, the adjacent slices of the three-dimensional medical image at the head end and the tail end of the volume generally have larger organ deformation, the pixels at the positions with larger