CN-122023477-A - Medical image processing method, system and medium

CN122023477ACN 122023477 ACN122023477 ACN 122023477ACN-122023477-A

Abstract

The application provides a medical image processing method, a system and a medium, which comprise the steps of receiving a three-dimensional MRI image and a US image of a prostate as input, respectively extracting feature images of three levels of the MRI image and the US image to generate a feature pyramid, taking the feature of the MRI image as a Query, taking the feature of the US image as a Key and a Value, calculating a similarity matrix of the Query and the Key, adopting Softmax to normalize and generate an attention weight image, weighting and summing the Value by using the attention weight, fusing with the feature of the MRI image to generate a multi-mode fusion feature, respectively generating an MRI prostate segmentation probability image, an US prostate segmentation probability image and an initial displacement field based on a multi-mode fusion feature golden sub-tower, adopting differential and homoembryo integration processing on the initial displacement field, and generating a registered MRI image.

Inventors

WANG HAIFENG
YAO YUHANG
LI QIANG
CHEN KUI
ZHANG ZHEMING
FAN PEIHUA
ZHANG BO

Assignees

上海市东方医院（同济大学附属东方医院）
无锡艾米特智能医疗科技有限公司

Dates

Publication Date: 20260512
Application Date: 20251227

Claims (10)

1. A method for processing medical images, comprising: Receiving a three-dimensional MRI image and an US image of the prostate as inputs, generating a modal existence indication mask for each input, gradually downsampling, respectively extracting three-level feature images of the MRI image and the US image, and generating a feature pyramid; On each feature level, taking an MRI image feature as a Query, taking a US image feature as a Key and a Value, calculating a similarity matrix of the Query and the Key, generating an attention weight graph by adopting Softmax normalization, carrying out weighted summation on the Value by using the attention weight, and fusing with the MRI image feature to generate a multi-mode fusion feature; gradually up-sampling based on a multi-mode fusion characteristic golden tower to respectively generate an MRI prostate segmentation probability map, an US prostate segmentation probability map and an initial displacement field; And performing differential stratospheric integration processing on the initial displacement field, and generating a registered MRI image.
2. The method of medical image processing according to claim 1, wherein receiving the three-dimensional MRI image and the US image of the prostate as inputs and generating a modality presence indication mask for each input comprises: Wherein, the For an input MRI image, For the US image to be input, A mask is indicated for the presence of an MRI image, An indication mask for US image presence; The splicing input is as follows: , 。
3. the method for processing a medical image according to claim 2, wherein, Feature extraction includes MRI and US branches; The MRI branch consists of standard 3D convolution, instance normalization, leakyReLU activation functions: The US branches introduce a hole convolution: Wherein, the , To at the first MRI signature of the layer(s), To at the first US profile of the layer.
4. The method for processing a medical image according to claim 3, wherein, Generating the equation of Query, key, value: Wherein, the 、、 Is independent of Weights of the convolutional layers; The attention weight calculation formula is: Wherein, the Representation pair The operation of the transposition is carried out, Representation of Each position And (3) with All positions Is used to determine the similarity score of the (c), The scale factor is represented as such, Is that Is avoided by the channel dimension of The function enters a saturation region; the formula for generating the multi-mode fusion characteristic is as follows: Wherein, the Representing attention weights And (3) with According to the weighted sum of (2) For a pair of The polymerization is carried out and the polymerization is carried out, Representing the weight coefficient.
5. The method for processing a medical image according to claim 4, it is characterized in that the method comprises the steps of, The step up-sampling is realized by adopting deconvolution and jump connection, and the formula is as follows: Wherein, the Is shown in the first Decoding characteristics of the layer; The generation formula of the segmentation probability map is as follows: Wherein, the A MRI prostate segmentation probability map is represented, A US prostate segmentation probability map is represented, Representation of The function is activated and the function is activated, A final layer decoding feature map is represented, Representing one of MRI The weights of the convolutional layer are calculated, Representing one of US Weights of the convolutional layers; the generation formula of the initial displacement field is as follows: Wherein, the Representing the hyperbolic tangent activation function, Representing one Weights of the convolutional layers.
6. The method of claim 5, wherein the differential stratospheric integration is applied to the initial displacement field by the formula: Wherein, the Representing the displacement field, the initial displacement field ; Representing the number of integration times; Representing the composite operation of the displacement field, and performing composite iteration: The definition is as follows: The formula for generating the registered MRI image is as follows: Wherein, the Representing a voxel coordinate in US image space, Expressed in coordinates Differential embryo displacement field The displacement vector is given.
7. The method of claim 6, wherein the loss function formula of the deep learning network is: Wherein, the Indicating the total loss of the total of the components, The segmentation loss is indicated as a function of the segmentation loss, Representing a loss of registration, Representing a loss of physical constraint, Representing the weight of the partition loss, Representing the registration loss weight(s), Representing physical constraint loss weights; indicating a loss of Dice (r) that is, Representing the weighted cross entropy loss, Representing the weighted cross entropy loss coefficient, Representing predicted first The probability that a pixel belongs to the prostate, Represent the first The true label of the individual pixels is that, Representing the total number of pixels; Representing a loss of similarity of the images, Indicating a loss of smoothness and, therefore, The coefficient of loss of smoothness is indicated, Representing the normalized cross-correlation function, The image domain is represented by a representation of the image domain, Representing displacement fields In coordinates of A spatial gradient at the location of the gradient, Represents the square of the L2 norm; representing the number of sample points used to calculate the loss, Representing a probability map of prostate segmentation Using displacement fields The segmentation map obtained after registration deformation is performed, A function of the distance of the symbol is represented, Representing the L1 norm.
8. A medical image processing system, comprising: the input and dual-feature extraction module is used for receiving a three-dimensional MRI image and an US image of the prostate as input, generating a modal existence indication mask for each input, gradually downsampling, respectively extracting feature images of three levels of the MRI image and the US image, and generating a feature pyramid; The cross-modal attention fusion module is used for calculating a similarity matrix of the Query and the Key by taking MRI image features as the Query and taking US image features as the Key and the Value on each feature level, generating an attention weight graph by adopting Softmax normalization, carrying out weighted summation on the Value by using the attention weight, and fusing with the MRI image features to generate a multi-modal fusion feature; the multi-task decoding and outputting module is used for gradually up-sampling based on the multi-mode fusion characteristic golden tower to respectively generate an MRI prostate segmentation probability map, an US prostate segmentation probability map and an initial displacement field; And the mapping and registering constraint module is used for adopting differential and coherent integration processing to the initial displacement field and generating a registered MRI image.
9. A computer device comprising a memory and a processor, said memory and said processor being communicatively coupled to each other, said memory having stored therein computer instructions, said processor implementing a method of processing a medical image according to any of claims 1-7 by executing said computer instructions.
10. A computer-readable storage medium storing computer instructions that, when executed by a processor, implement the method of processing medical images according to any one of claims 1-7.

Description

Medical image processing method, system and medium Technical Field The application relates to the technical field of medical image processing, in particular to a medical image processing method, a medical image processing system and a medical image processing medium. Background Prostate cancer is one of the common malignant tumors in men, and its accurate diagnosis and treatment are severely dependent on guidance of multi-modal medical images. Magnetic Resonance Imaging (MRI) provides high resolution soft tissue anatomy, clearly shows the prostate region, but is generally not available in real time during surgery, whereas Ultrasound (US) imaging has the advantages of real time, portability, and low cost, and is the main modality of intra-operative navigation. Therefore, the accurate segmentation and registration fusion of the preoperative MRI and the intraoperative US image is important for realizing accurate prostate cancer targeted biopsy and radiotherapy. The inventors have found that the conventional approach mostly employs a split, serial process flow. The mode of sequentially executing the related tasks causes the gradual transmission and accumulation of errors among links, forms an inherent error chain, and causes the overall accuracy of the system to have an upper limit which is difficult to break through. The existing method has serious shortages in the cooperative utilization of multi-mode data. The complementary information provided by different imaging modes is often processed in isolation, and a unified understanding framework cannot be constructed, so that the migration and enhancement of knowledge among different modes cannot be realized, and the cognitive depth and robustness of a complex anatomical structure are limited. Current technology relies heavily on idealized, complete annotation data. The method has strict requirements on training data, cannot adapt to common reality of data isomerism, sparse labeling or incomplete labeling in clinical practice, and has low data utilization efficiency, so that generalization capability and practicability in real-world scenes are greatly reduced. Disclosure of Invention In view of the above, the present application provides a method, a system and a medium for processing medical images, so as to solve the technical problem of segmentation, registration and cleavage in the prior art. In a first aspect, the present application provides a method for processing a medical image, including: Receiving a three-dimensional MRI image and an US image of the prostate as inputs, generating a modal existence indication mask for each input, gradually downsampling, respectively extracting three-level feature images of the MRI image and the US image, and generating a feature pyramid; On each feature level, taking an MRI image feature as a Query, taking a US image feature as a Key and a Value, calculating a similarity matrix of the Query and the Key, generating an attention weight graph by adopting Softmax normalization, carrying out weighted summation on the Value by using the attention weight, and fusing with the MRI image feature to generate a multi-mode fusion feature; gradually up-sampling based on a multi-mode fusion characteristic golden tower to respectively generate an MRI prostate segmentation probability map, an US prostate segmentation probability map and an initial displacement field; And performing differential stratospheric integration processing on the initial displacement field, and generating a registered MRI image. Preferably, the receiving the three-dimensional MRI image and the US image of the prostate as inputs and generating a modality presence indication mask for each input comprises: Wherein, the For an input MRI image,For the US image to be input,A mask is indicated for the presence of an MRI image,An indication mask for US image presence; The splicing input is as follows: ,。 preferably, the feature extraction comprises an MRI branch and a US branch; The MRI branch consists of standard 3D convolution, instance normalization, leakyReLU activation functions: The US branches introduce a hole convolution: Wherein, the ,To at the firstMRI signature of the layer(s),To at the firstUS profile of the layer. Preferably, the formula Query, key, value is generated: Wherein, the 、、Is independent ofWeights of the convolutional layers; The attention weight calculation formula is: Wherein, the Representation pairThe operation of the transposition is carried out,Representation ofEach positionAnd (3) withAll positionsIs used to determine the similarity score of the (c),The scale factor is represented as such,Is thatIs avoided by the channel dimension ofThe function enters a saturation region; the formula for generating the multi-mode fusion characteristic is as follows: Wherein, the Representing attention weightsAnd (3) withAccording to the weighted sum of (2)For a pair ofThe polymerization is carried out and the polymerization is carried out,Representing