CN-122000080-A - Breast cancer adjuvant therapy pathology complete remission prediction method and electronic equipment

CN122000080ACN 122000080 ACN122000080 ACN 122000080ACN-122000080-A

Abstract

The invention provides a method and electronic equipment for predicting complete alleviation of breast cancer assisted treatment pathology, wherein the method comprises the first stage of training an image feature encoder with tumor dynamic change recognition capability through a dynamic supervision pre-training mode based on multi-time point multi-parameter MRI images of a patient before and after assisted treatment, and the second stage of constructing a multi-mode deep fusion network based on the image feature encoder so as to integrate image features, radiology features and clinical pathology features of the patient and output a prediction result of complete alleviation of pathology, and the radiology features are extracted from the MRI images. The purposes of high precision and strong generalization capability are achieved.

Inventors

YANG JIALIANG
Yuan Pi
YANG WENYU
Liu Zanfei
Zhou Taizi
YANG ZIXUAN
TIAN GENG
ZHAO HAIWEN

Assignees

元码基因科技（北京）股份有限公司
中国医学科学院肿瘤医院

Dates

Publication Date: 20260508
Application Date: 20260118

Claims (10)

1. A method for predicting complete remission of a breast cancer adjuvant therapy pathology comprising: The method comprises the steps of firstly, training an image feature encoder with tumor dynamic change recognition capability through a dynamic supervision pre-training mode based on multi-time point multi-parameter MRI images of a patient before and after auxiliary treatment; And in the second stage, a multi-mode depth fusion network is constructed based on the image feature encoder so as to integrate image features, radiological features and clinical pathology features of a patient, and a prediction result of complete alleviation of pathology is output, wherein the radiological features are extracted from an MRI image.
2. The method of claim 1, wherein the dynamic supervised pretraining in the first phase comprises: adopting a double-tower encoder-decoder network model to respectively process MRI images before treatment and after treatment; training a dual-tower encoder-decoder by jointly optimizing three loss functions, the three loss functions comprising: reconstruction loss for constraining the reconstruction accuracy of the encoder-decoder to the input image; contrast loss for distinguishing between a pathologically complete remission patient and a non-pathologically complete remission patient based on differences in characteristics before and after treatment; And the auxiliary classification loss is used for correlating the difference characteristics of the pretreatment, the treatment and the treatment with the pathological complete remission label.
3. The method for predicting complete remission of breast cancer adjuvant therapy pathology according to claim 2, characterized in that the calculation process of the adjuvant classification loss comprises: stitching the pre-treatment features, the post-treatment features and the difference features; inputting the spliced characteristics into a convolution attention module for weighting; The weighted features are input into a linear classifier, and the difference from the true pathology complete remission label is calculated by using a binary cross entropy loss function.
4. The method for predicting complete remission of breast cancer adjuvant therapy pathology according to claim 3, characterized in that the calculation formula of the reconstruction loss is: , Wherein, the Representing the MRI modality of the patient, Representing a dynamic enhanced MRI, Representing a diffusion-weighted imaging of the image, And Representing raw MRI images before and after treatment respectively, And Representing the pre-treatment and post-treatment features extracted by the encoder respectively, The representation corresponds to a modality Is a decoder of (a); the calculation formula of the contrast loss is as follows: , Wherein the method comprises the steps of As the distance of the features, As a marginal super-parameter, the method comprises the steps of, A complete remission label for pathology; the calculation formula of the auxiliary classification loss is as follows: , Wherein, the As a feature of the difference value, , A convolution attention module is shown and, A linear layer is represented and is shown as such, The activation function is represented as a function of the activation, Representing a binary cross entropy loss function.
5. The method of claim 1, wherein the multimodal depth fusion network in the second phase comprises: the frequency domain transformation fusion module is used for carrying out self-adaptive weighting and frequency domain information mining on the image features from different sequences to obtain fusion image features; The cross-modal attention dynamic fusion module is used for carrying out attention weighting on the radiology characteristics based on the clinical characteristics and dynamically fusing the clinical and radiology characteristics to obtain fusion form characteristics; And the bidirectional cross attention fusion module is used for carrying out bidirectional interactive alignment on the fusion image features and the fusion form features to obtain full-mode comprehensive features.
6. The method of claim 5, wherein the frequency domain transform fusion module comprises: Generating weights through a gating mechanism, and carrying out self-adaptive weighting on image features from DCE-MRI and DWI modes; performing fast fourier transform on the weighted features to extract frequency domain information; And carrying out inverse Fourier transform on the frequency domain information and outputting the frequency domain information.
7. The method of claim 5, wherein the cross-modal attention dynamic fusion module comprises: Taking the clinical feature vector as a query, taking the radiological feature vector as a key and a value respectively, and calculating the attention weight; Weighting the radiology group characteristics according to the attention weight to obtain attention weighted radiology group characteristics; Clinical feature vectors are fused with attention weighted radiological features by learnable dynamic weights.
8. The method for predicting complete remission of breast cancer adjuvant therapy pathology according to claim 1, characterized in that, in the second stage, The multi-modal depth fusion network is optimized using a combined Loss function that includes the Focal Loss and cross entropy Loss with class weights, thereby completely alleviating the pathology from positive and negative sample balance.
9. The method for predicting complete remission of breast cancer adjuvant therapy pathology according to claim 1, characterized in that the second stage comprises: visualizing a key prediction area in the MRI image by using a gradient weighting activation map; based on the visualization results, SHAP analysis is utilized to quantify the contribution of each input feature to the final prediction result.
10. An electronic device, the electronic device comprising: At least one processor, and A memory communicatively coupled to the at least one processor, wherein, The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the breast cancer adjuvant therapy pathology complete remission prediction method of any one of claims 1-9.

Description

Breast cancer adjuvant therapy pathology complete remission prediction method and electronic equipment Technical Field The invention belongs to the technical field of medical image processing, and particularly relates to a method for predicting complete alleviation of pathology of breast cancer adjuvant therapy and electronic equipment. Background Breast cancer is a global main cause of high malignant tumor and cancer related death of women, new Adjuvant Therapy (NAT) is a core therapeutic strategy of locally advanced breast cancer, can reduce tumor volume, reduce metastasis risk, promote breast protection operation rate, and provide basis for evaluating tumor drug response, and pathologically complete remission (pCR) is a key index for measuring NAT curative effect and long-term prognosis of patients. The current pCR prediction method has the remarkable defects that clinical and pathological characteristics (such as hormone receptor and HER2 state) can only reflect partial information and cannot cover the complexity of tumor microenvironment, medical imaging (such as DCE-MRI) can provide morphological, blood perfusion and permeability information, but the traditional subjective evaluation or RECIST standard is insufficient in accuracy and timeliness, the traditional AI model depends on single mode or single time point data, the radiometric manual characteristic extraction is limited, the deep learning model does not fully utilize the dynamic change information of tumors in the NAT process, and the single-center small sample training is mostly carried out, the generalization capability is weak, the requirements of accurate and general prediction tools of clinical alignment are difficult to meet, and partial patients cannot obtain individual treatment even accept unnecessary operations. Disclosure of Invention Aiming at the problems in the prior art, the invention provides a method for predicting complete alleviation of breast cancer adjuvant therapy pathology and electronic equipment, which at least partially solve the problems of low prediction precision and weak generalization capability in the prior art. In a first aspect, embodiments of the present disclosure provide a method for predicting complete remission of a breast cancer adjuvant therapy pathology, comprising: The method comprises the steps of firstly, training an image feature encoder with tumor dynamic change recognition capability through a dynamic supervision pre-training mode based on multi-time point multi-parameter MRI images of a patient before and after auxiliary treatment; And in the second stage, a multi-mode depth fusion network is constructed based on the image feature encoder so as to integrate image features, radiological features and clinical pathology features of a patient, and a prediction result of complete alleviation of pathology is output, wherein the radiological features are extracted from an MRI image. Optionally, the dynamic supervision pre-training in the first stage includes: adopting a double-tower encoder-decoder network model to respectively process MRI images before treatment and after treatment; training a dual-tower encoder-decoder by jointly optimizing three loss functions, the three loss functions comprising: reconstruction loss for constraining the reconstruction accuracy of the encoder-decoder to the input image; contrast loss for distinguishing between a pathologically complete remission patient and a non-pathologically complete remission patient based on differences in characteristics before and after treatment; And the auxiliary classification loss is used for correlating the difference characteristics of the pretreatment, the treatment and the treatment with the pathological complete remission label. Optionally, the calculation process of the auxiliary classification loss includes: stitching the pre-treatment features, the post-treatment features and the difference features; inputting the spliced characteristics into a convolution attention module for weighting; The weighted features are input into a linear classifier, and the difference from the true pathology complete remission label is calculated by using a binary cross entropy loss function. Optionally, the calculation formula of the reconstruction loss is: , Wherein, the Representing the MRI modality of the patient,Representing a dynamic enhanced MRI,Representing a diffusion-weighted imaging of the image,AndRepresenting raw MRI images before and after treatment respectively,AndRepresenting the pre-treatment and post-treatment features extracted by the encoder respectively,The representation corresponds to a modalityIs a decoder of (a); the calculation formula of the contrast loss is as follows: , Wherein the method comprises the steps of As the distance of the features,As a marginal super-parameter, the method comprises the steps of,A complete remission label for pathology; the calculation formula of the auxiliary classification loss is as follows: