Search

CN-122023894-A - Intelligent assessment method and system for excision cleanliness in minimally invasive rotary cut of breast tumor

CN122023894ACN 122023894 ACN122023894 ACN 122023894ACN-122023894-A

Abstract

The invention relates to the technical field of medical image analysis, and discloses an intelligent evaluation method and system for cutting cleanliness in minimally invasive rotary cutting of breast tumor, wherein the method comprises the steps of firstly acquiring ultrasonic images before and during operation of a patient, matching a region of interest through structural similarity indexes, and respectively intercepting a main image and a local patch; the method comprises the steps of firstly, obtaining a local characteristic of a main image, then, respectively extracting the global characteristic and the texture characteristic of the local patch by using a deep neural network, then, performing spatial collaborative attention processing on the global characteristic to establish spatial association between preoperative and intra-operative images, performing multi-instance pooling operation on the local characteristic to aggregate key details, and inputting a classifier to output an evaluation result after the global characteristic and the local characteristic are adaptively fused through a gate control network. The system also introduces a sample source weighting strategy and a man-machine cooperation closed-loop mechanism optimization model training. The method effectively solves the problem of characteristic alignment caused by soft tissue deformation in operation, and remarkably improves the objectivity and accuracy of the assessment of the excision cleanliness.

Inventors

  • MA FEI
  • WANG LI

Assignees

  • 上海智滨科技有限公司

Dates

Publication Date
20260512
Application Date
20260126

Claims (10)

  1. 1. The intelligent evaluation method for the excision cleanliness in the minimally invasive rotary cutting operation of the breast tumor is characterized by comprising the following steps of: Step S1, acquiring a pre-operation ultrasonic image and an intra-operation ultrasonic image of a breast tumor of a patient, unifying image sizes through standardization processing, determining a pre-operation region of interest based on the pre-operation ultrasonic image, determining the intra-operation region of interest based on the intra-operation ultrasonic image, intercepting a pre-operation main image region and intercepting a plurality of pre-operation local patches by taking the center of the pre-operation region of interest as a reference, intercepting the intra-operation main image region and intercepting the plurality of intra-operation local patches by taking the center of the intra-operation region of interest as a reference; S2, constructing a deep neural network model, and respectively extracting features of the preoperative main image area and the intraoperative main image area by using the deep neural network model to obtain a preoperative main image feature map and an intraoperative main image feature map; S3, performing spatial collaborative attention processing on the preoperative main image feature map and the intra-operative main image feature map to obtain a collaborative attention fusion feature map; executing multi-instance pooling operation on the pre-operation patch feature matrix and the intra-operation patch feature matrix, and polymerizing to obtain pre-operation patch aggregation features and intra-operation patch aggregation features; S4, carrying out self-adaptive fusion on the collaborative attention fusion feature map, the preoperative patch aggregation feature and the intraoperative patch aggregation feature through a gating network to obtain a final fusion feature vector; And S5, inputting the final fusion feature vector into a classifier, and outputting an evaluation result of the cutting cleanliness.
  2. 2. The method for intelligently assessing the cleanliness of ablation in minimally invasive rotational atherectomy of breast tumor according to claim 1, wherein the determining the intraoperative region of interest based on the intraoperative ultrasound image comprises the following specific steps: Calculating the structural similarity index of the preoperative region of interest and each candidate region in the intraoperative ultrasonic image; Setting a similarity threshold, and if the structural similarity index of the candidate region and the preoperative region of interest is larger than the similarity threshold, automatically selecting the candidate region with the largest structural similarity index as the intraoperative region of interest; And if the structural similarity indexes of all the candidate regions and the preoperative region of interest are smaller than or equal to the similarity threshold, manually designating the intraoperative region of interest by receiving a manual instruction.
  3. 3. The intelligent assessment method for excision cleanliness in minimally invasive rotary cutting of breast tumor according to claim 1, wherein a RepVGGDW depth separable convolution module and a LSConv local sparse convolution module are integrated in the deep neural network model; the feature extraction stage of the deep neural network model is divided into a plurality of continuous stages, and the plurality of continuous stages are sequentially provided with different embedding dimensions and convolution block numbers.
  4. 4. The method for intelligently assessing the cleanliness of ablation in minimally invasive rotational atherectomy of breast tumor according to claim 1, wherein the performing of a spatial collaborative attention process on the pre-operative main image feature map and the intra-operative main image feature map comprises the following steps: defining a preoperative feature projection weight matrix, an intra-operative feature key projection weight matrix and an intra-operative feature value projection weight matrix; Performing dimension conversion on the preoperative main image feature map by using the preoperative feature projection weight matrix to obtain query features, performing dimension conversion on the operative main image feature map by using the intra-operative feature key projection weight matrix to obtain key features, and performing dimension conversion on the operative main image feature map by using the intra-operative feature value projection weight matrix to obtain value features; Flattening the query feature and the key feature into a one-dimensional sequence, and calculating the spatial correlation between the query feature and the key feature through dot product operation to obtain an attention score matrix; Introducing a scaling factor to carry out numerical adjustment on the attention score matrix, and carrying out normalization processing on the attention score matrix by adopting a Softmax function to obtain an attention weight matrix; performing matrix multiplication operation on the attention weight matrix and the value characteristic to obtain a weighted attention characteristic diagram; And remodelling the weighted attention feature map into a two-dimensional feature map, and carrying out element-level addition fusion on the remodelled two-dimensional feature map and the original preoperative main image feature map to obtain the collaborative attention fusion feature map.
  5. 5. The method for intelligently evaluating the resecting cleanliness in minimally invasive rotational atherectomy of breast tumor according to claim 1, wherein the performing of multi-instance pooling on the pre-operative patch feature matrix and the intra-operative patch feature matrix comprises the steps of: constructing a two-layer neural network comprising a linear transformation layer and an output layer; Inputting each preoperative patch characteristic vector in the preoperative patch characteristic matrix into the two-layer neural network, and calculating to obtain the attention score of each preoperative patch characteristic vector; Normalizing attention scores of all the preoperative patch feature vectors by adopting a Softmax function to obtain attention weight of each preoperative patch feature vector; multiplying each pre-operation patch characteristic vector with the corresponding attention weight at element level, and summing all product results to obtain the pre-operation patch aggregation characteristic; And calculating the intraoperative patch feature matrix according to the same flow to obtain the intraoperative patch aggregation feature.
  6. 6. The method for intelligently assessing the cleanliness of ablation in minimally invasive rotational atherectomy of breast tumor according to claim 1, wherein the self-adaptive fusion of the cooperative attention fusion feature map, the preoperative patch aggregation feature and the intraoperative patch aggregation feature is performed through a gating network, specifically comprising the following steps: Global average pooling processing is carried out on the collaborative attention fusion feature map, and pooled fusion image features are obtained; Splicing the preoperative patch aggregation features and the intraoperative patch aggregation features according to channel dimensions, and projecting the spliced vectors back to preset dimensions through a linear projection layer to obtain fused patch features; Splicing the pooled fused image features and the fused patch features according to channel dimensions, and inputting the spliced fused image features and the fused patch features into the gate control network; The gating network comprises two parallel linear layers, normalization weights are respectively output, and the normalization weights are constrained between zero and one by using a Sigmoid function; and carrying out weighted summation on the pooled fused image features and the fused patch features by using the normalized weights output by the gating network to obtain the final fused feature vector.
  7. 7. The method for intelligently assessing the cleanliness of ablation in minimally invasive rotational atherectomy of breast tumor according to claim 1, wherein the step S5 further comprises model training, specifically comprising the steps of: defining a focus loss function, wherein the focus loss function comprises a category balance factor for balancing the difference of the number of samples and a focus parameter for adjusting the attention degree of the samples difficult to classify; introducing a sample source weighting strategy, and setting different sample weights for image data of different sources; Setting a first sample weight for a sample manually circumscribing the intraoperative region of interest by a human; for automatically selecting samples of the intra-operative region of interest by an algorithm, setting a second sample weight, and the first sample weight value is greater than the second sample weight value; and carrying out weighted calculation on the focus loss function based on the sample weight to obtain the final training batch loss, and updating parameters of the deep neural network model by using a random gradient descent optimizer.
  8. 8. The method for intelligently assessing the cleanliness of ablation in minimally invasive rotational atherectomy of breast tumor according to claim 1, wherein the step S3 further comprises a visual feedback process, specifically comprising the steps of: Extracting the attention weight matrix during the spatially collaborative attention process and the attention weight during the multi-instance pooling operation; Converting the attention weight matrix into a spatial collaborative attention thermodynamic diagram through a pseudo-color mapping algorithm, and superposing the spatial collaborative attention thermodynamic diagram on the preoperative ultrasonic image and the intra-operative ultrasonic image; and generating marks with different color depths at the corresponding local patch positions according to the attention weight of the multi-instance pooling operation, forming a multi-instance pooling thermodynamic diagram and performing superposition display.
  9. 9. The method for intelligently evaluating the ablation cleanliness in minimally invasive rotary cutting of breast tumor according to claim 1, wherein the step S5 further comprises a man-machine cooperation closed-loop optimization process, and specifically comprises the following steps: Acquiring the prediction probability output by the classifier, and prompting a doctor to check if the prediction probability is lower than a preset confidence coefficient threshold; Receiving a test result input by a doctor, and if the test result judges that the evaluation result of the classifier is wrong, storing the corresponding preoperative ultrasonic image, the intraoperative ultrasonic image and the correct test result in a sample database in a correlated manner; Samples marked as errors are periodically extracted from the sample database and added into a training data set, and the deep neural network model is retrained.
  10. 10. An intelligent evaluation system for the excision cleanliness in minimally invasive rotary cutting of breast tumor, which is used for realizing the intelligent evaluation method for the excision cleanliness in minimally invasive rotary cutting of breast tumor according to any one of claims 1-9, and is characterized by comprising the following modules: The data preprocessing module is used for acquiring a preoperative ultrasonic image and an intraoperative ultrasonic image of a patient, executing image standardization processing, determining a preoperative region of interest and an intraoperative region of interest, and intercepting a preoperative main image region, an intraoperative main image region, a plurality of preoperative local patches and a plurality of intraoperative local patches; The feature extraction module is used for operating a deep neural network model and respectively extracting global features of the preoperative main image area and the intraoperative main image area and local features of the preoperative local patch and the intraoperative local patch; The feature enhancement module is used for performing spatial collaborative attention processing to correlate preoperative and intraoperative main image features and performing multi-instance pooling operation to aggregate local patch features; the feature fusion module is used for fusing the enhanced global features and the aggregated local features through a gating network and a self-adaptive weight distribution mechanism to generate a final fused feature vector; And the classification evaluation module is used for mapping the final fusion feature vector into a cut-off cleanliness class and executing model training and parameter updating based on a weighted focus loss function.

Description

Intelligent assessment method and system for excision cleanliness in minimally invasive rotary cut of breast tumor Technical Field The invention relates to the technical field of medical image analysis, in particular to an intelligent assessment method and system for excision cleanliness in minimally invasive rotary cutting of breast tumors. Background Minimally invasive rotary cutting of mammary gland is an important means for clinically diagnosing and treating mammary gland focus at present, and the breast tumor is completely resected or biopsied mainly through a vacuum auxiliary rotary cutting system under ultrasonic guidance. The evaluation of the cleanliness of the surgical resection, namely, the accurate judgment of whether the tumor tissue is completely removed without residue, is a core link for determining success or failure of the surgery and reducing the risk of postoperative recurrence. In the operation process, the cutting range is monitored and the cutting edge state is confirmed by a real-time image means, so that the method has important significance for guaranteeing the prognosis of a patient. In the prior art, the evaluation of the cleanliness of surgical resection mainly depends on the high-frequency ultrasonic imaging technology. A physician typically acquires an ultrasound image of a tumor prior to surgery to determine lesion location, size, and morphological features, as a benchmark for surgical planning. After the operation is completed, a doctor scans the operation residual cavity area by using an ultrasonic probe, observes ultrasonic echo manifestation of the residual cavity wall and surrounding tissues, and compares the operation image acquired in real time with the preoperation image in a visual aspect. The clinician mainly uses personal experience to subjectively infer whether tumor residues exist by identifying whether abnormal areas similar to the echo characteristics of the original tumor exist at the periphery of the residual cavity. However, a key challenge when evaluating in the manner described above is the difficulty in alignment of features caused by soft tissue deformation. Because of the high flexibility of mammary tissue, the mechanical pulling of the rotary cutter, the negative pressure suction effect and the residual cavity collapse caused by tumor removal during the operation can lead to the remarkable non-rigid deformation of the tissue structure of the intraoperative wound surface area. The drastic spatial morphological change causes that the intraoperative ultrasonic image and the preoperative reference image lose a direct corresponding relation in geometric structure, so that the focus characteristic position in the preoperative image cannot be accurately mapped to the corresponding coordinate of the intraoperative image, and therefore, the accurate identification of the tiny residual focus in the complicated deformed wound surface is difficult by relying on vision alone or a simple image superposition technology. Disclosure of Invention Aiming at the defects of the prior art, the invention provides an intelligent assessment method and system for the excision cleanliness in the minimally invasive rotary cutting operation of breast tumors, and solves the problems mentioned in the background art. In order to achieve the above purpose, the invention is realized by the following technical scheme that the invention provides an intelligent assessment method and a system for the excision cleanliness in the minimally invasive rotary cut of breast tumor, and the first aspect of the invention provides an intelligent assessment method for the excision cleanliness in the minimally invasive rotary cut of breast tumor, which comprises the following steps: Step S1, medical image acquisition and preprocessing And acquiring preoperative ultrasonic images and intraoperative ultrasonic images of the patient, and unifying the sizes through standardization treatment. And automatically matching or manually designating the intraoperative region of interest by calculating the structural similarity index of each candidate region and the preoperative region of interest based on the intraoperative ultrasonic image. And then taking the center of the region of interest as a reference, respectively intercepting the preoperative and intraoperative main image regions containing global information and a plurality of preoperative and intraoperative local patches containing local texture information. Step S2, feature extraction Constructing and utilizing a deep neural network model, respectively carrying out convolution operation on a preoperative main image area and an intraoperative main image area, and extracting a global feature map; and simultaneously, carrying out convolution operation on each preoperative local patch and each intra-operative local patch to extract a local feature matrix. The depth neural network model is internally integrated with a depth separable convolution module and