Search

CN-121999018-A - Medical image registration method, system and medium

CN121999018ACN 121999018 ACN121999018 ACN 121999018ACN-121999018-A

Abstract

The invention relates to a medical image registration method, a system and a medium, and the medical image registration method comprises the steps of receiving a preoperative CT image and an intraoperative X-ray image, utilizing a differentiable X-ray renderer to project the preoperative CT image into a two-dimensional synthetic X-ray image, filtering extracted features of the two-dimensional synthetic X-ray image and the intraoperative X-ray image layer by layer through an encoder to obtain a feature enhancement and suppression processed layer by layer feature map, extracting multidimensional features of the two-dimensional synthetic X-ray image and the intraoperative X-ray image by combining a convolution and attention mechanism, fusing the multidimensional features to obtain global integrated features, performing iterative optimization on the pose of a three-dimensional image, and outputting the final pose estimation after optimization. The invention can realize high-efficiency and accurate spatial registration between the X-ray image in the two-dimensional operation and the CT image before the three-dimensional operation, and remarkably improves the accuracy and speed of registration.

Inventors

  • LI TIANXING
  • DU WANTING
  • SHI RUI

Assignees

  • 北京工业大学

Dates

Publication Date
20260508
Application Date
20251121

Claims (9)

  1. 1. A method of medical image registration for differential rendering and enhancement of feature expression, comprising: Step 1, receiving a preoperative CT image and an intraoperative X-ray image; step 2, utilizing a differentiable X-ray renderer to project the preoperative CT image into a two-dimensional synthetic X-ray image; Step 3, extracting features of the two-dimensional synthesized X-ray image and the intra-operative X-ray image layer by layer through an encoder, generating channel weights and acting on the layer of feature images to obtain a layer-by-layer feature image after feature enhancement and suppression processing; Step 4, extracting multidimensional features from the two-dimensional synthesized X-ray image and the intraoperative X-ray image by combining convolution and an attention mechanism, and fusing to obtain global integrated features; Step 5, iteratively optimizing the pose of the three-dimensional image by utilizing the differentiable X-ray renderer and the sparse multiscale normalized cross-correlation loss; and 6, outputting the optimized final pose estimation.
  2. 2. A medical image registration method according to claim 1, wherein in step 2, the differentiable X-ray renderer is configured to extract the image from the source point The radiation is emitted, passes through the three-dimensional CT voxels and finally falls on the pixel points of the detector Calculating the path of the emitted rays : , wherein, Is a regularized weight for controlling the proportion of geometric constraints in the total loss, on which path each three-dimensional CT voxel has a linear attenuation coefficient Calculating to obtain a detector pixel point Is set, the predicted projection value of: , wherein, Is a detector pixel point Is used for the prediction strength of the (c) signal, Is the initial intensity of the radiation which, Is the first on the ray path The attenuation coefficient of the individual sample points, Is the traversing length of the ray within the three-dimensional CT voxel, Is the total number of intersecting rays and three-dimensional CT voxels, and synthesizes the X-ray image based on the predicted projection values of each pixel point of the detector.
  3. 3. The medical image registration method according to claim 1, wherein in step 3, specifically comprising: Firstly, carrying out global average pooling on each layer of feature map to obtain channel statistics: , wherein, Is the first The channel being in position Is used for the characteristic value of the (c), Is to the first The individual channels are globally averaged pooled statistics, Is the pixel height of the layer profile, Is the pixel width of the layer profile, Is an index in the height direction and, Is an index in the width direction; Modeling the correlation among channels based on the channel statistics, and calculating the channel attention weight vector : , wherein, Is a Sigmoid activation function, and the output range is within , Is a vector of all channel statistics , Finally, broadcasting the weight to the space dimension, multiplying the weight by the original characteristic channel by channel, and adding the weight to the original characteristic residual access form to obtain the output characteristic of each channel: , wherein, Is weighted and then The output characteristics of the individual channels are such that, Is the first And obtaining the processed layer-by-layer characteristic map by adopting the steps.
  4. 4. The medical image registration method according to claim 1, wherein in step 4, specifically comprising: Step 41, after the overall feature extraction of the backbone network, two parallel branches are arranged, namely a convolution branch is used for extracting transverse and longitudinal structural modes, and an attention branch is used for obtaining a query feature Q, a key feature K and a value feature V through convolution decomposition and carrying out additive similarity calculation to obtain an attention weight matrix: ; Wherein, the Is a Softmax function, the similarity is normalized to a probability distribution, Is a query vector or matrix, representing the features that currently need to be matched, Is a key vector or matrix, represents identification information for each location, Is a similarity matrix, representing the degree of matching between the query and the key, Is a attention weight matrix; And to value characteristics Weighting and projecting, and calculating to obtain attention enhancement features: , wherein, Is the first The output characteristics of the individual positions are used, Is the first Number of position pairs The attention weight of the individual locations, Is the first A value characteristic of the individual locations; step 42 by a learnable non-negative weight Carrying out normalized fusion on the convolution branch, the attention branch and the identity branch to obtain a normalized weight coefficient: , wherein, Is a non-negative weight parameter that can be learned, A non-negative guaranteed weight that is not negative, Is the sum of all the weights of the two, Is a stable term that prevents the division by zero, Is the normalized weight coefficient; Step 43, weighting the convolution branch, the attention branch and the identity branch to obtain a global integrated feature: , wherein, Is a characteristic of the output of a convolved branch, Is an attention branching output feature, Is an input feature that is used to determine the input, Is the normalized weight of each branch, Is the fused output feature.
  5. 5. A medical image registration method according to claim 1, wherein in step 5, radiation is emitted from the X-ray source point to the detector pixels, and the voxels are line-integrated along the radiation path to obtain predicted projection values at specific locations of the detector pixels: , wherein, Is at the detector pixel The predicted projection value at the location is calculated, Is the first The linear attenuation coefficient of each voxel, Is the ray at the first The path length within a voxel is determined by the number of voxels, Is a cumulative summation along the entire ray direction; a sparse sampling mask is constructed on the projection image, rendering only the patch center area to speed up mNCC computations.
  6. 6. The medical image registration method according to claim 1, wherein in step 5, the method specifically comprises the steps of calculating a local and global normalized cross-correlation at a plurality of scales: , wherein, Is the pixel intensity vector of the observed X-rays and the predicted DRR, Is the length of the number of pixels of the vector, Is the first The value of the individual pixels is determined, Is the mean value of the corresponding vector, Is the standard deviation of the corresponding vector, Is a stable term, prevents the denominator from being zero, Normalized cross-correlation similarity; step 52, calculating the similarity by weighting the local and global terms : , wherein, Is the NCC similarity of the local region, Is the NCC similarity of the global region, Is the weight coefficient of the weight of the object, Is the final weighted similarity; parameterizing camera pose into lie algebra minimum vector Obtaining a transformation matrix through exponential mapping: , wherein, Is used for mapping of the Liquorice index In the form of an antisymmetric matrix of (a), Is a matrix-index mapping of the matrix, Is a pose transformation matrix, belonging to SE (3) group; step 53, obtaining an optimized objective function through calculation by weighting the similarity term and the geometric regular term: , wherein, Is a similarity term from the NCC, Is a geometric regular term for constraining the pose, , Is regularized weight, and adopts a gradient-based optimization method to carry out iterative updating.
  7. 7. The method for registration of medical images according to claim 6, wherein in step 5, the pose optimization objective function is as follows: , wherein, Is regularization weight for balancing similarity term and regularization term, Is a regularization term for pose parameters: , is the normalized cross-correlation loss between the predicted DRR image and the actual X-ray image: , wherein, Is the pixel value of the predicted DRR image, Is the pixel value of the real X-ray image, Is a predictive DRR image Is a mean value of all the pixel values of (a), Is a true X-ray image Is used for the average value of (a), Is covariance, representing the correlation of two images at the pixel level, Is the variance of the predicted DRR image, Is the variance of the real X-ray image, Is a normalization factor for eliminating scale effects, and the similarity takes the negative sign as a loss value.
  8. 8. A medical image registration system for differential rendering and enhancing feature expression of medical images, comprising: The data acquisition module is used for receiving the preoperative CT image and the intraoperative X-ray image; The data processing module is used for projecting the preoperative CT image into a two-dimensional synthetic X-ray image by using a differentiable X-ray rendering method of physical modeling; The characteristic enhancement module is used for extracting characteristics of the two-dimensional synthesized X-ray image and the intra-operative X-ray image layer by layer through the encoder, generating channel weights and acting on the layer of characteristic images to obtain a layer-by-layer characteristic image after characteristic enhancement and suppression processing; the integrated feature processing module is used for extracting multidimensional features from the two-dimensional synthesized X-ray image and the intraoperative X-ray image by combining convolution and an attention mechanism, and fusing the multidimensional features to obtain global integrated features; The pose optimization module is used for carrying out iterative optimization on the pose of the three-dimensional image by utilizing the differentiable X-ray renderer and the sparse multiscale normalization cross-correlation loss; And the pose estimation module is used for outputting the optimized final pose estimation.
  9. 9. A computer readable storage medium having stored therein instructions which, when executed, perform a medical image registration method as claimed in any one of claims 1-7.

Description

Medical image registration method, system and medium Technical Field The present invention relates to the field of medical image processing. And more particularly to a medical image registration method, system, and medium. Background The primary purpose of medical image registration is to determine the anatomical correspondence of a pair of input images in space, thereby assisting the physician in performing the task. Medical image registration has wide application in the clinical direction and has been receiving attention from the medical field. The direction of navigation and the like plays a crucial role in image segmentation. In scenes such as navigation during surgery and robot-assisted surgery, two-dimensional intra-operative X-ray images and three-dimensional pre-operative CT images are often registered to determine spatial anatomical correspondence. However, the existing method has the following defects that 1, the traditional optimization method based on intensity is easy to fall into a local extremum and has low efficiency, 2, the deep learning method has limited feature extraction capability although the speed is improved, and the registration accuracy is insufficient due to redundant features and lack of global expression, and 3, the existing network has defects in layer-by-layer feature modeling and global information integration, so that the detail and the global are difficult to be compatible. How to overcome the defects is a problem to be solved. Disclosure of Invention The invention provides a medical image registration method, a medical image registration system and a medical image registration medium, which are used for solving the problems of low efficiency and insufficient precision of space registration of an X-ray image in a two-dimensional operation and a CT image before a three-dimensional operation. To achieve the above object, in a first aspect, the present invention relates to a medical image registration method for differential rendering and enhancing feature expression of medical image registration, comprising: Step 1, receiving a preoperative CT image and an intraoperative X-ray image; step 2, utilizing a differentiable X-ray renderer to project the preoperative CT image into a two-dimensional synthetic X-ray image; Step 3, extracting features of the two-dimensional synthesized X-ray image and the intra-operative X-ray image layer by layer through an encoder, generating channel weights and acting on the layer of feature images to obtain a layer-by-layer feature image after feature enhancement and suppression processing; Step 4, extracting multidimensional features from the two-dimensional synthesized X-ray image and the intraoperative X-ray image by combining convolution and an attention mechanism, and fusing to obtain global integrated features; Step 5, iteratively optimizing the pose of the three-dimensional image by utilizing the differentiable X-ray renderer and the sparse multiscale normalized cross-correlation loss; and 6, outputting the optimized final pose estimation. To achieve the above object, in a second aspect, the present invention relates to a medical image registration system for differential rendering and medical image registration of enhanced feature expression, comprising: The data acquisition module is used for receiving the preoperative CT image and the intraoperative X-ray image; The data processing module is used for projecting the preoperative CT image into a two-dimensional synthetic X-ray image by using a differentiable X-ray rendering method of physical modeling; The characteristic enhancement module is used for extracting characteristics of the two-dimensional synthesized X-ray image and the intra-operative X-ray image layer by layer through the encoder, generating channel weights and acting on the layer of characteristic images to obtain a layer-by-layer characteristic image after characteristic enhancement and suppression processing; the integrated feature processing module is used for extracting multidimensional features from the two-dimensional synthesized X-ray image and the intraoperative X-ray image by combining convolution and an attention mechanism, and fusing the multidimensional features to obtain global integrated features; The pose optimization module is used for carrying out iterative optimization on the pose of the three-dimensional image by utilizing the differentiable X-ray renderer and the sparse multiscale normalization cross-correlation loss; And the pose estimation module is used for outputting the optimized final pose estimation. To achieve the above object, the present invention in a third aspect also relates to a computer readable storage medium having stored therein instructions which, when executed, perform a medical image registration method as described above. Compared with the prior art, the medical image registration method, the medical image registration system and the medium have the following beneficial effects: The