CN-121998976-A - Lumen endoscopic lipid three-dimensional deep analysis method based on physical constraint and neural interpretable mapping

CN121998976ACN 121998976 ACN121998976 ACN 121998976ACN-121998976-A

Abstract

The invention discloses a three-dimensional deep analysis method of lumen endoscopic lipid based on physical constraint and neural interpretable mapping. The method comprises the steps of inputting an original intensity image into a pre-trained classification model to be processed to obtain a lipid probability vector corresponding to each frame of original intensity image, obtaining an initial class activation significance map according to a feature map output by a last convolution layer of the classification model, obtaining an optical attenuation coefficient according to the original intensity image, obtaining lipid significance according to the optical attenuation coefficient and the initial class activation significance map, respectively obtaining a lipid upper boundary and a lipid lower boundary according to the original intensity image and the lipid significance, obtaining thickness according to the lipid upper boundary and the lipid lower boundary, reconstructing three-dimensional distribution of lipid, and further obtaining a lipid volume, a minimum thickness and spatial positions of the lipid volume and the minimum thickness in circumferential direction and withdrawal direction according to the three-dimensional distribution of the lipid. The invention has the advantages of enhanced physical consistency, strong interpretability, high three-dimensional stability, friendly engineering deployment and the like.

Inventors

LI PENG
LIU JIEHONG
CHEN YITIAN
CHEN ZIYE
WANG BIN

Assignees

浙江大学嘉兴研究院
浙江大学

Dates

Publication Date: 20260508
Application Date: 20260408

Claims (10)

1. A three-dimensional deep analysis method of lumen endoscopic lipid based on physical constraint and neural interpretable mapping is characterized by comprising the following steps: s1, acquiring a B-scan original intensity image sequence acquired according to a withdrawal sequence by OCT imaging in a cavity, and inputting each frame of original intensity image in the sequence into a pre-trained classification model for processing to obtain a lipid probability vector corresponding to each frame of original intensity image; s2, acquiring an initial class activation significance map according to a feature map output by a last convolution layer of the pre-trained classification model, acquiring an optical attenuation coefficient according to an original intensity image, and acquiring final lipid significance according to the optical attenuation coefficient and the initial class activation significance map; S3, respectively acquiring a lipid upper boundary and a lipid lower boundary of each A-line according to the original intensity image and the lipid significance, and obtaining thickness according to the lipid upper boundary and the lipid lower boundary; And S4, reconstructing three-dimensional distribution of the lipid according to the upper boundary, the lower boundary and the significance of the lipid, and further obtaining the volume, the minimum thickness and the spatial positions of the lipid in the circumferential direction and the retraction direction according to the three-dimensional distribution of the lipid.
2. The method for three-dimensional depth analysis of a lumen endoscopic lipid based on physical constraint and neural interpretable mapping according to claim 1, wherein the step S1 is specifically: Acquiring a B-scan original intensity image sequence according to a retraction order, and carrying out original intensity image generation on each frame in the sequence Inputting the images into a pre-trained classification model to obtain an original intensity image of each frame Lipid probability for all corresponding circumferential A-lines Lipid probability for all circumferential A-lines Constitutive lipid probability vector Wherein In order to retract the directional frame index, And The index in the depth (radial) direction and the index in the circumferential direction, respectively.
3. The method for three-dimensional depth analysis of lumen endoscopic lipids based on physical constraints and neural interpretable mapping according to claim 1, wherein the step S2 is specifically: S21, recording a feature map output by the last convolution layer of the pre-trained classification model as a feature map Calculating a feature map And taking a global average in the depth direction h to obtain the importance weight of each channel k for A-line classification Feature map Weighted summation is carried out along the channel dimension, and finally a depth saliency map is obtained through a ReLU activation function Then to depth saliency map Performing one-dimensional median or Gaussian smoothing along the depth direction h and then performing linear normalization to obtain an initial class activation significance map ; S22, for the original intensity image Taking the points within the neighborhood window [ h-r, h+r ] in the depth direction of (h, w) for each position (h, w) Fitting a straight line model with Huber loss as a robustness criterion And according to the fit slope Obtaining initial estimation value of local optical attenuation coefficient Initial estimation of local optical attenuation coefficient Applying one-dimensional total variation smoothing regularization in depth direction h to obtain final optical attenuation coefficient ; S23, activating the saliency map according to the initial class And an optical attenuation coefficient Obtaining final lipid significance by adopting a product-normalization-gating-based method or an optimization-based iterative fine tuning method 。
4. A method for three-dimensional depth analysis of intraluminal lipid based on physical constraints and neural interpretable maps as set forth in claim 3, wherein: the product-normalization-gating-based method is processed according to the following formula to obtain the lipid significance : Wherein, the Is lipid significance; depth saliency, which is physical consistency; is an indication function; Lipid class probability for the w th A-line of the z-th frame; is a probability gating threshold; Is an intermediate fusion result; Is a depth direction index; To find the index i of all depth directions on w-th A-line Is the maximum value of (2); activating a saliency map for an initial class; activating a function for Sigmoid; Is a scaling parameter; are bias parameters; is the optical attenuation coefficient.
5. A method for three-dimensional depth analysis of a lumen endoscopic lipid based on physical constraints and neural interpretable mapping according to claim 3, wherein the method for iterative fine tuning based on optimization is specifically: D1, according to the lipid significance to be optimized And an optical attenuation coefficient Build alignment loss : Wherein, the Loss of alignment; KL divergence; Is a Sigmoid function; Is a scaling parameter; are bias parameters; representing the lipid significance to be optimized; d2, according to straight line model And an optical attenuation coefficient Construction of physical losses : Wherein, the Is a physical loss; Weight coefficients that are monotonicity constraint terms; Is an activation function; To represent logarithmic intensity images First partial derivative in depth direction h; Weight coefficients that are the decay coefficient smoothing term; Is a norm; Is the optical attenuation coefficient A gradient in the depth direction h; D3, defining an objective function to be set according to the following formula: D4, lipid significance to be optimized initially Activating saliency maps with initial classes As an initial value, and then obtaining an optimal solution for the objective function through iterative search The optimal solution is processed according to the following formula to obtain the final lipid significance ; Wherein, the Is a smoothing term weight coefficient; is one-dimensional total variation; is an indication function; Lipid class probability for the w th A-line of the z-th frame; Is a probability gating threshold.
6. The method for three-dimensional depth analysis of lumen endoscopic lipids based on physical constraints and neural interpretable mapping according to claim 1, wherein the step S3 is specifically: S31, for the original intensity image Deriving along the depth direction h to obtain a depth gradient map Lipid significance And depth gradient map Gradient amplitude of (2) Linear combination is carried out to obtain an upper boundary sensitive response diagram For the fixed circumferential direction w of each A-line, searching an upper boundary sensitive response graph in a preset searching range in the depth direction Depth position of local maximum point of (a) selected as lipid upper boundary corresponding to A-line ; S32, lipid-based significance Setting a significance threshold Will satisfy Is marked as candidate lipid core depth region, and calculating optical attenuation coefficient Gradient in depth direction h By setting a positive threshold Will satisfy Marking the region with attenuation coefficient as region with forward mutation, combining the lipid core candidate region with attenuation coefficient with forward mutation to form candidate depth point set for lower boundary detection ; S33, using the candidate depth point set The position h of (a) is taken as the lower boundary Constructing a one-dimensional energy function, and solving the minimized energy function by adopting a dynamic programming or graph-cut algorithm to obtain a lipid lower boundary corresponding to each A-line ; S34, lipid lower boundary Reduction of lipid upper boundary Obtain the thickness of 。
7. The method for three-dimensional depth analysis of lumen-endoscopic lipids based on physical constraints and neural interpretable mapping according to claim 6, wherein the method comprises the following steps: the energy function is set according to the following formula: Wherein, the As a function of energy; The weight coefficient of the data item; weight coefficients that are smoothing terms; And Respectively in the circumferential direction, adjacent to And The lower lipid boundaries corresponding to the two a-lines; to be at the lower boundary of lipid Lipid significance value at; to be at the lower boundary of lipid The gradient magnitude of the optical attenuation coefficient at that location, Only in candidate point sets And (5) internal selection.
8. The method for three-dimensional depth analysis of lumen endoscopic lipids based on physical constraints and neural interpretable mapping according to claim 1, wherein the step S4 is specifically: S41 according to the lipid upper boundary Lower lipid boundary And lipid significance Construction of saliency volumes for three-dimensional voxel levels using Huber losses, respectively And a smooth boundary surface A variation energy function of (2); S42, performing discrete solution on the variation energy function, and respectively performing three-dimensional voxel level saliency volume And a smooth boundary surface According to three-dimensional voxel level saliency volume Or a smooth boundary surface Can be reconstructed to obtain three-dimensional lipid distribution; S43, calculating the volume, the minimum thickness and the spatial positions of the lipid in the circumferential direction and the retraction direction according to the three-dimensional lipid distribution.
9. The method for three-dimensional depth analysis of lumen-endoscopic lipids based on physical constraints and neural interpretable mapping according to claim 8, wherein: the three-dimensional voxel level saliency volume And a smooth boundary surface The variation energy function of (2) is set according to the following formula: Wherein, the Representing a three-dimensional voxel level saliency volume; an initial observation value for a three-dimensional voxel level saliency volume; representing a smooth boundary surface; an initial observation value of the boundary curved surface; And All using robust Huber losses; And Are all weight coefficients; Is first order smoothing; is the partial derivative of the withdrawal direction z; is a second order difference in the circumferential direction w; indicating lipid significance.
10. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method according to any one of claims 1 to 9 when the processor executes the computer program.

Description

Lumen endoscopic lipid three-dimensional deep analysis method based on physical constraint and neural interpretable mapping Technical Field The invention belongs to the technical field of medical image processing and computer vision, and particularly relates to a three-dimensional depth analysis method for lumen endoscopic lipid based on physical constraint and neural interpretable mapping. Background As an important representative of the endoscopic imaging technique, optical Coherence Tomography (OCT) has become the gold standard imaging tool for identifying vulnerable plaque (e.g., lipid core) in coronary intervention by virtue of its high axial resolution on the order of microns. However, the inherent physical properties of OCT imaging pose serious challenges to automated lipid recognition, namely 1) the blurring of tissue boundaries due to strong speckle noise, low signal-to-noise ratio, 2) the near exponential decay of signals with depth, which makes deep tissue features weak and indistinguishable from noise, and 3) the blocking shadows caused by strong reflective structures such as calcification, which cause downstream signal loss, forming artifacts. Together, these factors result in OCT image interpretation that is highly dependent on physician experience and varies widely from observer to observer. In recent years, OCT lipid recognition methods based on deep learning have been significantly advanced. However, the existing methods have the following key limitations and defects, which limit the clinical transformation and reliable application: 1. The identification granularity is coarse, and the lack of fine depth positioning is that the existing method is mostly based on image block (patch) or whole frame classification, and the output of the existing method is usually region-level probability (such as plaque existence probability) or pixel-level binary segmentation map. Such methods cannot stably and accurately output the upper and lower boundaries of the lipid core along the depth direction of the vessel wall, and boundary information (especially the thinnest fibrous cap thickness) is the most critical clinical index for assessing plaque vulnerability. The root is that the feature learning of the network in the depth dimension lacks explicit anatomical and physical constraints. 2. The three-dimensional space consistency is poor, the existing method generally carries out independent analysis on the sequence images collected by retraction frame by frame, and the natural continuity of the vascular structure and the spatial continuity of the lipid plaque in the retraction direction (longitudinal direction) are ignored. The recognition result is unreasonable in jitter, fracture or mutation between adjacent frames, namely the problem of 'cross-frame inconsistency', a smooth and coherent three-dimensional lipid distribution model cannot be formed, and the reliability and the visualization effect of the result are reduced. 3. Ignoring imaging physical prior, the prior neural network is poor in interpretation, and the prior neural network is used as a 'black box' model, so that the decision process lacks of physical interpretation. Of particular importance, they fail to explicitly incorporate the core physical law of approximately exponential decay of OCT signals with depth. Thus, network-generated Class Activation Maps (CAMs) for interpreting decisions tend to exhibit physically unreasonable responses in the depth direction, such as activation in highly attenuated noise regions, or weak or even absent activation due to signal attenuation in real lipid regions (i.e., "depth drift"). This results in reduced reliability of the CAM, which is difficult to document with clinical physical awareness. 4. The robustness to imaging artifacts and noise is inadequate, despite the use of data enhancement and the like, the existing methods are still significantly inadequate for modeling speckle noise, occlusion shadows, and signal attenuation inherent in OCT. In areas with poor image quality (such as deep layers and shadow areas), model prediction is easy to generate isolated false positives or false negatives, and stability is required to be improved. In summary, the defects of the prior art can be summarized in that the existing deep learning lipid identification method has the problems of coarse output granularity, inconsistent three dimensions, unexplained physics, sensitivity to noise artifacts and the like in the OCT complex imaging environment, so that the stable, accurate and interpretable lipid core quantitative analysis result with three-dimensional spatial consistency is difficult to provide, and the clinical requirements of accurate interventional diagnosis and treatment cannot be completely met. Disclosure of Invention In order to solve the problems in the background art, the invention provides a lumen endoscopic lipid three-dimensional depth analysis method based on physical constraint and neural interpreta