CN-121685568-B - Rock FIB-SEM sequence image multi-phase registration segmentation method and system

CN121685568BCN 121685568 BCN121685568 BCN 121685568BCN-121685568-B

Abstract

The invention belongs to the technical field of image processing, and provides a rock FIB-SEM sequence image multi-phase registration segmentation method and a system, wherein the technical scheme is based on a constructed detail retention type non-rigid registration sub-network, and registered images, displacement vector fields and jacobian thereof are output through affine correction and non-rigid registration; dividing the registered image based on the constructed deformation perception type multi-phase segmentation sub-network to obtain probability diagrams of a plurality of channels, carrying out joint training on the sub-network by taking the registration precision and the segmentation accuracy as targets and combining the constructed loss function to obtain each sub-network after parameter optimization, and processing the target sequence image based on each sub-network after parameter optimization to obtain three-dimensional image volume data and multi-phase semantic segmentation three-dimensional labels aligned with the three-dimensional image volume data. The high-precision image registration and semantic segmentation can be synchronously completed on the premise of keeping high-frequency details and noise characteristics with geological significance in the original image.

Inventors

XIONG QINGRONG
LI BO
CHEN YUKAI
WANG LIGE
ZHANG QIANGYONG
DUAN KANG
LIU QIFANG

Assignees

山东大学

Dates

Publication Date: 20260508
Application Date: 20260210

Claims (6)

1. The rock FIB-SEM sequence image multi-phase registration segmentation method is characterized by comprising the following steps of: Performing data preprocessing on the obtained rock FIB-SEM sequence image; Carrying out affine correction on the preprocessed floating image to obtain an affine corrected floating image, and carrying out non-rigid registration on the affine corrected floating image and the reference image through a detail retention type non-rigid registration sub-network to obtain a registered image and a displacement vector field and a jacobian corresponding to the registered image; the affine corrected floating image and the reference image are combined to carry out non-rigid registration through a detail retention type non-rigid registration sub-network, and the method comprises the following steps: Floating image after affine correction Reference image Sending the images into a U-Net network in a channel splicing way, and obtaining an affine corrected floating image through a decoder-encoder structure The displacement vector field of the pixel coordinates of (2) is used for generating an image through reverse mapping and bilinear interpolation, non-rigid registration is completed, and a non-rigid registration image is obtained The partial derivative calculation based on displacement vector field obtains jacobian, reflects local volume change, and the detail reservation is to introduce Sobel gradient extraction module at the output end of the registration sub-network to calculate non-rigid registration image With reference images L1 norm distance over the Sobel gradient domain; Dividing the registered image based on the displacement vector field, the jacobian and the constructed deformation perception type multi-phase dividing sub-network to obtain probability diagrams of a plurality of channels, wherein the probability diagrams correspond to various phases of an actual rock sample respectively; the displacement vector field and jacobian and constructed deformation perception type multi-phase segmentation sub-network based on the displacement vector field segments the registered image to obtain a probability map of a plurality of channels, and the method comprises the following steps: extracting features of the registered images to obtain a multi-scale main feature map; Splicing the displacement vector field output by the detail-preserving non-rigid registration sub-network and the jacobian to form tensor, and encoding to obtain a multi-scale deformation characteristic map with the same size as the multi-scale main characteristic map; Channel splicing is carried out on the multi-scale feature map and the deformation feature map according to the scale one by one, and then CBAM attention weighting is introduced to obtain a multi-scale fusion feature map; activating the multi-scale fusion feature map to output probability maps of a plurality of channels through a1×1 convolution channel map and Softmax; the registration accuracy and the segmentation accuracy are used as targets, and the detail-preserving type non-rigid registration sub-network and the deformation perception type multi-phase segmentation sub-network are combined and trained by combining the constructed loss function, so that each sub-network is obtained after parameter optimization; the loss function is formed by weighting two parts of registration loss and segmentation loss, and is expressed as: , , , Wherein, the For the registration of the loss weights, In order to register for the sub-network loss, In order to split the sub-network dynamic combining loss, 、 And A weight coefficient representing the corresponding term; For the dynamic weight to adjust according to the phase ratio and the prediction accuracy, Representing a loss of similarity of the images, Representing a deformation field smoothing regularization term, Representing the loss of the gradient of the detail retention, Indicating a loss of Dice (r) that is, Representing Focal loss; registration sub-network loss The first term is image similarity loss The alignment degree of the image and the reference image on the structure after registration is measured by adopting normalized cross correlation, and the second term is a deformation field smoothing regular term Constraining the second derivative of the displacement vector field by using bending energy, reserving gradient loss for details by a third term, and defining the L1 norm distance between the registered image and the reference image on a Sobel gradient domain; and processing the target FIB-SEM sequence image based on each sub-network after parameter optimization to obtain three-dimensional image volume data and multi-phase semantic segmentation three-dimensional labels aligned with the three-dimensional image volume data.
2. The rock FIB-SEM sequential image multi-phase registration segmentation method according to claim 1, wherein affine correction is carried out on the preprocessed floating image, the method comprises the steps of splicing the single-channel floating image with a reference image according to channels, obtaining input tensors, inputting 3 layers of convolution layers, extracting abstract features from edge textures to global deformation, outputting feature images by the 3 layers of convolution layers, flattening the 3 layers of convolution output feature images to form one-dimensional vectors, constructing an affine transformation matrix through a full-connection layer, and finally carrying out radiation transformation on the floating image based on the reference image by combining corresponding functions of the affine transformation matrix and Pytorch, so as to obtain the affine corrected floating image.
3. The rock FIB-SEM sequential image multi-phase registration segmentation method according to claim 1, wherein registration sub-network is lost And in the calculation, after the initial segmentation result is generated, detecting an extract phase boundary mask through a Canny edge, and re-merging the extract phase boundary mask into a registration loss function as soft constraint to guide a deformation field to carry out fine alignment in a key boundary area.
4. A rock FIB-SEM sequence image multi-phase registration segmentation system, which adopts the rock FIB-SEM sequence image multi-phase registration segmentation method according to any one of claims 1-3, and is characterized by comprising: the image preprocessing module is used for preprocessing the data of the obtained rock FIB-SEM sequence images; The image registration module is used for carrying out affine correction on the preprocessed floating image to obtain an affine corrected floating image, carrying out non-rigid registration on the affine corrected floating image and the reference image through a detail-preserving non-rigid registration sub-network to obtain a registered image and a displacement vector field and a jacobian corresponding to the registered image; the joint training module is used for carrying out joint training on the detail-retaining non-rigid registration sub-network and the deformation perception type multi-phase segmentation sub-network by taking registration precision and segmentation accuracy as targets and combining the constructed loss function to obtain each sub-network after parameter optimization; And the result output module is used for processing the target FIB-SEM sequence image based on each sub-network after parameter optimization to obtain three-dimensional image volume data and multi-phase semantic segmentation three-dimensional labels aligned with the three-dimensional image volume data.
5. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the steps in the rock FIB-SEM sequential image multi-phase registration segmentation method according to any one of claims 1-3.
6. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor performs the steps in the rock FIB-SEM sequential image multi-phase registration segmentation method according to any one of claims 1-3 when the program is executed.

Description

Rock FIB-SEM sequence image multi-phase registration segmentation method and system Technical Field The invention belongs to the technical field of image processing, and particularly relates to a rock FIB-SEM sequence image multi-phase registration segmentation method and system. Background The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art. The focused ion beam scanning electron microscope (FIB-SEM) technology can acquire a continuous two-dimensional section image sequence of the nanometer scale inside a rock sample by alternately executing ion beam layer-by-layer cutting and electron beam high-resolution imaging, so as to establish a high-fidelity digital core model to precisely quantify pore structures, mineral distribution, connectivity and interface characteristics, and has irreplaceable effects on deep energy engineering such as oil gas exploitation, underground hydrogen storage, carbon dioxide geological storage and the like. In continuous slicing processes of FIB-SEM devices for hours or even days, samples are affected by factors such as ion beam bombardment, thermal drift, mechanical vibration, and differences in shrinkage of different mineral phases, and complex non-rigid deformations are generated between adjacent slices, which can introduce dislocation, tearing, or blurring artifacts if reconstructed by direct stacking. The existing traditional rigid/affine registration method based on cross correlation or phase correlation cannot effectively process local large deformation, while the recently raised deep learning unsupervised registration method can estimate a dense deformation field, the optimization target of the method is guided by maximizing global gray similarity and assisted by strong smooth regularization, so that high-frequency components such as mineral grain boundaries, microcracks, nanopore edges and the like can be excessively smoothed and even erased, the key information is lost, the true texture is lost although the key information is macroscopically aligned, and the reality of a digital core is directly weakened by detail loss. On the other hand, different minerals, various pores and the like in the rock FIB-SEM image usually have the characteristics of gray level overlapping, boundary dispersion, low signal to noise ratio and the like, the traditional segmentation method (such as Otsu threshold, watershed and Canny edge detection) is difficult to be applied, the application of the deep semantic segmentation network is limited by high manual labeling cost and subjective difference, in addition, the existing segmentation model assumes that the input image is completely and accurately aligned in detail, the segmentation link and the registration link are fractured, and in fact, the segmentation network makes decisions on the image with poor information due to filtering high-frequency discrimination clues in the preprocessing step, so that the accumulation of geometric distortion and semantic errors is further aggravated. Disclosure of Invention In order to solve at least one technical problem in the background art, the invention provides a rock FIB-SEM sequence image multi-phase registration segmentation method and a system, which can synchronously complete high-precision non-rigid image registration and pixel-level semantic segmentation of multiple microcosmic phases such as minerals, pores and the like on the premise of retaining high-frequency details and noise characteristics with geological significance in an original FIB-SEM image, thereby providing key technical support for constructing a high-fidelity digital core. In order to achieve the above purpose, the present invention adopts the following technical scheme: The first aspect of the invention provides a rock FIB-SEM sequence image multi-phase registration segmentation method, which comprises the following steps: Performing data preprocessing on the obtained rock FIB-SEM sequence image; Carrying out affine correction on the preprocessed floating image to obtain an affine corrected floating image, and carrying out non-rigid registration on the affine corrected floating image and the reference image through a detail retention type non-rigid registration sub-network to obtain a registered image and a displacement vector field and a jacobian corresponding to the registered image; Dividing the registered image based on the displacement vector field, the jacobian and the constructed deformation perception type multi-phase dividing sub-network to obtain probability diagrams of a plurality of channels, wherein the probability diagrams correspond to various phases of an actual rock sample respectively; the registration accuracy and the segmentation accuracy are used as targets, and the detail-preserving type non-rigid registration sub-network and the deformation perception type multi-phase segmentation sub-network are combined and tra