Search

CN-121999028-A - Processing method and device for measuring fiber diameter based on scanning electron microscope

CN121999028ACN 121999028 ACN121999028 ACN 121999028ACN-121999028-A

Abstract

The embodiment of the invention relates to a processing method and a processing device for measuring fiber diameters based on a scanning electron microscope image, wherein the method comprises the steps of constructing and training a prediction model, receiving an SEM image X 0 of any fiber sample, inputting the prediction model to obtain corresponding edge point and center point probability images, extracting unique paths of each fiber according to the center point probability images by using a graph theory algorithm to obtain a plurality of fiber centerlines, identifying positive and negative normal direction radiuses of each point of each centerline according to the edge point probability images to obtain corresponding radius sequences, optimizing the diameters of the centerlines according to an energy function-driven sub-pixel finishing method, removing abnormal diameters based on global statistical distribution characteristics of the diameter sequences, and visually marking the fiber diameters of X 0 based on each fiber centerline and the corresponding diameter sequences. The invention can improve the measurement accuracy and the measurement robustness of the fiber diameter.

Inventors

  • YANG YAOTIAN
  • XIA ZHIYI
  • HONG YANHUI
  • ZHANG LINFENG

Assignees

  • 北京深势科技股份有限公司

Dates

Publication Date
20260508
Application Date
20260212

Claims (12)

  1. 1. A processing method for measuring fiber diameter based on scanning electron microscope, which is characterized by comprising the following steps: Establishing a deep learning model for predicting fiber edges and fiber center lines of a fiber sample scanning electron microscope image as a first prediction model, wherein the first prediction model is used for predicting fiber edges along lines and center line points of an SEM image X 0 input by the model and outputting a corresponding edge point probability image P 1 and a corresponding center point probability image P 2 , the first prediction model comprises a trunk feature extraction network, an edge point prediction network and a center point prediction network, the trunk feature extraction network is used for extracting trunk features of the SEM image X 0 to obtain a feature tensor Z 0 , and the edge point prediction network and the center point prediction network are respectively used for predicting corresponding edges along lines and the center line point probability image according to the feature tensor Z 0 to obtain a corresponding edge point probability image P 1 and a corresponding center point probability image P 2 ; training the first prediction model based on a preset tag data set; After model training is finished, a scanning electron microscope image of any fiber sample is received as a corresponding SEM image X 0 to be input into the first prediction model to obtain a corresponding edge point probability image P 1 and a corresponding center point probability image P 2 ; Extracting unique paths of each fiber according to the center point probability map P 2 by using a graph theory algorithm to obtain a plurality of corresponding fiber center lines S, wherein the fiber center lines S are formed by sequentially sequencing a plurality of center points S k (i k ,j k ), and indexes k are positive integers, counted from 1 and sequentially added with 1;i k 、j k to be row indexes and column indexes corresponding to the center points S k respectively; Identifying positive and negative normal direction radii R k+ 、r k- of the center points S k of each fiber center line S according to the edge point probability map P 1 to obtain a corresponding radius sequence R, wherein the radius sequence R is formed by sequentially sequencing a plurality of groups of radius pairs rc k (r k+ ,r k- ); The center line diameter of each radius sequence R is optimized according to an energy function driven sub-pixel finishing method to obtain a corresponding diameter sequence D, wherein the diameter sequence D consists of a plurality of diameters D k ; Performing abnormal diameter elimination processing based on global statistical distribution characteristics of the diameter sequences D to obtain a corresponding diameter sequence D opt , wherein the diameter sequence D opt is formed by sequentially sequencing a plurality of diameters D o , an index o is a positive integer, the number is counted from 1, the number is sequentially increased by 1, and each diameter D o corresponds to a group of center points Edge points Edge points ; 、 As the center point Row index and column index of (a); 、 For the edge points Row index and column index of (a); 、 For the edge points Row index and column index of (a); And carrying out corresponding fiber diameter visualization marking treatment on the SEM image X 0 based on each fiber center line S and the corresponding diameter sequence D opt .
  2. 2. The method for processing fiber diameter measurement based on scanning electron microscope according to claim 1, wherein, The SEM image X 0 is a fiber sample scanning electron microscope image, the image comprises a plurality of single fibers, the shape of the SEM image X 0 is 1 XH X W, the H, W is the height and the width of the SEM image X 0 , the characteristic dimension of a pixel point of the SEM image X 0 is 1, the SEM image X 0 consists of H X W pixel gray scales X i,j , row index i is not less than 1 and not more than H, and column index j is not less than 1 and not more than W; The shapes of the edge point probability map P 1 and the center point probability map P 2 are 1 XH×W, the characteristic dimension of the pixel point is 1, and the edge point probability map P 1 and the center point probability map P 2 are formed by corresponding H×W edge line site probabilities Probability of centerline locus Composition; The model input end of the first prediction model is used for receiving the SEM image X 0 , and the first model output end and the second model output end are used for outputting the corresponding edge point probability image P 1 and the corresponding center point probability image P 2 ; The input end of the trunk feature extraction network is connected with the input end of the model, and the output end of the trunk feature extraction network is connected with the input ends of the edge point prediction network and the center point prediction network respectively; the trunk feature extraction network is realized by improving a conventional U-Net framework, specifically, a convolution module between a downsampling coding network and an upsampling decoding network of the conventional U-Net framework is replaced by an ASPP module; The downsampling encoding network is used for performing four-wheel encoding downsampling processing on the SEM image X 0 to obtain four-size feature tensors X 1 、X 2 、X 3 、X 4 , sending the feature tensors X 4d of the feature tensors X 4 to the ASPP module, wherein the shape of the feature tensors X 1 、X 2 、X 3 、X 4 is C 1 ×H×W、C 2 ×(H/2)×(W/2)、C 3 ×(H/4)×(W/4)、C 4 ×(H/8)×(W/8);C 1 、C 2 、C 3 、C 4 , C 1 is 64, C 2 =2C 1 、C 3 =2C 2 、C 4 =2C 3 is default, and the shape of the feature tensors X 4d is C 4 × (H/16) × (W/16) respectively; The ASPP module is used for carrying out hole space pyramid pooling coding on the characteristic tensor X 4d to obtain a corresponding characteristic tensor Y 0 and sending the corresponding characteristic tensor Y 0 to the up-sampling decoding network, wherein the shape of the characteristic tensor Y 0 is C 4 X (H/16) X (W/16); The up-sampling decoding network is configured to perform four-wheel up-sampling decoding processing according to the feature tensor Y 0 and the feature tensor X 1 、X 2 、X 3 、X 4 to obtain a corresponding feature tensor Z 0 , and send the feature tensor Z 0 to the edge point prediction network and the center point prediction network, where a shape of the feature tensor Z 0 is C 1 ×h×w; the edge point prediction network and the center point prediction network are formed by sequentially connecting a group of corresponding 2D-CNN networks, 1D-CNN layers and Sigmoid layers, wherein the 2D-CNN networks are formed by sequentially connecting one or more 2D-CNN modules; The edge point prediction network is used for performing feature dimension reduction processing on the feature tensor Z 0 through the corresponding 2D-CNN network to obtain a feature tensor Z 1 ,C 5 with the shape of C 5 xH xW as a preset fifth feature dimension 2<C 5 ＜C 1 , wherein the feature tensor Z 1 consists of H xW first feature vectors with the vector length of C 5 , performing one-dimensional convolution operation on each first feature vector of the feature tensor Z 1 through the corresponding 1D-CNN layer to obtain a corresponding first feature scalar, forming a feature tensor Z 2 with the shape of 1 xH xW by the obtained H xW first feature scalar, calculating the corresponding edge point probability map P 1 according to each first feature scalar of the feature tensor Z 2 through the corresponding Sigmoid layer, and forming the corresponding edge point probability map by the obtained H xW first feature scalar; the central point prediction network is used for performing feature dimension reduction processing on the feature tensor Z 0 through the corresponding 2D-CNN network to obtain a feature tensor Z 3 with the shape of C 5 xH2xW, wherein the feature tensor Z 3 consists of H xW second feature vectors with the vector length of C 5 , performing one-dimensional convolution operation on each second feature vector of the feature tensor Z 3 through the corresponding 1D-CNN layer to obtain a corresponding second feature scalar, forming a feature tensor Z 4 with the shape of 1 xH2xW by the obtained H xW second feature scalar, calculating corresponding central point probability according to each second feature scalar of the feature tensor Z 4 through the corresponding Sigmoid layer, and forming and outputting a corresponding central point probability map P 2 by the obtained H xW central point probability; The label data set comprises a plurality of first data records, wherein the first data records comprise a first electron microscope image, a first label image and a second label image, the first electron microscope image is a fiber sample scanning electron microscope image, the first label image and the second label image are corresponding edge point probability label images and center point probability label images respectively, image data formats of the first electron microscope image, the first label image and the second label image are consistent with corresponding SEM images X 0 , the edge point probability image P 1 and the center point probability image P 2 , the probability of the edge line site corresponding to an edge line site and a background point on the first label image is 1 and 0 respectively, and the probability of the center line site corresponding to a center line site and a background point on the second label image is 1 and 0 respectively.
  3. 3. The method for processing fiber diameter measurement based on scanning electron microscope according to claim 2, wherein, The downsampling coding network is formed by sequentially connecting a first coding layer, a first downsampling layer, a second coding layer, a second downsampling layer, a third coding layer, a third downsampling layer, a fourth coding layer and a fourth downsampling layer, wherein the coding and downsampling layers are consistent with the coding and downsampling layer model structures of the conventional U-Net framework; The up-sampling decoding network is formed by sequentially connecting a first up-sampling layer, a first decoding layer, a second up-sampling layer, a second decoding layer, a third up-sampling layer, a third decoding layer, a fourth up-sampling layer and a fourth decoding layer, wherein a connecting module is embedded between each up-sampling layer and the corresponding decoding layer and used for carrying out characteristic channel connection on the corresponding coding characteristic tensor and the up-sampling characteristic tensor; The first encoding layer is configured to perform feature encoding on the SEM image X 0 to obtain a corresponding feature tensor X 1 , and send the feature tensor X 1 to the first downsampling layer and a fourth connection module between the fourth upsampling layer and the fourth decoding layer; The first downsampling layer is configured to downsample the feature tensor X 1 to obtain a corresponding feature tensor X 1d , and send the feature tensor X 1d to the second coding layer, where the feature tensor X 1d has a shape of C 1 × (H/2) × (W/2); The second encoding layer is configured to perform feature encoding on the feature tensor X 1d to obtain a corresponding feature tensor X 2 , and send the feature tensor X 2 to the second downsampling layer and a third connection module between the third upsampling layer and the third decoding layer; the second downsampling layer is configured to downsample the feature tensor X 2 to obtain a corresponding feature tensor X 2d , and send the feature tensor X 2d to the third coding layer, where the feature tensor X 2d has a shape of C 2 × (H/4) × (W/4); the third encoding layer is configured to perform feature encoding on the feature tensor X 2d to obtain a corresponding feature tensor X 3 , and send the feature tensor X 3 to the third downsampling layer and a second connection module between the second upsampling layer and the second decoding layer; The third downsampling layer is configured to downsample the feature tensor X 3 to obtain a corresponding feature tensor X 3d , and send the feature tensor X 3d to the fourth coding layer, where the feature tensor X 3d has a shape of C 3 × (H/8) × (W/8); the fourth encoding layer is configured to perform feature encoding on the feature tensor X 3d to obtain a corresponding feature tensor X 4 , and send the feature tensor X 4 to the fourth downsampling layer and a first connection module between the first upsampling layer and the first decoding layer; The fourth downsampling layer is configured to downsample the feature tensor X 4 to obtain a corresponding feature tensor X 4d , and send the feature tensor X 4d to the ASPP module; The ASPP module is used for carrying out hole space pyramid pooling coding on the characteristic tensor X 4d to obtain the characteristic tensor Y 0 and sending the characteristic tensor Y 0 to the first upsampling layer; the first upsampling layer is configured to upsample the feature tensor Y 0 to obtain a corresponding feature tensor Y 0u , and send the feature tensor Y 0u to the first connection module, where a shape of the feature tensor Y 0u is C 4 × (H/8) × (W/8); The first connection module is configured to perform feature channel connection on the feature tensor Y 0u and the feature tensor X 4 to obtain a corresponding feature tensor Y 0c , and send the corresponding feature tensor Y 0c to the first decoding layer, where a shape of the feature tensor Y 0c is 2C 4 × (H/8) × (W/8); the first decoding layer is configured to perform feature decoding on the feature tensor Y 0c to obtain a corresponding feature tensor Y 1 , and send the feature tensor Y 1 to the second upsampling layer, where a shape of the feature tensor Y 1 is C 3 × (H/8) × (W/8); the second upsampling layer is configured to upsample the feature tensor Y 1 to obtain a corresponding feature tensor Y 1u , and send the feature tensor Y 1u to the second connection module, where a shape of the feature tensor Y 1u is C 3 × (H/4) × (W/4); The second connection module is configured to perform feature channel connection on the feature tensor Y 1u and the feature tensor X 3 to obtain a corresponding feature tensor Y 1c , and send the corresponding feature tensor Y 1c to the second decoding layer, where a shape of the feature tensor Y 1c is 2C 3 × (H/4) × (W/4); The second decoding layer is configured to perform feature decoding on the feature tensor Y 1c to obtain a corresponding feature tensor Y 2 , and send the feature tensor Y 2 to the third upsampling layer, where a shape of the feature tensor Y 2 is C 2 × (H/4) × (W/4); The third upsampling layer is configured to upsample the feature tensor Y 2 to obtain a corresponding feature tensor Y 2u , and send the feature tensor Y 2u to the third connection module, where a shape of the feature tensor Y 2u is C 2 × (H/2) × (W/2); The third connection module is configured to perform feature channel connection on the feature tensor Y 2u and the feature tensor X 2 to obtain a corresponding feature tensor Y 2c , and send the corresponding feature tensor Y 2c to the third decoding layer, where a shape of the feature tensor Y 2c is 2C 2 × (H/2) × (W/2); the third decoding layer is configured to perform feature decoding on the feature tensor Y 2c to obtain a corresponding feature tensor Y 3 , and send the feature tensor Y 3 to the fourth upsampling layer, where a shape of the feature tensor Y 3 is C 1 × (H/2) × (W/2); The fourth upsampling layer is configured to upsample the feature tensor Y 3 to obtain a corresponding feature tensor Y 3u , and send the feature tensor Y 3u to the fourth connection module, where a shape of the feature tensor Y 3u is C 1 ×h×w; The fourth connection module is configured to perform feature channel connection on the feature tensor Y 3u and the feature tensor X 1 to obtain a corresponding feature tensor Y 3c , and send the corresponding feature tensor Y 3c to the fourth decoding layer, where a shape of the feature tensor Y 3c is 2c 1 ×h×w; And the fourth decoding layer is configured to perform feature decoding on the feature tensor Y 3c to obtain a corresponding feature tensor Z 0 , and send the feature tensor Z 0 to the edge point prediction network and the center point prediction network.
  4. 4. The method for processing fiber diameter measurement based on scanning electron microscope according to claim 2, wherein the training the first prediction model based on the preset label data set specifically comprises: Step 41, dividing the label data set into two sub data sets based on a preset first dividing ratio, and marking the sub data sets as a corresponding first training set and a first evaluation set; The first training set and the first evaluation set are composed of a plurality of first data records, wherein the total record ratio of the first training set and the first evaluation set meets the first segmentation ratio; Step 42, taking a first one of the first data records of the first training set as a current training record; Step 43, using the first electron microscope image, the first label image and the second label image of the current training record as the corresponding SEM image X 0 and label probability image 、 Inputting the SEM image X 0 into the first prediction model for processing, and recording the edge point probability image P 1 and the center point probability image P 2 obtained by the processing as corresponding prediction probability images 、 ; The tag probability map 、 Respectively by H x W probabilities 、 Composition of the predictive probability map 、 Respectively by H x W probabilities 、 Composition; step 44, for the tag probability map Performing morphological expansion to obtain a corresponding expansion probability map M 0 , and performing morphological expansion on the expansion probability map M 0 and the label probability map Performing feature clipping on the difference value graph of the edge neighborhood probability graph M 1 to obtain a corresponding edge neighborhood probability graph M 1 , and calculating a corresponding weight graph W based on the edge neighborhood probability graph M 1 ; The calculation modes of the expansion probability map M 0 , the edge neighborhood probability map M 1 and the weight map W are as follows: , , ; MaxPool (P, kernel_size, stride, padding) is a processing function that maximally pools the map P based on a pooling core size parameter, kernel_size, a stride parameter stride, a bit-pattern-of-complement parameter padding; Clip (P, d min ,d max ) is a feature clipping function for constraining the pixel feature scalar P i,j of map P to a set range of values P min ,p max , P min 、p max is the minimum and maximum boundary values of the numerical range, and p min ＜p max is more than or equal to 0 and less than or equal to 1; scalar is the pixel characteristic after clipping; For the expansion probability map M 0 and the label probability map I is H multiplied by W full 1 tensor, omega is a preset edge weight coefficient, and the weight graph W consists of H multiplied by W weights W i,j ; Step 45, for predictive probability map Performing morphological expansion and corrosion calculation to obtain a corresponding expansion probability map M 2 and a corrosion probability map M 3 , and calculating a difference map M 4 of the expansion probability map M 2 and the corrosion probability map M 3 ; The calculation modes of the expansion probability map M 2 , the corrosion probability map M 3 and the difference map M 4 are as follows: , , ; the difference map M 4 consists of h×w probability differences M i,j ; Step 46, mapping the tag probability map 、 The predictive probability map 、 The weight map W and the difference map M 4 are brought into a preset model loss function L total to calculate a corresponding first loss value; The model loss function L total is composed of edge focusing loss L edge , center line loss L cen and center line topology loss L top , and specifically includes: ; , ; ; 、 、 alpha and gamma are two preset focus loss super parameters, L BCE 、L Dice is a two-class cross entropy loss function and a Dice loss function; step 47, identifying whether the first loss value meets a preset first loss value range, if so, identifying whether the current training record is the last first data record of the first training set, if so, turning to step 48, if not, taking the next first data record of the first training set as a new current training record and returning to step 43, and if not, carrying out one-round optimization on model parameters of the first prediction model based on a preset first model optimizer towards a direction enabling the model loss function L total to reach a minimum value, and returning to step 43 when the one-round optimization is finished; The first model optimizer comprises an Adam optimizer and an SGD optimizer; Step 48, taking each first data record of the first evaluation set as a current evaluation record, taking the first electron microscope image of the current evaluation record as a corresponding SEM image X 0 to input the first prediction model for processing, forming a corresponding first prediction-label pair by the edge point probability image P 1 obtained by the processing and the first label image of the current evaluation record, forming a corresponding second prediction-label pair by the center point probability image P 2 obtained by the processing and the second label image of the current evaluation record, calculating a corresponding first F1 score based on all the obtained first prediction-label pairs, and calculating a corresponding second F1 score based on all the obtained second prediction-label pairs; Step 49, identifying a first F1 score and a second F1 score, returning to step 41 if the first F1 score does not meet a preset first F1 score range or the second F1 score does not meet a preset second F1 score range, and stopping training and confirming that model training of the first prediction model is finished if the first F1 score meets the first F1 score range and the second F1 score meets the second F1 score range.
  5. 5. The method for measuring fiber diameter based on scanning electron microscope according to claim 2, wherein the extracting unique paths of each fiber according to the center point probability map P 2 by using a graph theory algorithm to obtain a plurality of fiber centerlines S comprises: Step 51, performing binary image conversion on the center point probability image P 2 based on a preset center point probability threshold P cen to obtain a corresponding binary image B 0 ; Wherein the binary pattern B 0 is composed of H W binarized pixel features Composition of the pixel characteristics Is 1 or 0; The pixel features The characteristic value setting mode is as follows: ; Step 52, refining the single-pixel skeleton map on the central line of the binary map B 0 based on a preset morphological refinement algorithm to obtain a corresponding binary map B 1 ; wherein the morphological refining algorithm at least comprises a Zhang-Suen refining algorithm, a Guo-Hall refining algorithm and a Morphological-Thinning refining algorithm; The binary image B 1 is composed of H W binarized pixel features Composition of the pixel characteristics Is 1 or 0; Step 53, constructing a corresponding undirected graph G based on the binary graph B 1 , specifically: based on the pixel feature with each feature value of 1 in the binary image B 1 Setting a corresponding node V, performing one round of traversal on all the nodes V, taking the node V currently traversed as a current node in the round of traversal, taking an eight-neighborhood pixel space of the current node in the binary image B 1 as a current neighborhood space, identifying whether other pixel points corresponding to the node V exist in the current neighborhood space, if so, creating a corresponding node edge E for the current node and each other node V in the current neighborhood space, and forming a corresponding node set V by all the obtained nodes V when the round of traversal is finished, performing de-duplication processing on all the obtained node edges E, forming a corresponding edge set E by all the de-duplicated node edges E, and forming a corresponding undirected image G by the node set V and the edge set E; Wherein the eight neighborhood pixel space comprises eight directions of up, down, left, right, left up, right down and left down; Step 54, performing path endpoint pair identification based on the node set V to obtain a corresponding endpoint pair set, specifically: Step 541, marking all nodes V with the degree of 1 in the node set V as path end points, and forming a corresponding end point set by all the path end points; Wherein each of said path endpoints is connected to only one of said nodes v; Step 542, identifying whether the total number of path endpoints of the current endpoint set is greater than or equal to 2, if yes, going to step 543, otherwise going to step 544; Step 543, calculating a path distance of the shortest communication path between every two path endpoints in the current endpoint set to obtain a corresponding point pair distance, taking the longest point pair distance as the current distance, identifying whether the current distance is 0, if so, turning to step 544, if not, forming a corresponding first endpoint pair by two path endpoints corresponding to the current distance, removing the two path endpoints of the current first endpoint pair from the current endpoint set, and returning to step 542; Step 544, forming the endpoint pair set from all the obtained first endpoint pairs; the first endpoint pair set consists of a plurality of first endpoint pairs, wherein each first endpoint pair consists of two path endpoints; And step 55, taking a path curve formed by the shortest communication path of each first endpoint pair in the center point probability map P 2 as a corresponding first curve, and sampling each first curve equidistantly from the initial position to the end position according to a preset unit curve distance to obtain a corresponding fiber center line S.
  6. 6. The method for measuring fiber diameter based on a scanning electron microscope according to claim 2, wherein the identifying the positive and negative normal direction radius R k+ 、r k- of the center point S k of each fiber center line S according to the edge point probability map P 1 to obtain the corresponding radius sequence R specifically comprises: Step 61, taking each fiber central line S as a current central line, and taking each central point S k of the current central line as a current central point; step 62, sampling the normal line of the current center point according to a preset sampling interval Δr and a maximum sampling distance r hold to obtain a corresponding sampling point sequence { r g }, specifically: The tangent and normal of the current central line are processed on the current central point to obtain a corresponding tangent vector t k and a normal vector n k , and the sampling is respectively carried out on the positive and negative normal directions according to the sampling interval Deltar and the maximum sampling distance r hold to obtain 2 (r hold /Deltar) +1 sampling points r g to form a corresponding sampling point sequence { r g }; The maximum sampling distance r hold is an integer multiple of the sampling interval Deltar, the value range of the index g is [ -r hold /△r,r hold /Deltar ], and the index g corresponding to the current center point is 0; Step 63, according to bilinear interpolation algorithm, according to the edge point probability map P 1 , sampling point probabilities of the sampling points r g of the sampling point sequence { r g }, respectively Calculating and deriving from all said sampling point probabilities Composition of corresponding sample point probability sequence { The specific steps are: The sampling points r g are used as the current sampling points, the sub-pixel coordinates (i g ,j g ) of the current sampling points are identified, and the four edge line site probabilities that the pixel coordinates are closest to the current sub-pixel coordinates (i g ,j g ) on the edge point probability map P 1 Marking as the corresponding edge line site probability 、 、 、 And according to bilinear interpolation algorithm, according to the edge line locus probability 、 、 、 Probability of the sampling point corresponding to the current sampling point Calculated and the probability of the obtained 2 (r hold /DELTAr) +1 sampling points is calculated Composing the corresponding sampling point probability sequence { }; The four sets of coordinates (i a ,j a )、(i b ,j b )、(i c ,j c )、(i d ,j d ) are calculated in the following manner: i a =floor(i g ),j a =floor(j g ), i b =floor(i g ),j b =floor(j g +1), i c =floor(i g +1),j c =floor(j g +1), i d =floor(i g +1),j d =floor(j g ); floor is a downward rounding function; step 64, based on the sampling point probability sequence { Identifying the radius of the positive and negative normal direction corresponding to the current center point to obtain a corresponding radius pair rc k (r k+ ,r k- ), specifically: Based on the sampling point probability sequence { Performing curve fitting to obtain a corresponding probability curve, taking a curve point corresponding to the current center point on the probability curve as a current reference point, taking the nearest curve peak points on the left side and the right side of the current reference point on the probability curve as corresponding left and right peak probabilities, identifying sub-pixel coordinates (i Left ,j Left )、(i Right ,j Right ) corresponding to the left and right peak probabilities, calculating the linear distance from the current center point to the sub-pixel coordinates (i Left ,j Left ) and the sub-pixel coordinates (i Right ,j Right ), and taking the calculation result as a group of corresponding radius pairs rc k (r k+ ,r k- consisting of a corresponding negative normal direction radius r k- and a positive normal direction radius r k+ ; Step 65, forming a corresponding radius sequence R by all the radius pairs rc k (r k+ ,r k- ) corresponding to the current center line.
  7. 7. The method for measuring fiber diameter based on scanning electron microscope according to claim 6, wherein the sub-pixel refinement method driven according to the energy function optimizes the center line diameter of each radius sequence R to obtain a corresponding diameter sequence D, specifically comprising: Step 71, taking each radius sequence R as a current radius sequence, taking the fiber center line S corresponding to the current radius sequence as a current center line, and counting the total number K of center points of the current center line; The current radius sequence consists of K radius pairs rc k (r k+ ,r k- ), the current central line consists of K central points s k , K is more than or equal to 1 and less than or equal to K, and the central points s k are in one-to-one correspondence with the radius pairs rc k (r k+ ,r k- ); step 72) setting corresponding trimming variables for each of the radius pairs rc k (r k+ ,r k- ) 、 And setting the corresponding distance 、 And will move from each of the center points s k by a distance in the normal direction The obtained point sub-pixel sitting marks are corresponding coordinates Distance of movement along its negative normal direction The obtained point sub-pixel sitting marks are corresponding coordinates And the pixel coordinates and the coordinates on the edge point probability map P 1 Probability of four edge line sites nearest to each other Forming a corresponding first probability group, pixel coordinates and the coordinates Probability of four edge line sites nearest to each other Forming a corresponding second probability group, and calculating the coordinates according to the first probability group by bilinear interpolation algorithm The probability variable of (2) is recorded as The coordinates to be calculated according to the second probability group by bilinear interpolation algorithm The probability variable of (2) is recorded as And coordinate each set To the point of Is marked as the corresponding corrected normal vector ; Step 73, based on K sets of probability variables 、 Setting a corresponding data item energy function E data and based on K sets of the normal vector n k and the corrected normal vector Setting the energy function E orth of the corresponding orthogonal constraint term and based on the K groups of distances 、 Setting a corresponding symmetrical constraint term energy function E sym and an axial smooth constraint term energy function E smooth , and setting a corresponding integral energy function E total based on the four energy functions; Wherein, the expression of the data item energy function E data is: ; The expression of the orthogonal constraint term energy function E orth is: ; The expression of the symmetrical constraint term energy function E sym is: ; the expression of the axial smoothness constraint term energy function E smooth is as follows: ; The expression of the integral energy function E total is: ; 、 、 、 The method comprises the steps of obtaining preset data item weight coefficients, orthogonal constraint item weight coefficients, symmetrical constraint item weight coefficients and axial smoothness constraint item weight coefficients; Step 74, targeting the overall energy function E total to a minimum, trimming the variables for K sets 、 Solving to obtain corresponding K groups of fine tuning quantities 、 And setting the diameters d k corresponding to the K groups based on the solving result, And composing the corresponding diameter sequence D by all the diameters D k obtained.
  8. 8. The method for processing fiber diameter measurement based on a scanning electron microscope according to claim 1, wherein the performing abnormal diameter rejection processing based on the global statistical distribution characteristics of each diameter sequence D to obtain a corresponding diameter sequence D opt specifically comprises: Step 81, based on a histogram main peak locking mode, performing abnormal diameter rejection processing on each diameter sequence D to obtain a corresponding first diameter sequence, specifically: The diameter sequence D is used as a current diameter sequence, a diameter histogram is built based on all diameters D k of the current diameter sequence, the maximum peak value D peak of the diameter histogram is identified, the diameter standard deviation sigma d of the current diameter sequence is calculated, a corresponding main peak interval is set based on the maximum peak value D peak and the diameter standard deviation sigma d , the diameter D k which is not in the main peak interval in the current diameter sequence is removed, and the current diameter sequence with the diameter removed is used as the corresponding first diameter sequence; wherein the main peak interval is , The first diameter sequence consists of a plurality of diameters d k1 , wherein the index k1 is a positive integer and is counted from 1 and sequentially added with 1; Step 82, based on the median absolute deviation, performing abnormal diameter rejection processing on each first diameter sequence to obtain a corresponding second diameter sequence, specifically: Taking each first diameter sequence as a current diameter sequence, calculating the median of the current diameter sequence as a first median, calculating the first absolute difference between each diameter d k1 of the current diameter sequence and the first median, forming a first absolute difference sequence by all the obtained first absolute differences, calculating the median of the first absolute difference sequence as a second median, and calculating the first absolute difference sequence based on a preset first median coefficient The first threshold value corresponding to the second median is calculated, the diameter d k1 with the first absolute difference larger than the first threshold value is eliminated, and the current diameter sequence with the diameter eliminated is used as the corresponding second diameter sequence; the first median, the first absolute difference, the second median and the first threshold are calculated in the following ways: first median=median (first diameter sequence d k1 ), First absolute difference= |d k1 -first median|, Second Median = Median (first absolute difference sequence first absolute difference), First threshold value = X second median; median is a Median calculation function; The second diameter sequence consists of a plurality of diameters d k2 , wherein the index k2 is a positive integer and is counted from 1 and sequentially added with 1; Step 83, based on the relative change rate of the adjacent diameters, performing abnormal diameter rejection processing on each second diameter sequence to obtain a corresponding third diameter sequence, specifically: Taking each second diameter sequence as a current diameter sequence, calculating relative change rates a k2 of every two adjacent diameters d k2 、d k2+1 in the current diameter sequence, forming a relative change rate sequence by all obtained relative change rates a k2 , calculating the median of the relative change rate sequence to obtain a third median, calculating the second absolute differences of each relative change rate a k2 and the third median, forming a second absolute difference sequence by all obtained second absolute differences, calculating the median of the second absolute difference sequence to obtain a fourth median, and calculating the third median based on a preset second median coefficient A step of The third median and the fourth median calculate the corresponding second threshold, count the total number N K2 of diameters of the second diameter sequence, and take the relative change rate a k2 of each diameter larger than the second threshold as an abnormal change rate, and make one round of traversal on all abnormal change rates when the total number of abnormal change rates is larger than 0, take the abnormal change rate of the current traversal as a current change rate in the current round of traversal, take the index k2 corresponding to the current change rate as a current index, and identify the current index, take the diameter d k2 corresponding to the current change rate as a diameter to be eliminated if the current index is 1, take the diameter d k2+1 corresponding to the current change rate as a diameter to be eliminated if the current index is N K2 -1, take the diameter d k2+1 corresponding to the current change rate as a diameter to be eliminated if the current index is larger than or equal to 2 and smaller than N K2 -1, and identify the difference delta pre,k2 of diameters corresponding to the current change rate, Calculating and identifying a rear diameter difference Deltad nxt,k2 , marking a diameter d k2 corresponding to the current change rate as a diameter to be rejected if the front diameter difference Deltad pre,k2 is larger than the rear diameter difference Deltad nxt,k2 , marking a diameter d k2+1 corresponding to the current change rate as a diameter to be rejected if the front diameter difference Deltad pre,k2 is smaller than or equal to the rear diameter difference Deltad nxt,k2 , rejecting all the diameters to be rejected from the current diameter sequence when the round of traversal is finished, and taking the current diameter sequence with diameter rejection as the corresponding third diameter sequence; The relative change rate a k2 , the third median, the second absolute difference, the fourth median, the second threshold, the front diameter difference Δd pre,k2 , and the rear diameter difference Δd nxt,k2 are calculated in the following ways: , Third Median = Median (relative rate of change sequence a k2 ), Second absolute difference= |a k2 -third median|, Fourth Median = Median (second absolute difference sequence second absolute difference), Second threshold = third median + X the fourth median number of times, The front diameter difference Deltad pre,k2 =|d k2 -d k2-1 I, The rear diameter difference Δd nxt,k2 =|d k2+2 -d k2+1 |; The index k3 is a positive integer and is counted from 1 and sequentially added with 1; step 84, taking each diameter D k3 of the third diameter sequence as a corresponding diameter D o , composing the corresponding diameter sequence D opt from all the diameters D o , and composing a corresponding center point by row index i k and column index j k of the center point s k corresponding to each diameter D o And from each of the center points Corresponding normal direction and two diameter edge points of the diameter d o to the current center point 、 Confirmation is performed.
  9. 9. The method according to claim 1, wherein the processing of the SEM image X 0 based on the fiber center line S and the diameter sequence D opt corresponding thereto for the fiber diameter visualization includes: taking each fiber center line S as a current center line, taking the diameter sequence D opt corresponding to the current center line as a current diameter sequence, drawing corresponding center lines based on the current center line on the SEM image X 0 , and taking each diameter D o and the corresponding center point of the current diameter sequence on the current drawn center line The edge point The edge point Drawing the corresponding center point and the diameter line segment, and marking the diameter length of the currently drawn diameter line segment.
  10. 10. An apparatus for performing the processing method for measuring fiber diameter based on scanning electron microscope according to any one of claims 1 to 9, wherein the apparatus comprises a model construction module, a model training module, a model prediction module, a center line extraction module, a diameter estimation module, a diameter optimization module, a diameter noise reduction module, and a diameter visualization marking module; The model construction module is used for constructing a deep learning model for predicting fiber edges and fiber center lines of a fiber sample scanning electron microscope image and recording the deep learning model as a first prediction model, wherein the first prediction model is used for predicting fiber edges and center line points along the line of an SEM image X 0 input by the model and outputting a corresponding edge point probability image P 1 and a corresponding center point probability image P 2 , the first prediction model comprises a trunk feature extraction network, an edge point prediction network and a center point prediction network, the trunk feature extraction network is used for extracting trunk features of the SEM image X 0 to obtain a feature tensor Z 0 , and the edge point prediction network and the center point prediction network are respectively used for predicting corresponding edges and center line point probability images along the line and the center point probability images according to the feature tensor Z 0 to obtain a corresponding edge point probability image P 1 and a corresponding center point probability image P 2 ; the model training module trains the first prediction model based on a preset label data set; The model prediction module is used for receiving a scanning electron microscope image of any fiber sample as a corresponding SEM image X 0 to input the first prediction model to obtain a corresponding edge point probability image P 1 and a corresponding center point probability image P 2 after model training is finished; The central line extraction module is used for extracting a unique path of each fiber according to the central point probability map P 2 by using a graph theory algorithm to obtain a plurality of corresponding fiber central lines S, wherein the fiber central lines S are formed by sequentially sequencing a plurality of central points S k (i k ,j k ), the index k is a positive integer, and the row index and the column index corresponding to the central points S k are counted from 1 and sequentially added with 1;i k 、j k ; The diameter estimation module is used for identifying positive and negative normal direction radii R k+ 、r k- of the center points S k of each fiber center line S according to the edge point probability map P 1 to obtain a corresponding radius sequence R, wherein the radius sequence R is formed by sequentially sequencing a plurality of groups of radius pairs rc k (r k+ ,r k- ); the diameter optimization module is used for optimizing the central line diameter of each radius sequence R according to an energy function driven sub-pixel finishing method to obtain a corresponding diameter sequence D, wherein the diameter sequence D consists of a plurality of diameters D k ; The diameter noise reduction module performs abnormal diameter elimination processing based on the global statistical distribution characteristics of the diameter sequences D to obtain a corresponding diameter sequence D opt , the diameter sequence D opt is formed by sequentially sequencing a plurality of diameters D o , an index o is a positive integer, the number is counted from 1, the number is sequentially increased by 1, and each diameter D o corresponds to a group of center points Edge points Edge points ; 、 As the center point Row index and column index of (a); 、 For the edge points Row index and column index of (a); 、 For the edge points Row index and column index of (a); The diameter visual marking module performs corresponding fiber diameter visual marking processing on the SEM image X 0 based on each fiber center line S and the corresponding diameter sequence D opt .
  11. 11. An electronic device comprising a memory, a processor, and a transceiver; The processor being adapted to couple with the memory, read and execute instructions in the memory to implement the method of any one of claims 1-9; the transceiver is coupled to the processor and is controlled by the processor to transmit and receive messages.
  12. 12. A computer readable storage medium storing computer instructions which, when executed by a computer, cause the computer to perform the method of any one of claims 1-9.

Description

Processing method and device for measuring fiber diameter based on scanning electron microscope Technical Field The invention relates to the technical field of data processing, in particular to a processing method and a device for measuring fiber diameter based on a scanning electron microscope. Background Scanning electron microscope (Scanning Electron Microscope, SEM) is also called scanning electron microscope for short, which is a key tool for micro-nano scale morphology and component analysis in material science, life science and nanotechnology. The microscopic geometrical characteristics (diameter, orientation, distribution) of fibrous materials (e.g., carbon fibers, nanofibers, biological fibers, etc.) are key factors in determining their macroscopic properties. SEM images are the primary characterization of their microstructure. However, accurate extraction of fiber diameter from complex SEM images still faces many challenges, 1) fibers tend to be highly dense, multi-overlapping, with low signal-to-noise ratio, simple image processing (e.g., thresholding, canny edge detection) cannot effectively distinguish fiber edges from background noise, and also cannot handle cross occlusion problems, 2) existing deep learning methods generally consider the problem as simple "semantic segmentation" (classifying pixels as foreground or background), ignoring the topological properties (e.g., center line continuity) and geometric properties (e.g., edge normal consistency) of fibers as continuous linear entities, which can lead to fracture or sticking problems of the prediction results, and cannot be directly used for high-precision measurements, and 3) even if segmentation masks are obtained, traditional measurement methods (e.g., distance conversion) are limited by the pixel accuracy of segmentation masks, can not achieve sub-pixel level physical measurements, and can hardly guarantee that the measurement direction of fiber diameter is strictly perpendicular to the fiber axis. Disclosure of Invention The invention aims at overcoming the defects of the prior art and provides a processing method, a device, electronic equipment and a computer readable storage medium for measuring fiber diameter based on a scanning electron microscope. The invention uses a U-Net model combined with a cavity space pyramid pooling (Atrous SPATIAL PYRAMID Pooling, ASPP) module as a shared trunk feature extraction network, builds a multitask prediction model (namely a first prediction model) for synchronously predicting fiber edges and fiber center lines by butting two parallel prediction networks (edge point prediction network and center point prediction network) at the output end of the trunk network, realizes the accurate separation of crossed/adhered fibers and the reconstruction of continuous ordered center lines by the first prediction model, morphological refinement and graph theory path extraction technology, solves the problems that the traditional image processing method cannot effectively distinguish fiber edges from background noise and is difficult to process crossed shielding, ensures that the predicted fiber edges and center lines can further meet the topological and geometric characteristics of continuous, slender and normal consistency on the basis of pixel division by adding an edge focusing loss and center line topological loss into the model loss, setting an orthogonal constraint term in an energy function, and an axial smoothing constraint term, further reduces the fracture and adhesion phenomenon of a prediction result, and further combines the accurate interpolation method with a sub-pixel correction function driving sub-pixel correction method, and the diameter correction method in the optimal diameter measurement method, and the diameter measurement method can be combined with the optimal diameter measurement method in the three-dimensional direction, and the diameter measurement accuracy is further improved. To achieve the above object, a first aspect of the present invention provides a processing method for measuring a fiber diameter based on a scanning electron microscope, the method comprising: Establishing a deep learning model for predicting fiber edges and fiber center lines of a fiber sample scanning electron microscope image as a first prediction model, wherein the first prediction model is used for predicting fiber edges along lines and center line points of an SEM image X 0 input by the model and outputting a corresponding edge point probability image P 1 and a corresponding center point probability image P 2, the first prediction model comprises a trunk feature extraction network, an edge point prediction network and a center point prediction network, the trunk feature extraction network is used for extracting trunk features of the SEM image X 0 to obtain a feature tensor Z 0, and the edge point prediction network and the center point prediction network are respectively used for predicting correspondi