CN-116486071-B - Image blocking feature extraction method, device and storage medium

CN116486071BCN 116486071 BCN116486071 BCN 116486071BCN-116486071-B

Abstract

The application provides an image blocking feature extraction method, an image blocking feature extraction device and a storage medium. The method comprises the steps of inputting image blocks to be processed into a graph attention network model to obtain feature vectors of the image blocks to be processed, wherein the graph attention network model is obtained through training based on the following steps of splicing a semantic segmentation graph of a sample image block with an original graph of the sample image block to obtain spliced image blocks, determining topological features of the sample image block based on the spliced image blocks by utilizing a graph attention mechanism, and training the graph attention network model based on the topological features. According to the image block feature extraction method, device and storage medium provided by the embodiment of the application, the local feature information and the global feature information of the image block to be processed can be obtained through the drawing attention network model obtained through training, so that the feature information extraction accuracy is improved.

Inventors

MA XIBO
Jie Chenlu

Assignees

中国科学院自动化研究所

Dates

Publication Date: 20260505
Application Date: 20230313

Claims (10)

1. An image blocking feature extraction method is characterized by comprising the following steps: Inputting image blocks to be processed into a graph annotation meaning network model, and obtaining feature vectors of the image blocks to be processed; the graph meaning network model is obtained based on training of the following steps: splicing the semantic segmentation map of the sample image block with the original map of the sample image block to obtain a spliced image block; determining topological features of the sample image blocks by using a graph attention mechanism based on the spliced image blocks; training a graph attention network model based on the topological feature; The step of splicing the semantic segmentation map of the sample image block with the original map of the sample image block to obtain a spliced image block comprises the following steps: Patterning the sample image blocks based on a graph neural network to obtain a semantic segmentation graph of the sample image blocks, wherein the sample image blocks are taken as nodes, and the connection relationship among the sample image blocks is taken as edges; and splicing the semantic segmentation map of the sample image block with the original map of the sample image block to obtain a spliced image block.
2. The method for extracting image blocking features according to claim 1, wherein before the step of splicing the semantic segmentation map of the sample image block with the original map of the sample image block to obtain a spliced image block, the method further comprises: Performing semantic segmentation on the sample image blocks based on a semantic segmentation model to obtain an initial segmentation result; and determining a semantic segmentation map of the sample image segmentation by using a channel attention mechanism based on the initial segmentation result.
3. The image blocking feature extraction method according to claim 2, wherein the loss function expression of the semantic segmentation model is as follows: ; Wherein, the Representing coordinates of the pixel points; Representing the total number of categories of the pixel points; Representing the true value of the pixel point; A predicted value representing a pixel point; representing the distance of the pixel point from the boundary; A constant value is represented for preventing the denominator from being 0; representing the loss function value.
4. The image tile feature extraction method of claim 1, wherein determining topological features of the sample image tile using a graph attention network based on stitched image tiles comprises: Inputting the spliced image blocks into a residual neural network for compression to obtain node characteristics; determining the weight of the connection relation between the nodes through an attention mechanism; and determining the topological characteristic of the sample image block based on the node characteristic and the weight of the connection relation between the nodes.
5. The image blocking feature extraction method according to claim 1, wherein training a graph attention network model based on the topological feature comprises: Determining feature vectors of the sample image blocks based on topological features of the sample image blocks; And training a graph attention network model based on the feature vectors of the sample image blocks.
6. The image patch feature extraction method according to claim 5, wherein determining feature vectors of sample image patches based on topological features of the image patches, comprises: distributing the weight of adjacent nodes to each node, wherein the adjacent nodes refer to nodes adjacent to the node; And updating the feature vector of the sample image block based on the topological feature and the weight of the adjacent node.
7. The method for extracting image blocking features according to claim 1, wherein before the step of splicing the semantic segmentation map of the sample image block with the original map of the sample image block to obtain a spliced image block, the method further comprises: Acquiring a mask of a thumbnail based on the thumbnail of a full slice image by using a maximum inter-class variance method OTSU; Determining an initial image partition of the full-slice image based on the compressed full-slice image; and carrying out data enhancement on the initial image blocks based on the masks of the thumbnails to obtain the sample image blocks.
8. An image blocking feature extraction device, characterized by comprising: The feature extraction module is used for inputting the image blocks to be processed into the graph annotation meaning network model to obtain feature vectors of the image blocks to be processed; the graph meaning network model is obtained based on training of the following steps: splicing the semantic segmentation map of the sample image block with the original map of the sample image block to obtain a spliced image block; determining topological features of the sample image blocks by using a graph attention mechanism based on the spliced image blocks; training a graph attention network model based on the topological feature; The step of splicing the semantic segmentation map of the sample image block with the original map of the sample image block to obtain a spliced image block comprises the following steps: Patterning the sample image blocks based on a graph neural network to obtain a semantic segmentation graph of the sample image blocks, wherein the sample image blocks are taken as nodes, and the connection relationship among the sample image blocks is taken as edges; and splicing the semantic segmentation map of the sample image block with the original map of the sample image block to obtain a spliced image block.
9. An electronic device comprising a memory, a transceiver, and a processor; The memory is used for storing a computer program, the transceiver is used for receiving and transmitting data under the control of the processor, and the processor is used for reading the computer program in the memory and executing the image blocking feature extraction method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program for causing a computer to execute the image blocking feature extraction method according to any one of claims 1 to 7.

Description

Image blocking feature extraction method, device and storage medium Technical Field The present application relates to the field of computer vision, and in particular, to a method and apparatus for extracting image blocking features, and a storage medium. Background With the rapid development of artificial intelligence, deep learning algorithms are commonly applied in image processing methods. The convolutional neural network is used as a representative, deep characteristic information contained in the picture is obtained in an end-to-end mode through a large number of data training models with labels, and the method is well-applied in the fields of picture classification, semantic segmentation and the like. For ultra-high resolution images containing an ultra-large data volume, such as the processing of full-slice images (white SLIDE IMAGE, WSI), which are high-magnification large-scale digital images that are available for computer display, transmission and processing by a dedicated scanning imaging system, it is necessary to cut such images into image segments of suitable size and then use deep learning for feature extraction. However, after such a picture is divided into a plurality of image blocks, the difficulty in extracting useful information in the image is increased, and the extracted characteristic information is inaccurate due to the characteristic extraction method. Disclosure of Invention The embodiment of the application provides a method, a device and a storage medium for extracting image blocking features, which are used for solving the technical problem that the feature information of the image blocking is extracted inaccurately in the prior art. In a first aspect, an embodiment of the present application provides an image blocking feature extraction method, including: Inputting image blocks to be processed into a graph annotation meaning network model, and obtaining feature vectors of the image blocks to be processed; the graph meaning network model is obtained based on training of the following steps: splicing the semantic segmentation map of the sample image block with the original map of the sample image block to obtain a spliced image block; determining topological features of the sample image blocks by using a graph attention mechanism based on the spliced image blocks; And training a graph attention network model based on the topological features. In some embodiments, before the semantic segmentation map of the sample image segmentation is spliced with the original map of the sample image segmentation to obtain the spliced image segmentation, the method further includes: Performing semantic segmentation on the sample image blocks based on a semantic segmentation model to obtain an initial segmentation result; and determining a semantic segmentation map of the sample image segmentation by using a channel attention mechanism based on the initial segmentation result. In some embodiments, the loss function expression of the semantic segmentation model is as follows: Wherein x and Y represent coordinates of the pixel point, c represents a category of the pixel point, and Y i represents a true value of the pixel point; the prediction value of the pixel point is represented, the distance from the boundary of the pixel point is represented by D x,y, the constant value is represented by ρ, and the loss function value is represented by 0;L. In some embodiments, determining topological features of the sample image patches using a graph attention network based on stitched image patches includes: Inputting the spliced image blocks into a residual neural network for compression to obtain node characteristics; determining the weight of the connection relation between the nodes through an attention mechanism; and determining the topological characteristic of the sample image block based on the node characteristic and the weight of the connection relation between the nodes. In some embodiments, training a graph attention network model based on the topological features includes: Determining feature vectors of the sample image blocks based on topological features of the sample image blocks; And training a graph attention network model based on the feature vectors of the sample image blocks. In some embodiments, determining feature vectors for sample image patches based on topological features of the image patches includes: distributing the weight of adjacent nodes to each node, wherein the adjacent nodes refer to nodes adjacent to the node; And updating the feature vector of the sample image block based on the topological feature and the weight of the adjacent node. In some embodiments, before the semantic segmentation map of the sample image segmentation is spliced with the original map of the sample image segmentation to obtain the spliced image segmentation, the method further includes: Acquiring a mask of a thumbnail based on the thumbnail of a full slice image by using a maximum inter-class vari