CN-114332530-B - Image classification method, device, computer equipment and storage medium

Abstract

The embodiment of the application discloses an image classification method, an image classification device, computer equipment and a storage medium, and belongs to the technical field of computers. The method comprises the steps of obtaining image features of pathological images to be classified, extracting at least one local feature corresponding to each scale from the image features for each scale in a plurality of scales, polymerizing the at least one local feature corresponding to each scale to obtain an aggregated feature, splicing the obtained plurality of aggregated features to obtain spliced image features, and classifying the spliced image features to obtain the category of the pathological images. According to the method provided by the embodiment of the application, the information contained in the local features corresponding to different scales is different, so that the aggregation features contain the feature information corresponding to different scales, the feature information of the spliced image features is enriched after the aggregation features are spliced into the spliced image features, and the category to which the pathological image belongs is determined based on the spliced image features, so that the accuracy of the category is ensured.

Inventors

• ZHAO YU
• LIN ZHENYU
• YAO JIANHUA

Assignees

• 腾讯科技（深圳）有限公司

Dates

Publication Date

20260505

Application Date

20211222

Claims (20)

1. 1. A method of classifying images, the method comprising: acquiring image features of pathological images to be classified; For each scale in a plurality of scales, extracting at least one local feature corresponding to the scale from the image feature, and aggregating the at least one local feature corresponding to the scale to obtain an aggregated feature; splicing the acquired multiple aggregation features to acquire spliced image features; Classifying the spliced image features to obtain the category to which the pathological image belongs; The aggregation of the at least one local feature corresponding to the scale to obtain an aggregated feature comprises the following steps: respectively splicing the second sub-features in each extracted local feature to obtain a first feature vector corresponding to each local feature; Based on first feature vectors corresponding to a plurality of local features, updating each first feature vector to obtain second feature vectors corresponding to each first feature vector, wherein the second feature vectors corresponding to each first feature vector are integrated with the first feature vectors corresponding to other local features; A plurality of second feature vectors are formed into the aggregated feature based on the locations of the plurality of local features in the image feature.
2. 2. The method of claim 1, wherein the stitched image features comprise first sub-features located at a plurality of locations, and wherein classifying the stitched image features to obtain a category to which the pathology image belongs comprises: Based on a plurality of first sub-features, updating each first sub-feature to obtain a first updated feature corresponding to each first sub-feature; based on the positions of the plurality of first sub-features in the spliced image feature, forming an updated spliced image feature by the first updated features corresponding to the plurality of first sub-features; and classifying the updated spliced image features to obtain the category to which the pathological image belongs.
3. 3. The method according to claim 2, wherein updating each first sub-feature based on the plurality of first sub-features to obtain a first updated feature corresponding to each first sub-feature includes: for a target sub-feature in the plurality of first sub-features, acquiring weights of the plurality of first sub-features, wherein the weights indicate the association degree between the corresponding first sub-feature and the target sub-feature; and based on the weights of the plurality of first sub-features, carrying out weighted fusion on the plurality of first sub-features to obtain a first updated feature corresponding to the target sub-feature.
4. 4. The method of claim 3, wherein the obtaining weights for the plurality of first sub-features for a target sub-feature of the plurality of first sub-features comprises: For a target sub-feature in the plurality of first sub-features, acquiring a distance feature between each first sub-feature and the target sub-feature, wherein the distance feature indicates a distance between a position where the first sub-feature and the target sub-feature are located in the spliced image feature; and acquiring the weight of each first sub-feature based on the similarity between each first sub-feature and the target sub-feature and the distance feature.
5. 5. The method of claim 4, wherein the obtaining distance features between each of the first sub-features and the target sub-feature for a target sub-feature of the plurality of first sub-features comprises: Determining a target distance corresponding to each first sub-feature based on the positions of each first sub-feature and the target sub-feature in the spliced image feature, wherein the target distance represents a distance between the first sub-feature and the position of the target sub-feature; And mapping the target distance corresponding to each first sub-feature to obtain the distance feature between each first sub-feature and the target sub-feature.
6. 6. The method of claim 2, wherein the first updated feature is a vector, wherein the constructing the updated stitched image feature from the first updated features corresponding to the first plurality of sub-features based on the locations of the first plurality of sub-features in the stitched image feature comprises: And based on the positions of the plurality of first sub-features in the spliced image features, forming a three-dimensional feature matrix by the first updated features corresponding to the plurality of first sub-features, and determining the three-dimensional feature matrix as the updated spliced image features.
7. 7. The method of claim 1, wherein the stitched image features comprise first sub-features located at a plurality of locations, and wherein the classifying the stitched image features results in the pathology image belonging to a category, the method further comprising: fusing each first sub-feature with a corresponding position feature to obtain a second updated feature corresponding to each first sub-feature, wherein the position feature indicates the position of the corresponding first sub-feature in the spliced image feature; And forming updated spliced image features by using second updated features corresponding to the first sub-features based on the positions of the first sub-features in the spliced image features.
8. 8. The method of claim 7, wherein the second updated feature is a vector, wherein the constructing the updated stitched image feature from the second updated feature corresponding to the first plurality of sub-features based on the locations of the first plurality of sub-features in the stitched image feature comprises: And based on the positions of the plurality of first sub-features in the spliced image features, forming a three-dimensional feature matrix by using second updated features corresponding to the plurality of first sub-features, and determining the three-dimensional feature matrix as the updated spliced image features.
9. 9. The method of claim 1, wherein said constructing a plurality of second feature vectors into said aggregated feature comprises: And constructing a three-dimensional feature matrix by a plurality of second feature vectors, and determining the three-dimensional feature matrix as the aggregation feature.
10. 10. The method of claim 1, wherein the feature sizes of the plurality of aggregated features are the same, and the stitching the plurality of aggregated features to obtain stitched image features comprises: splicing the features in the same position in the plurality of aggregation features to obtain feature vectors corresponding to the plurality of positions; and determining a three-dimensional feature matrix formed by the feature vectors corresponding to the positions as the features of the spliced image.
11. 11. The method according to any one of claims 1-10, wherein the steps of obtaining image features of a pathology image to be classified, extracting at least one local feature corresponding to the scale from the image features for each scale of a plurality of scales, aggregating the at least one local feature corresponding to the scale to obtain an aggregate feature, stitching the obtained plurality of aggregate features to obtain stitched image features, and classifying the stitched image features to obtain a class to which the pathology image belongs are implemented based on a classification model.
12. 12. The method of claim 11, wherein the classification model includes a feature extraction sub-model and a classification sub-model, the method further comprising: Acquiring a sample pathology image and a sample label, wherein the sample label indicates the category to which the sample pathology image belongs; based on the feature extraction sub-model, obtaining sample image features of the sample pathology image; Based on the classification sub-model, extracting at least one sample local feature corresponding to each scale from the sample image features, and aggregating the at least one sample local feature corresponding to the scale to obtain sample aggregate features; classifying the sample spliced image features to obtain a prediction label of the sample pathological image, wherein the prediction label indicates the category of the predicted sample pathological image; Training the classification model based on the predictive label and the sample label.
13. 13. An image classification apparatus, the apparatus comprising: the acquisition module is used for acquiring image features of the pathological images to be classified; The aggregation module is used for extracting at least one local feature corresponding to each scale from the image features, and aggregating the at least one local feature corresponding to the scale to obtain an aggregated feature; the splicing module is used for splicing the acquired multiple aggregation features to acquire spliced image features; the classification module is used for classifying the spliced image features to obtain the category to which the pathological image belongs; the local feature comprises at least one second sub-feature located in at least one position, and the aggregation module is used for: respectively splicing the second sub-features in each extracted local feature to obtain a first feature vector corresponding to each local feature; Based on first feature vectors corresponding to a plurality of local features, updating each first feature vector to obtain second feature vectors corresponding to each first feature vector, wherein the second feature vectors corresponding to each first feature vector are integrated with the first feature vectors corresponding to other local features; A plurality of second feature vectors are formed into the aggregated feature based on the locations of the plurality of local features in the image feature.
14. 14. The apparatus of claim 13, wherein the stitched image features comprise first sub-features located at a plurality of locations, and wherein the classification module comprises: the updating unit is used for updating each first sub-feature based on the plurality of first sub-features to obtain a first updated feature corresponding to each first sub-feature; The construction unit is used for constructing the updated spliced image feature by the first updated features corresponding to the plurality of first sub-features based on the positions of the plurality of first sub-features in the spliced image feature; And the classification unit is used for classifying the updated spliced image features to obtain the category to which the pathological image belongs.
15. 15. The apparatus of claim 14, wherein the updating unit is configured to: for a target sub-feature in the plurality of first sub-features, acquiring weights of the plurality of first sub-features, wherein the weights indicate the association degree between the corresponding first sub-feature and the target sub-feature; and based on the weights of the plurality of first sub-features, carrying out weighted fusion on the plurality of first sub-features to obtain a first updated feature corresponding to the target sub-feature.
16. 16. The apparatus of claim 15, wherein the updating unit is configured to: For a target sub-feature in the plurality of first sub-features, acquiring a distance feature between each first sub-feature and the target sub-feature, wherein the distance feature indicates a distance between a position where the first sub-feature and the target sub-feature are located in the spliced image feature; and acquiring the weight of each first sub-feature based on the similarity between each first sub-feature and the target sub-feature and the distance feature.
17. 17. The apparatus of claim 16, wherein the updating unit is configured to: Determining a target distance corresponding to each first sub-feature based on the positions of each first sub-feature and the target sub-feature in the spliced image feature, wherein the target distance represents a distance between the first sub-feature and the position of the target sub-feature; And mapping the target distance corresponding to each first sub-feature to obtain the distance feature between each first sub-feature and the target sub-feature.
18. 18. The apparatus of claim 14, wherein the first update feature is a vector, and wherein the means for constructing is configured to: And based on the positions of the plurality of first sub-features in the spliced image features, forming a three-dimensional feature matrix by the first updated features corresponding to the plurality of first sub-features, and determining the three-dimensional feature matrix as the updated spliced image features.
19. 19. The apparatus of claim 13, wherein the stitched image feature comprises a first sub-feature located at a plurality of positions, the apparatus further comprising: The fusion module is used for fusing each first sub-feature with a corresponding position feature to obtain a second updated feature corresponding to each first sub-feature, wherein the position feature indicates the position of the corresponding first sub-feature in the spliced image feature; and the construction module is used for constructing the updated spliced image characteristic by the second updated characteristic corresponding to the first sub-characteristics based on the positions of the first sub-characteristics in the spliced image characteristic.
20. 20. The apparatus of claim 19, wherein the second updated feature is a vector, and wherein the means for constructing constructs a three-dimensional feature matrix from second updated features corresponding to the first sub-features based on where the first sub-features are located in the stitched image feature, and determines the three-dimensional feature matrix as the updated stitched image feature.

Description

Image classification method, device, computer equipment and storage medium Technical Field The embodiment of the application relates to the technical field of computers, in particular to an image classification method, an image classification device, computer equipment and a storage medium. Background With the development of computer technology, the image classification technology is more and more widely applied, and can be applied to various scenes, such as face recognition scenes or medical image classification scenes. In the related art, feature extraction is performed on an image to be classified to obtain image features of the image, and the image features of the image are directly classified to obtain the category to which the image belongs. The method directly classifies the extracted image features, so that the information quantity in the image features is small, and the classification accuracy is poor. Disclosure of Invention The embodiment of the application provides an image classification method, an image classification device, computer equipment and a storage medium, which can improve classification accuracy. The technical scheme is as follows: in one aspect, there is provided an image classification method, the method comprising: acquiring image features of pathological images to be classified; For each scale in a plurality of scales, extracting at least one local feature corresponding to the scale from the image feature, and aggregating the at least one local feature corresponding to the scale to obtain an aggregated feature; splicing the acquired multiple aggregation features to acquire spliced image features; and classifying the spliced image features to obtain the category to which the pathological image belongs. In one possible implementation manner, the acquiring the image features of the pathological image to be classified includes: dividing the pathological image to obtain a plurality of sub-images; extracting the characteristics of each sub-image to obtain the image characteristics of each sub-image; And based on the positions of the plurality of sub-images in the pathological image, splicing the image features of the plurality of sub-images to obtain the image features of the pathological image. In another aspect, there is provided an image classification apparatus, the apparatus comprising: the acquisition module is used for acquiring image features of the pathological images to be classified; The aggregation module is used for extracting at least one local feature corresponding to each scale from the image features, and aggregating the at least one local feature corresponding to the scale to obtain an aggregated feature; the splicing module is used for splicing the acquired multiple aggregation features to acquire spliced image features; And the classification module is used for classifying the spliced image features to obtain the category to which the pathological image belongs. In one possible implementation, the stitched image feature comprises a first sub-feature located at a plurality of positions, and the classification module comprises: the updating unit is used for updating each first sub-feature based on the plurality of first sub-features to obtain a first updated feature corresponding to each first sub-feature; The construction unit is used for constructing the updated spliced image feature by the first updated features corresponding to the plurality of first sub-features based on the positions of the plurality of first sub-features in the spliced image feature; And the classification unit is used for classifying the updated spliced image features to obtain the category to which the pathological image belongs. In another possible implementation manner, the updating unit is configured to obtain weights of the plurality of first sub-features for a target sub-feature in the plurality of first sub-features, where the weights indicate a degree of association between the corresponding first sub-feature and the target sub-feature, and perform weighted fusion on the plurality of first sub-features based on the weights of the plurality of first sub-features to obtain a first updated feature corresponding to the target sub-feature. In another possible implementation manner, the updating unit is configured to obtain, for a target sub-feature in the plurality of first sub-features, a distance feature between each first sub-feature and the target sub-feature, where the distance feature indicates a distance between a location where the first sub-feature and the target sub-feature are located in the stitched image feature, and obtain a weight of each first sub-feature based on a similarity between each first sub-feature and the target sub-feature and the distance feature. In another possible implementation manner, the updating unit is configured to determine, based on the positions of each first sub-feature and the target sub-feature in the stitched image feature, a target distance cor