CN-115546574-B - Image classification, model training method, apparatus, storage medium, and computer program

Abstract

The embodiment of the application discloses an image classification and model training method, equipment, a storage medium and a computer program, and belongs to the field of image processing. In the method, a target image is processed through a current neural network model to obtain a current classification result, wherein the current neural network model is a neural network model i corresponding to the maximum probability in a selection result output by a neural network model a, and the selection result comprises probabilities corresponding to p neural network models in m neural network models. And determining a current integration result based on the current classification result, and determining the category of the target image based on the current integration result. That is, a part of neural network models are selected from m neural network models according to the target image to process the target image, so that the category of the target image is determined, the classification result of each neural network model in the m neural network models is not required to be acquired, the calculation cost is greatly reduced, and the image classification efficiency is improved.

Inventors

• ZHANG YIKANG
• ZHONG ZHAO

Assignees

• 华为技术有限公司

Dates

Publication Date

20260512

Application Date

20210630

Claims (20)

1. 1. A method of classifying images, the method comprising: processing a target image to be classified through a current neural network model to obtain a current classification result output by the current neural network model; The current classification result comprises probabilities that the target image belongs to each of a plurality of categories, the current neural network model is a neural network model i corresponding to the maximum probability in selection results output by a neural network model a, the selection results comprise probabilities corresponding to p neural network models in m trained neural network models, the p neural network models are neural network models allowed to be used for processing the target image after the neural network model a, the p neural network models comprise the neural network model i, the m neural network models comprise the neural network model a, the m is an integer greater than 1, the p is an integer greater than or equal to 1, and the p is smaller than the m; Determining a current integration result based on the current classification result, wherein the current integration result comprises integrated probabilities that the target image belongs to each of the multiple categories; and determining the category of the target image based on the current integration result.
2. 2. The method of claim 1, wherein the probability of the first category in the current integrated result comprises an average of probabilities of the first category in a plurality of classification results, the first category being one of the plurality of categories, the plurality of classification results comprising a classification result output by the neural network model a and a classification result output by the neural network model i, or The probability of the first category in the current integration result comprises the probability of the first category in the classification result output by the neural network model i.
3. 3. The method of claim 1, wherein determining the category of the target image based on the current integration result comprises determining a category corresponding to a maximum probability in the current integration result as the category of the target image; Or alternatively The current neural network model further outputs a current selection result, wherein the current selection result comprises probabilities corresponding to d neural network models in the m neural network models, the d neural network models are neural network models which are allowed to be used for processing the target image after the neural network model i, d is an integer greater than or equal to 1, d is smaller than m, the classification of the target image is determined based on the current integration result, and the classification corresponding to the maximum probability in the current integration result is determined to be the classification of the target image if the current integration result meets a first convergence condition or the current neural network model meets a second convergence condition.
4. 4. The method of claim 3, wherein the current integration result satisfying the first convergence criterion includes a confidence level of the current integration result being greater than a confidence threshold.
5. 5. The method of claim 4, wherein after determining a current integration result based on the current classification result, further comprising: determining the maximum probability of the current integration result as the confidence of the current integration result, or And inputting the target image and the current integration result into a confidence coefficient network model to obtain the confidence coefficient of the current integration result output by the confidence coefficient network model.
6. 6. The method of any of claims 3-5, wherein the current neural network model satisfying the second convergence condition includes a number of iterations of the current neural network model being greater than or equal to an iteration number threshold.
7. 7. The method of any of claims 1-5, wherein prior to processing the target image to be classified by the current neural network model, further comprising: acquiring n sample images and n sample tags, wherein the n sample tags are in one-to-one correspondence with the n sample images; determining gradients of various parameters of each neural network model to be trained in the m neural network models to be trained based on the n sample images and the n sample tags; Updating the m neural network models to be trained based on gradients of all parameters of the m neural network models to be trained to obtain the trained m neural network models.
8. 8. The method of claim 7, wherein the determining a gradient of the respective parameter of each of the m neural network models to be trained based on the n sample images and the n sample tags comprises: Determining gradients of all parameters of the neural network model k to be trained in the m neural network models to be trained according to the following operation until determining the gradients of all parameters of each neural network model to be trained: determining a training function value corresponding to the neural network model k to be trained based on the n sample images and the n sample labels; and determining the gradient of each parameter of the neural network model k to be trained based on the training function value corresponding to the neural network model k to be trained.
9. 9. The method of claim 8, wherein the training function values comprise n classification function values, the n classification function values being in one-to-one correspondence with the n sample images; The determining the training function value corresponding to the neural network model k to be trained based on the n sample images and the n sample labels includes: Selecting one sample image from the n sample images, and determining a classification function value corresponding to the selected sample image according to the following operation until the classification function value corresponding to each sample image is determined: inputting the selected sample image into the neural network model k to be trained so as to obtain a classification result output by the neural network model k to be trained; And determining a classification function value corresponding to the selected sample image based on the sample label corresponding to the selected sample image and a classification result output by the neural network model k to be trained.
10. 10. The method of claim 8, wherein the training function values comprise n weight function values, the n weight function values being in one-to-one correspondence with the n sample images; The determining the training function value corresponding to the neural network model k to be trained based on the n sample images and the n sample labels includes: Determining, by b neural network models to be trained, a standard probability that each of t neural network models to be trained is used to process each of the n sample images to obtain n×t standard probabilities, t=m-b, where the b neural network models to be trained are models of the m neural network models that are allowed to process the n sample images before the neural network model k to be trained, the t neural network models to be trained include the neural network model k to be trained, and models of the m neural network models to be trained that are allowed to process the n sample images after the neural network model k to be trained; determining a first weight corresponding to each sample image in the n sample images based on the n×t standard probabilities; and carrying out normalization processing on the first weight corresponding to each sample image in the n sample images to obtain a weight function value corresponding to each sample image in the n sample images.
11. 11. The method of claim 10, wherein determining, from the b neural network models to be trained, a standard probability that each of the t neural network models to be trained is used to process each of the n sample images to obtain n x t standard probabilities comprises: determining, by the b neural network models to be trained, an average probability that each of the t neural network models to be trained is used to process each of the n sample images to obtain n×t average probabilities; Dividing the average probability corresponding to the same neural network model to be trained in the t neural network models to be trained in the n x t average probabilities into a probability set to obtain t probability sets; and based on the n×t average probabilities and the t probability sets, performing normalization processing on the n×t average probabilities to obtain n×t standard probabilities.
12. 12. The method of claim 11, wherein the determining, by the b neural network models to be trained, an average probability that each of the t neural network models to be trained is used to process each of the n sample images to obtain n x t average probabilities comprises: selecting one sample image from the n sample images, selecting one neural network model to be trained from the t neural network models to be trained, and determining the average probability that the selected neural network model to be trained is used for processing the selected sample image by the b neural network models to be trained according to the following operation until the n×t average probabilities are determined: determining, by each of the b neural network models to be trained, probabilities that the selected neural network model to be trained is used to process the selected sample image to obtain b probabilities; an average of the b probabilities is determined to obtain an average probability that the selected neural network model to be trained is used to process the selected sample image.
13. 13. The method of claim 11, wherein the normalizing the n x t average probabilities based on the n x t average probabilities and the t probability sets to obtain the n x t standard probabilities comprises: selecting one probability set from the t probability sets, and normalizing each average probability in the selected probability set until the n×t standard probabilities are determined according to the following operations: Determining a mean and standard deviation of average probabilities in the selected probability set; And determining a standard probability corresponding to each average probability in the selected probability set based on the average probability in the selected probability set, the mean value and the standard deviation.
14. 14. The method of claim 10, wherein the determining the first weight for each of the n sample images based on the n x t standard probabilities comprises: forming the n x t standard probabilities into a standard probability set, and determining a probability relative value corresponding to each standard probability in the standard probability set; determining a maximum probability relative value from probability relative values corresponding to standard probabilities in the standard probability set; determining a sample image corresponding to the maximum probability relative value from the n sample images, and determining a neural network model to be trained corresponding to the maximum probability relative value from the t neural network models to be trained; determining a first weight of a sample image corresponding to the maximum probability relative value based on a neural network model to be trained corresponding to the maximum probability relative value; removing standard probabilities corresponding to the sample images corresponding to the maximum probability relative values from the standard probability set; if the standard probability set is not empty after removal, updating model determination times, wherein the model determination times are times for determining the neural network model to be trained corresponding to the maximum probability relative value from the t neural network models to be trained; And if the model determination times are greater than or equal to n/t, removing the standard probability corresponding to the neural network model to be trained and corresponding to the maximum probability relative value from the standard probability set, and returning to the step of determining the probability relative value corresponding to each standard probability in the standard probability set.
15. 15. The method of claim 14, wherein the determining the first weight of the sample image corresponding to the maximum probability relative value based on the neural network model to be trained corresponding to the maximum probability relative value comprises: if the neural network model to be trained corresponding to the maximum probability relative value is the neural network model k to be trained, determining a first weight of a sample image corresponding to the maximum probability relative value as ; If the neural network model to be trained corresponding to the maximum probability relative value is not the neural network model k to be trained, determining a first weight of a sample image corresponding to the maximum probability relative value as ; Wherein, the For a set real number, satisfy And (2) and 。
16. 16. The method of claim 8, wherein the training function values comprise n label function values, the n label function values being in one-to-one correspondence with the n sample images; The determining the training function value corresponding to the neural network model k to be trained based on the n sample images and the n sample labels includes: determining standard probabilities that each sample image in the n sample images belongs to a corresponding sample label through each to-be-trained neural network model in s to-be-trained neural network models, so as to obtain n x s standard probabilities, wherein the s to-be-trained neural network models are models, in the m to-be-trained neural network models, which are allowed to be used for processing the n sample images after the to-be-trained neural network model k; and determining a label function value corresponding to each sample image in the n sample images based on the n x s standard probabilities.
17. 17. The method of claim 16, wherein the determining the label function value for each of the n sample images based on the n x s standard probabilities comprises: Forming the n-s standard probabilities into a standard probability set, and determining a probability relative value corresponding to each standard probability in the standard probability set; determining a maximum probability relative value from probability relative values corresponding to standard probabilities in the standard probability set; determining a sample image corresponding to the maximum probability relative value from the n sample images, and determining a neural network model to be trained corresponding to the maximum probability relative value from the s neural network models to be trained; The position numbers of the neural network models to be trained corresponding to the maximum probability relative values in the s neural network models to be trained are determined to be the label function values of the sample images corresponding to the maximum probability relative values; removing standard probabilities corresponding to the sample images corresponding to the maximum probability relative values from the standard probability set; If the standard probability set is not empty after removal, updating model determination times, wherein the model determination times are times for determining the neural network model to be trained corresponding to the maximum probability relative value from the s neural network models to be trained; And if the number of model determination times is greater than or equal to n/s, removing the standard probability corresponding to the neural network model to be trained and corresponding to the maximum probability relative value from the standard probability set, and returning to the step of determining the probability relative value corresponding to each standard probability in the standard probability set.
18. 18. The method of claim 14 or 17, wherein determining a probability relative value for each standard probability in the set of standard probabilities comprises: selecting one standard probability from the standard probability set, and determining a probability relative value corresponding to the selected standard probability according to the following operation until determining the probability relative value corresponding to each standard probability: Determining a plurality of standard probabilities corresponding to the same sample image with the selected standard probability from other standard probabilities except the selected standard probability in the standard probability set; determining a maximum standard probability of the plurality of standard probabilities; and determining the difference value of the selected standard probability and the maximum standard probability as a probability relative value corresponding to the selected standard probability.
19. 19. The method of claim 16, wherein the training function values comprise n class loss function values, the n class loss function values being in one-to-one correspondence with the n sample images; The determining the training function value corresponding to the neural network model k to be trained based on the n sample images and the n sample labels includes: selecting one sample image from the n sample images, and determining a classification loss function value corresponding to the selected sample image according to the following operation until the classification loss function value corresponding to each sample image is determined: Determining the probability that each of the s neural network models to be trained is used for processing the selected sample image through the neural network model k to be trained, so as to obtain s probabilities; And determining a classification loss function value corresponding to the selected sample image based on the label function value corresponding to the selected sample image and the s probabilities.
20. 20. The method of claim 8, wherein the training function values comprise n model selection weight function values, the n model selection weight function values being in one-to-one correspondence with the n sample images; The determining the training function value corresponding to the neural network model k to be trained based on the n sample images and the n sample labels includes: determining standard probabilities that each sample image in the n sample images belongs to a corresponding sample label through each to-be-trained neural network model in s to-be-trained neural network models, so as to obtain n x s standard probabilities, wherein the s to-be-trained neural network models are models, in the m to-be-trained neural network models, which are allowed to be used for processing the n sample images after the to-be-trained neural network model k; and determining a model selection weight function value corresponding to each sample image in the n sample images based on the n x s standard probabilities.

Description

Image classification, model training method, apparatus, storage medium, and computer program Technical Field The embodiment of the application relates to the field of image processing, in particular to an image classification and model training method, device, storage medium and computer program. Background In recent years, deep learning is widely applied in the field of image classification, and a neural network is used as an important branch of the deep learning, so that the accuracy of a classification result is greatly improved after an image classification task uses the neural network due to the ultra-strong fitting capability of the neural network. The related art proposes an image classification method in which a plurality of neural network models can be trained in advance. In classifying the image, a classification result is determined by each of the plurality of neural network models to obtain a plurality of classification results. And then, integrating the plurality of classification results to process so as to obtain a final classification result. However, the last method needs to acquire a classification result of each neural network model, so that the calculation amount is multiplied, and the image classification efficiency is greatly reduced. Disclosure of Invention The embodiment of the application provides an image classification and model training method, equipment, a storage medium and a computer program, which can solve the problem of low image classification efficiency in the related technology. The technical scheme is as follows: According to the image classification method, a target image to be classified is processed through a current neural network model to obtain a current classification result output by the current neural network model, wherein the current classification result comprises probabilities that the target image belongs to each of a plurality of categories, the current neural network model is a neural network model i corresponding to the maximum probability in selection results output by a neural network model a, the selection results output by the neural network model a comprise probabilities corresponding to p neural network models in the trained m neural network models, the p neural network models are neural network models allowed to be used for processing the target image after the neural network model a, the p neural network models comprise the neural network models i, the m neural network models comprise the neural network models a, the m is an integer greater than 1, the p is an integer greater than or equal to 1, and the p is smaller than m, the current integration result is determined based on the current classification result, the current integration result comprises the integrated probabilities that the target image belongs to each of the plurality of categories, and the category of the target image is determined based on the current integration result. Wherein, in the embodiment of the present application, the neural network model allowed to process the target image after the neural network model a in the m trained neural network models may include one or more, and the one or more neural network models are other neural network models used to process the target image after the neural network model a. That is, p is an integer greater than or equal to 1, and p is less than m. Since the selection result output by the neural network model a includes probabilities corresponding to a neural network model which is allowed to process the target image after the neural network model a among the m trained neural network models, the current neural network model is the neural network model i corresponding to the maximum probability among the selection results output by the neural network model a, so that it can be determined that the classification result output by the neural network model a does not satisfy the condition, so that the neural network model i corresponding to the maximum probability among the selection results output by the neural network model a is determined as the current neural network model, and then the target image is processed through the current neural network model. That is, in the embodiment of the present application, the target image is processed in a loop iteration manner, and when the classification result obtained in the previous loop does not meet the condition, the neural network model for processing the target image next time can be determined through the selection result obtained in the previous loop. Since the classification accuracy of the neural network model for processing the target image next time is higher than that of the neural network model for processing the target image previous time, the accuracy of image classification can be improved after performing the loop iteration processing in this way. That is, the embodiment of the application can dynamically select part of the neural network models from the m neural network models