CN-117611887-B - Causal reasoning semi-supervised image classification method and device for class interval adaptation

CN117611887BCN 117611887 BCN117611887 BCN 117611887BCN-117611887-B

Abstract

The application relates to a causal reasoning semi-supervised image classification method and device with interval-like adaptation. The method comprises the steps of calculating a tendency score related to class edge distribution of an image sample according to class condition distribution of the image sample with the label and class edge distribution of the image sample without the label, introducing the tendency score of a corresponding class into class probability output by an image classification model to serve as a dynamic interval threshold related to the class, constructing a loss function according to the tendency score, and optimizing the loss function to obtain a trained image classification model to classify target images. The method can effectively solve the problem of unbalanced categories commonly existing in the unmarked image data in the dynamic open scene, and improves the accuracy and the robustness of image classification.

Inventors

ZENG YUJUN
HU XIAOCHANG
FANG QIANG
XU XIN
REN JUNKAI
LAN Yixing

Assignees

中国人民解放军国防科技大学

Dates

Publication Date: 20260512
Application Date: 20231121

Claims (8)

1. A causal reasoning semi-supervised image classification method for class interval adaptation, the method comprising: Calculating to obtain a tendency score related to the class edge distribution of the image sample according to the class condition distribution of the labeled image sample and the class edge distribution of the unlabeled image sample; Introducing a tendency score of a corresponding category into the category probability output by the image classification model to serve as a dynamic interval threshold related to the category, and constructing a loss function according to the tendency score; Optimizing the loss function to obtain a trained image classification model so as to classify the target image; the step of estimating a class edge distribution of the unlabeled image samples comprises: the historical class edge distribution is weighted using an exponential moving average to estimate the class edge distribution of the unlabeled image samples: ; Wherein, the Indicating that the label in the normalized label-free image sample corresponds to the first The distribution of the edges of the class, The representation belonging to the first Class image sample The corresponding output at the logic layer after the input of the image classification model, Class edge distribution for the class i unlabeled image samples, As an average scale factor, m represents the number of unlabeled exemplars and C is the total number of categories.
2. The method of claim 1, wherein the calculating a tendency score associated with the class edge distribution to which the image sample belongs based on the class condition distribution of the labeled image sample and the class edge distribution of the unlabeled image sample is: ; Wherein, the Representing that the label in the image sample corresponds to the first An edge distribution related tendency score for a class, r representing a binary tag loss indicator variable, r=1 representing labeled, r=0 representing unlabeled or tag loss, Indicating that the label in the label-free image sample corresponds to the first The distribution of the edges of the class, The label in the labeled image sample corresponds to the first The conditional distribution of the class.
3. The method of claim 2, wherein calculating a tendency score associated with a class edge distribution to which the image sample belongs based on the class condition distribution of the labeled image sample and the class edge distribution of the unlabeled image sample comprises: Setting the class condition distribution of the labeled image sample as uniform distribution, and further calculating to obtain a tendency score related to the class edge distribution of the image sample: 。
4. the method of claim 1, wherein training an image classification model comprises: And estimating unbiased pseudo labels of the unlabeled image samples, taking the unbiased pseudo labels as labels of the unlabeled image samples, and training an image classification model together with the labeled image samples to obtain an intermediate image classification model.
5. The method of claim 4, wherein estimating unbiased pseudo tags for unlabeled image samples is: ; Wherein, the Unbiased pseudo tags representing unlabeled image samples, To when the input is The logic output of the image classification model is then 、 , Representing parameters to be optimally determined in the image classification model, For the manually set de-biasing weight parameters, Class edge probability for estimated k, c-th unlabeled data.
6. The method of claim 1, wherein the expression of the loss function is as follows: ; Wherein, the For the image classification model parameters, Representing a predisposition score related to the edge distribution of the class to which the image belongs, and for any Has the following components , For a binary tag loss indicator variable, r=1 indicates marked, r=0 indicates no marking or tag loss, and C indicates the total number of image categories; Representing the ith training image sample and For the feature vector corresponding to the image sample, N is the number of the image samples with labels, and m is the number of the image samples without labels; Represents the unsupervised learning loss term corresponding to the unbiased pseudo tag estimate for the unlabeled image sample, Representing a supervised learning penalty term for a labeled image sample, in the form of a negative log penalty, The tendency is indicated by a compartmentalization process, Is a super parameter.
7. The method of claim 6, wherein the loss function is formalized as follows: ; Wherein C is the total number of categories contained in the sample dataset, To when the input is The logic output of the image classification model is then 、 , 、 Respectively corresponding to the input samples as Tendency score when inputting samples In the case of the presence of tag data, I.e. representing its corresponding label, when a sample is entered In the case of non-tag data, Then its corresponding pseudo tag is indicated.
8. A causal reasoning semi-supervised image classification apparatus for class interval adaptation, characterized in that the method of any of claims 1 to 7 is employed, the apparatus comprising: the tendency score calculation module is used for calculating a tendency score related to the class edge distribution to which the image sample belongs according to the class condition distribution of the labeled image sample and the class edge distribution of the unlabeled image sample; The loss function construction module is used for introducing the tendency score of the corresponding category into the category probability output by the image classification model to serve as a dynamic interval threshold value related to the category, and constructing a loss function according to the tendency score; And the loss function optimization module is used for optimizing the loss function to obtain a trained image classification model so as to classify the target image.

Description

Causal reasoning semi-supervised image classification method and device for class interval adaptation Technical Field The application relates to the technical field of image classification, in particular to a causal reasoning semi-supervised image classification method and device for class interval self-adaption. Background The existing image classification method based on the deep neural network is often based on a large-scale and high-quality manual marking data set, compared with the method for marking a large amount of reliable data, a non-marking sample is low in cost, and the image classification method based on semi-supervised learning assists in training of a classification model under a small amount of marking data by introducing a large amount of non-marking data, so that the method is an effective method for reducing manual marking cost. However, existing semi-supervised learning based image classification methods generally assume that the label distribution of unlabeled data is balanced, i.e., the number of samples in each class is nearly equal, while many real world datasets exhibit long tail distributions, typically manifested as a far greater number of samples for the head class than for the tail class. The contradiction between ideal data assumptions and practical applications presents a theoretical and performance double challenge to the effectiveness and robustness of such methods in classifying images in class imbalance scenarios. When a general deep semi-supervised method processes the data, the category distribution of the predicted pseudo tags tends to be of the head type, the bias is gradually strengthened along with continuous iteration of training, and finally the bias is reflected as serious deterministic bias, so that the final classification precision and the robustness are affected. The semi-supervised learning image classification for unbalanced category distribution is more challenging because the category distribution is a priori unknown. Related research works and technical schemes can be divided into three types, namely sample Re-sampling (sample Re-sampling), pseudo tag alignment (Pseudo Label Alignment) and sample Re-weighting (sample Re-weighting). Sample resampling methods attempt to resample training samples to construct balanced data sets. The CReST method is a typical resampling method, which assumes that labeled data and unlabeled data have similar class distributions, and proposes to estimate the proportion of unlabeled sample samples based on the number distribution of labeled samples. Similar to CReST sampling techniques, the CoSSL method decouples feature representations from the classifier and resamples samples with lower recall according to class proportions of labeled samples, except that the method uses random interpolation to augment minority class feature layers, mixing pseudo tags from different classes to reduce the overall bias of the pseudo tags. Similarly, the auxiliary classifier method (Auxiliary balanced classifier, ABC) adds an auxiliary classifier on the basis of the original semi-supervised classifier, uses a mask of Bernoulli distribution sampling, which is positively correlated with the proportion of marked sample categories, to ensure the category balance of the auxiliary classifier, and uses the consistent regularization loss of the applied category mask for unlabeled samples, and the mask probability is estimated based on the Bernoulli distribution between the number of pseudo-labeled categories and the number of marked sample categories. The disadvantage of this type of method is that it assumes that the labeled and unlabeled samples are similarly distributed, and cannot handle class imbalance scenes with dissimilar distributions. To alleviate this problem, the Adsh method considers that choosing the pseudo tag based on a fixed threshold is a main cause of class imbalance, sets a threshold variable for each class, and uses the concept of double-layer optimization (Bi-level Optimization) to solve the parameters of the pseudo tag threshold and the optimized classification model related to each class step by step. The pseudo tag alignment method eliminates deterministic bias of model accumulation by matching the pseudo tag to the desired distribution. Kim J et al construct the false annotation contrast to a convex optimization problem and implement this by minimizing the distribution KL divergence of the predicted false annotation and class distribution truth values, but this approach is based on the assumption that there are and no labeled samples with similar distributions and uses a confusion matrix to estimate the true class distribution of the unlabeled samples. In order to get rid of the assumption of the distribution similarity of the label samples in the method and the label samples in the method, the Oh et al find that the category bias of the semantic pseudo labels and the pseudo labels predicted by the classifier caused by the unba