CN-122023910-A - Class relation divergence-based adaptation method for open set image classification test

Abstract

The invention relates to an adaptation method in open set image classification test based on class relation divergence, which comprises the following steps of selecting a public image data set, selecting a general pre-training model to complete pre-training on the data set, selecting a target data set and dividing the target data set into a plurality of batches, calculating class relation divergence corresponding to each predicted image to be classified in each batch, presetting a class relation divergence threshold value, comparing the class relation divergence corresponding to each previously obtained image to be classified with the threshold value, forming a known class sample subset by the images to be classified which are smaller than the threshold value, and then calculating the image to be classified and the predicted image to be classified And selecting the class corresponding to the highest similarity value as the predicted class of the image to be classified, and updating the original class prototype for the predicted use of the next batch. The invention has remarkable effect in an open set test environment by introducing a sample discrimination and self-adaptive constraint mechanism based on class relation divergence in a test stage.

Inventors

• GE YONGXIN
• ZHOU JIANGHAO
• LI GUANGRUI
• ZHU ZHENGYU
• HUANG CHENG
• XU LING
• WANG HONGXING
• YANG MENGNING
• HONG MINGJIAN

Assignees

• 重庆大学

Dates

Publication Date

20260512

Application Date

20260130

Claims (3)

1. 1. The adaptation method for the open set image classification test based on the class relation divergence is characterized by comprising the following steps: S1, selecting a public image dataset as a source domain dataset , wherein, Representation of Is used for the image sample of the i-th image, A category label representing the ith image sample, Representation of The total number of image samples in (a); selecting a general pre-training model And construct a supervised cross entropy loss function For a pair of Pre-training to obtain trained And utilize Calculation of Upper part of the cylinder Class prototypes of a known class, class prototypes of a kth class are noted as ; S2, selecting a test sample image dataset to be classified as a target domain dataset The said The test sample in (1) does not contain image category information and will The image data of the test sample to be classified is divided into a plurality of batches; s3, selecting a batch, and calculating each test sample image to be classified in the batch Class relation divergence of (2) Wherein Representing the characteristics of the test sample, The distribution of the target relationship is represented, Representing a class prototype most similar to the image features of the test sample currently to be classified, Representing a source relationship distribution; Traversing all the test sample images to be classified in the batch to obtain class relation divergence corresponding to each test sample image to be classified; s4, presetting a class relation divergence threshold value, comparing each class relation divergence obtained in the S3 with the class relation divergence threshold value, judging a test sample image corresponding to the class relation divergence threshold value smaller than the class relation divergence threshold value as a known class sample, and forming a subset of the known class samples ; S5, from Any test sample image to be classified is selected Wherein The corresponding test sample is characterized by ; Calculation of And (3) with Calculating cosine similarity value between class prototypes of known classes, and setting class prototypes with highest cosine similarity value The corresponding category is taken as Category prediction results of (2); S6, utilizing Updating The class prototype of the belonging class updates the formula as follows: Wherein, the The magnitude of the update is indicated and, When the initial category prototype is ; S7, traversing the batch correspondence Repeating S5-S6 to obtain corresponding test sample images The method comprises the steps of obtaining class prediction results of all to-be-classified test sample images, updating class prototypes corresponding to each class, and using the updated class prototypes in class prediction of the next batch of image data; s8, judging whether to finish And (3) carrying out class prediction on all the to-be-classified test sample images in the batch, returning to the step (S3) if the class prediction is not completed, otherwise, outputting class prediction results of all the known class samples, and thus completing the image classification prediction work of the current target domain image dataset.
2. 2. The method for adaptation in classification test of open set image based on class relation divergence as set forth in claim 1, wherein said S1 is calculated The steps of category prototypes for several known categories above are as follows: constructing a supervised cross entropy loss function Will be As input pair Training, training the model by adopting gradient descent counter-propagation, when Stopping training when convergence or training reaches the maximum iteration number to obtain a trained pre-training model ; The calculation formula of (2) is as follows: Wherein, the Representing conditional probabilities; Will be As a means of Obtaining sample characteristics of each image and corresponding categories thereof; Sample features belonging to the same category are aggregated, and then a category prototype of the corresponding category is obtained by calculation, wherein the calculation formula is as follows: Wherein, the A category prototype representing the nth category, Representing the sample feature corresponding to the ith image sample, Indicating the function.
3. 3. The method for adaptation in classification test of open set image based on class relation divergence as set forth in claim 2, wherein said S3 is calculated Class relation divergence of (2) The method comprises the following steps: s3-1, any one test sample image is processed Input device Obtained by (1) Corresponding test sample features ; S3-2 calculation Cosine similarity value between each class prototype Obtaining cosine similarity value set Wherein And the first Personal class prototypes The cosine similarity value between the two formulas is as follows: Wherein, the An L2 norm representing the vector; Will be All cosine similarity values in the three are normalized to obtain Target relation distribution of (2) The calculation formula is as follows: Wherein, the Represent the first A personal class prototype; s3-3 according to Between each category prototype Will be maximum Class prototypes with corresponding values as Corresponding category prototypes of (a) The method described in the step S3-2 is adopted to calculate the product Source relation distribution of (a) ; S3-4 calculation of the degree of divergence using Jensen-Shannon Class relation divergence of (2) The calculation formula is as follows: Wherein, the Representation of Is used to determine the (j) th component of the (c), Representation of 。

Description

Class relation divergence-based adaptation method for open set image classification test Technical Field The invention relates to the technical field of computer vision, in particular to an adaptation method in open set image classification test based on class relation divergence. Background Deep learning models have been widely used in computer vision tasks for image classification, object recognition, etc., and their performance depends largely on the consistency of distribution between training data and actual application data. However, during actual deployment of the model, the data received during the testing phase often comes from an unknown or changing environment, and there is a significant difference in the source data distribution during the training phase, resulting in reduced model performance. Because the test data usually does not have manual labeling conditions, how to improve the adaptability of the model in the test stage under the unsupervised condition becomes an important problem in current research and application. To alleviate the above problems, a Test-Time Adaptation (TTA) method is proposed and is receiving attention. The method generally utilizes the continuously arrived test data to update the model parameters or intermediate statistics on line after the model deployment is completed, so that the model is gradually adapted to the data distribution change of the target environment. The existing self-adaptive method during test generally aims at minimizing prediction uncertainty, aligning feature distribution or maintaining model output stability, and can be used for alleviating the deviation of target domain distribution under the condition that source domain data is not required to be accessed or a model is not required to be retrained. In the model deployment stage, the existing TTA method updates model parameters or internal statistics on line by using continuously arrived test samples, so that the problem of distribution deviation between training data and test data is relieved. Under the condition that the test data category space is kept consistent with the training stage, the method can improve the adaptability of the model in an unknown environment to a certain extent. However, the above-mentioned adaptive method during testing generally depends on the prediction result or statistical characteristic of the test sample as the basis for adaptive update. When the test stage data contains unknown class samples which do not appear in the training stage, the model can still conduct forced prediction on the unknown class samples according to the existing class space, and the prediction result is used for a follow-up self-adaptive updating process. Because of the significant differences in the distribution of unknown class samples in the feature space from known classes, such indistinguishable adaptive updates are prone to introducing false gradients or false statistics, thereby adversely affecting model parameters. In addition, existing adaptive methods during testing often lack an evaluation mechanism for reliability of test samples, and it is difficult to determine whether a certain test sample is suitable for model adaptive updating. Under the open environment, when the proportion of the unknown class samples is high or the class distribution is changed drastically, the model can still continuously carry out self-adaptive operation, and the negative influence caused by error update is further amplified. The lack of the adaptive strategy of screening and constraint limits the usability and stability of the adaptive technology in real complex scenes in the existing test. Therefore, the prior art has the objective defects that the known type samples and the unknown type samples cannot be effectively distinguished in the test stage, so that the model is subjected to error self-adaptive update in an open set test environment, and the overall performance and the system reliability of the model are further affected. In view of the above drawbacks, there is a need for a technical solution that can identify unknown class samples during a testing phase and constrain and optimize the adaptation process based on the identification result, so as to achieve more stable and efficient adaptation at the time of testing under open world conditions. Disclosure of Invention Aiming at the problems in the prior art, the invention aims to solve the technical problem of how to realize self-adaption during test under a complex target environment based on class relation divergence. In order to solve the technical problems, the invention adopts the following technical scheme: an adaptation method for open set image classification test based on class relation divergence comprises the following steps: S1, selecting a public image dataset as a source domain dataset , wherein,Representation ofIs used for the image sample of the i-th image,A category label representing the ith image sample,Representation ofTh