Search

CN-122023939-A - Classification incremental classification method based on union graph modeling and fusion space

CN122023939ACN 122023939 ACN122023939 ACN 122023939ACN-122023939-A

Abstract

The invention discloses a category increment classification method based on a union graph modeling and fusion space, which comprises the following steps: and configuring an independent adaptation module for each incremental stage, extracting the category characteristics of each stage through freezing the pre-training visual backbone network, and constructing a stage category prototype set. And calculating cross-stage similarity among global category prototypes, establishing category confusion relations, and identifying confusion communication branches by utilizing a union structure. And fusing the characteristics of the corresponding incremental stage adaptation module and optimizing the fusion weight aiming at each communication branch to form a fusion characteristic space. In the reasoning stage, the preliminary classification result is used for judging whether the candidate category belongs to the confusion communication branch, if so, the final result is output by carrying out fine judgment in the fusion feature space, and otherwise, the preliminary classification result is directly output. The method effectively solves the problem of identifying the cross-stage confusing category, improves the classification precision, and simultaneously keeps the low incremental training cost and the model forgetting resistance.

Inventors

  • ZHAO SHENGJIE
  • WANG YU
  • ZHONG HANG

Assignees

  • 同济大学

Dates

Publication Date
20260512
Application Date
20260306

Claims (10)

  1. 1. The category increment classification method based on the union graph modeling and fusion space is characterized by comprising the following steps of: Constructing a class increment learning task environment, loading a pre-training visual backbone network and freezing parameters of the pre-training visual backbone network, and respectively configuring independent adaptation modules for each increment stage in the visual backbone network; training the adaptation module corresponding to the incremental stage by sequentially utilizing the new type data corresponding to the incremental stage according to a preset incremental stage sequence, fixing parameters of the adaptation module after training, extracting characteristic representations of each type of the incremental stage based on the trained visual backbone network and the corresponding adaptation module, and constructing a type prototype set of the incremental stage; After the training of all the incremental stages is completed, summarizing class prototype sets of all the incremental stages to form a global class prototype set; based on the global category prototype set, calculating the similarity relationship between category prototypes from different incremental stages, and establishing confusion association between categories when preset confusion judging conditions are met, so as to construct a cross-stage category confusion relationship set; Based on a class confusion relation set, constructing a cross-stage class confusion graph by taking each class as a node and the confusion association as an edge, and carrying out connectivity merging processing on the confusion graph by utilizing a merging data structure to obtain a plurality of class confusion communication branches; Aiming at the confusion communication branches, selecting corresponding adaptation modules to construct fusion feature spaces according to increment stages to which each class in the branches belongs, and enhancing distinguishing capability among the classes in the confusion communication branches by optimizing fusion weights to obtain fusion feature space models corresponding to the confusion communication branches; In the reasoning stage, for a sample to be classified, sample characteristics are extracted through an adaptation module of each increment stage, and the sample characteristics are matched with the global category prototype set to obtain a preliminary classification result; And when the candidate categories belonging to the same confusion communication branch and coming from different incremental stages exist in the preliminary classification result, calling a corresponding fusion feature space model to judge the candidate categories again, and outputting a final classification result, and if the candidate categories do not exist, directly outputting the preliminary classification result as the final classification result.
  2. 2. The method for classifying class increment based on the union graph modeling and fusion space according to claim 1, wherein the constructing class increment learning task environment specifically comprises: setting the class increment learning task to include the total increment stage number as The data sets corresponding to each increment stage are respectively Wherein the data set categories of any two different incremental stages are not overlapped with each other, thereby meeting the following requirements Wherein ; Data set for each incremental phase The training sample set including the new class in the increment stage and the corresponding class label set are specifically expressed as: Wherein, the Represent the first The first increment stage Input samples, the input samples being image data; A class label corresponding to the input sample is represented; Represent the first Total number of samples for each incremental stage; The data set of the incremental stage is used for performing supervision training on the adaptation module of the corresponding incremental stage, so that the adaptation module learns the characteristic representation of the newly added class of the stage under the condition of freezing the visual backbone network parameters, and is used for calculating class prototypes of each class of the stage after training is completed.
  3. 3. The method for classifying class increment based on the union graph modeling and fusion space according to claim 1, wherein the loading the pre-training visual backbone network and freezing parameters thereof, respectively configuring independent adaptation modules for each increment stage in the visual backbone network, specifically comprises: loading a pre-training visual backbone network, and setting all network parameters of the pre-training visual backbone network to be in a frozen state, so that the weight of the pre-training visual backbone network is kept not updated in the training process of each incremental stage; Inserting adaptation modules in each feature transformation layer of the visual backbone network , Each increment stage corresponds to an independent adaptation module, and the adaptation module adopts a bottleneck structure and comprises a dimension-reducing mapping matrix Nonlinear activation function and up-dimension mapping matrix ; The adaptation module inserts a feedforward network bypass of each layer of the visual backbone network in a residual connection mode, and the output correction of the feedforward network is as follows: Wherein, the For an input feature representation of the corresponding layer of the visual backbone network, Outputting the integrated adaptive module to a feedforward network; a feed-forward neural network module representing the layer in the visual backbone network; Representing a linear rectification activation function; initializing corresponding adaptation module parameters for each incremental stage The parameters of the adaptation modules of each increment stage are mutually independent and not shared, and are used for learning the characteristic representation of the corresponding stage category in different stages.
  4. 4. The classification method of class increment based on the union graph modeling and fusion space according to claim 1, wherein the training of the adaptation module corresponding to the increment stage by sequentially utilizing the new class data corresponding to the increment stage according to the preset increment stage sequence specifically comprises the following steps: For the first The increment stage fixes the visual backbone network parameters, and only the first stage Adaptation module corresponding to each increment stage Parameters of (2) Setting the training state; data set of the increment stage Input is by vision backbone network and adaptation module A characteristic extraction model is formed, and sample characteristic representation is obtained; constructing a classification prediction function on the basis of the characteristic representation, and obtaining prediction output through a linear classifier; Constructing a supervision loss function based on the real class labels in the data set and the prediction output, wherein the loss function adopts cross entropy loss; and minimizing the loss function by adopting a gradient descent optimization method, and only updating the parameters of the adaptation module and the classifier parameters of the corresponding stage until the preset convergence condition is met or the maximum training round number is reached.
  5. 5. The method for classifying class increment based on the union graph modeling and fusion space according to claim 1, wherein the training-based visual backbone network and the corresponding adaptation module extract feature representations of each class of the increment stage, and construct a class prototype set of the increment stage, specifically comprising: In the first place Training completion and fixed adaptation module for incremental stages After parameters, the classifier of the corresponding stage is removed or frozen, and only the frozen visual backbone network and the first stage are reserved The incremental stage adaptation modules are used as feature extraction models; Data sets to be associated with incremental phases Re-inputting the feature extraction model to obtain a feature representation of each sample: Wherein, the Represent the first In the increment stage A characteristic representation of the individual samples; Representing input samples via frozen visual backbone network and first Feature mapping functions processed by the incremental stage adaptation module; Represent the first The first increment stage A plurality of input samples; Represent the first The adaptation modules correspond to the increment stages; For the first Any one of the incremental phases Selecting a label meeting the category And carrying out mean value aggregation on the sample characteristic representations to obtain a category prototype of the category: Wherein, the Represent the first Class in incremental phases Is a category prototype of (2); Represent the first Class in incremental phases Is a sample number of (a); Represent the first Total number of samples for each incremental stage; For indicating functions, the value is 1 when the condition in brackets is satisfied, otherwise, the value is 0; Represent the first In the increment stage Category labels for the individual samples; Represent the first A set of categories for each incremental stage; Will be the first Class prototypes corresponding to all classes of the incremental stage form a class prototyping set of the incremental stage: Wherein, the Represent the first A set of class prototypes for each incremental phase.
  6. 6. The method for classifying class increment based on the union graph modeling and fusion space according to claim 1, wherein the calculating similar relations among class prototypes from different increment stages based on the global class prototype set, when a preset confusion judging condition is satisfied, establishing a confusion association among classes, and constructing a cross-stage class confusion relation set specifically comprises: for any two categories in the global category prototype set, arbitrarily selecting two category prototypes from different incremental phases from the global category prototype set And (3) with When meeting the following requirements When the prototype similarity between the two is calculated: Wherein, the Representing categories Category and category Prototype similarity of (a); Represent the first Class in incremental phases Is a category prototype of (2); Represent the first Class in incremental phases Is a category prototype of (2); An L2 norm representing the vector; setting a preset confusion determination threshold When meeting the following requirements At the time, the category is determined Category and category Confusion association exists in the feature space, and confusion edges between categories are established and recorded as Otherwise, it is marked as Wherein, the method comprises the steps of, Representing categories Category and category The confusion between them indicates the variable, when If not, the method indicates that no confusion association is established; will all meet And is also provided with Form a set of cross-phase category confusion relationships: Wherein, the Representing a set of cross-stage category confusion relationships; Indicating pairs of confusion associations that are determined to exist in a confusion relationship between different incremental phases.
  7. 7. The method for classifying class increment based on a union graph modeling and fusion space according to claim 1, wherein the class confusion relation set is characterized in that each class is taken as a node, the confusion association is taken as an edge, a cross-stage class confusion graph is constructed, and connectivity merging processing is carried out on the confusion graph by utilizing a union data structure to obtain a plurality of class confusion communication branches, and the method specifically comprises the steps of: Using the categories in the global category prototype set as nodes of the graph, and category confusion relation set Each pair of confusion association in (a) Obtaining a cross-stage class confusion graph as an edge of the graph Expressed as: Wherein, the Class nodes for a set of nodes of a graph, including all incremental phases ; All confusion association pairs in the category confusion relationship set E are included for the edge set of the graph; Initializing and checking a data structure, setting each category as an independent set, and checking the initial state of the set to be that each category node is an independent communication branch; For category nodes The corresponding initial set is: Wherein, the Representing category nodes Is provided with a node (a) which is a parent node of the (c), Representing category nodes Is a tree depth of (2); traversing confusion graphs Each edge of (a) For each pair of confusion associations, the following is performed: For each pair of class nodes And Find operation lookup using union sets, respectively And If the root node of (1) Executing Union operation to combine the sets of the two category nodes and updating the corresponding node And Information: after the processing of all the edges is completed, a union set comprising a plurality of connected branches is obtained, wherein each connected branch represents a category set which is connected in the graph through a confusion relationship; Dividing a plurality of confusion communication branches by combining each independent set in the searching set, wherein each confusion communication branch consists of a plurality of category nodes which are connected with each other through a confusion relation; and screening out the mixed connected branches to obtain connected branches containing at least two category nodes, and taking the connected branches as finally obtained category mixed connected branches.
  8. 8. The method for classifying class increment based on merging graph modeling and fusion space according to claim 1, wherein for the confusing communication branches, selecting corresponding adaptation modules to construct fusion feature space according to increment stages to which each class in the branches belongs, and enhancing distinguishing capability between classes in the confusing communication branches by optimizing fusion weights to obtain fusion feature space models corresponding to each confusing communication branches, specifically comprising: For each screened confusing connected branch , wherein, A node set for the graph; According to each category node Incremental phase to which the device belongs Selecting a corresponding adapting module with trained and fixed parameters As a module for class feature extraction; According to all kinds of nodes in the confusion communication branch Corresponding increment stage sending adapting module Defining a fused feature space that obfuscates connected branches Weighted combinations of features for the selected adaptation modules: Wherein, the Representing confusing connected branches Is a fusion feature space of (1); Representing confusing connected branches The number of medium class nodes; representing a corresponding adaptation module The learning fusion weight coefficient is used for adjusting the contribution of the features of each increment stage in the fusion space; Represent the first An adaptation module for each incremental stage; initializing fusion weight coefficients And satisfies the constraint conditions: Class prototypes that obfuscate classes within connected branches Mapping to fused feature space Obtaining a fused feature prototype ; In the fusion of feature space In computing feature prototypes for all classes within a confusing connected branch Paired euclidean distance between: Wherein, the And is also provided with , Representing euclidean distance; From feature prototypes The pairs of Euclidean distances between them define the optimization targets of the confusion communication branches : To maximize the optimization objective For the purpose, a gradient descent method is adopted to optimize the fusion weight coefficient Iterative updating is carried out until convergence or a preset maximum iterative frequency is reached, and optimized fusion weight coefficient and corresponding fusion characteristic space description information are saved to form a confusion communication branch Is a fused feature space model of (1) 。
  9. 9. The method for classifying class increment based on the combined view modeling and fusion space according to claim 1, wherein in the inference stage, for the sample to be classified, sample features are extracted through an adaptation module in each increment stage, and the sample features are matched with the global class prototype set to obtain a preliminary classification result, specifically comprising the following steps: inputting a sample x to be classified into a frozen visual backbone network, and respectively carrying out forward propagation through the adaptation modules of all incremental stages to obtain sample characteristics of each stage: Wherein, the Representing a sample to be classified Frozen visual backbone network and method Incremental phase adaptation module Processed sample features; representing the total number of incremental phases; For each incremental phase Calculating sample characteristics Cosine similarity to each class prototype in the incremental stage class prototype set ; Summarizing similarity calculation results of all stages, generating a global similarity sorting list, and selecting the front with highest similarity Individual categories as candidate category sets : Wherein each candidate class includes a class identification Incremental phase of Corresponding similarity score ; Representing a sample And incremental phase Category(s) Cosine similarity of (c); Set candidate classes As a result of the preliminary classification.
  10. 10. The method for classifying the class increment based on the union graph modeling and fusion space according to claim 1, wherein when candidate classes belonging to the same confusion communication branch and coming from different increment stages exist in the preliminary classification result, the corresponding fusion feature space model is called to judge the candidate classes again, and a final classification result is output, and if the candidate classes do not exist, the preliminary classification result is directly output as the final classification result, and the method specifically comprises the following steps: Checking the attribution relation between the top-1 candidate category and other candidate categories in the preliminary classification result, if at least one candidate category exists Triggering a second-stage refined judgment when the following two conditions are met: (a) The same confusion communication branch belongs to the top-1 candidate category; (b) From a different incremental stage than the top-1 candidate class; when the second-stage refined judgment is triggered, the confusion communication branch is called Is a fused feature space model of (1) Sample to be classified The features of (2) are mapped to a fusion feature space to obtain a sample to be classified Enhancement feature representations in a fused feature space; In the fusion of feature space In the method, a sample to be classified is calculated Is represented by enhanced features of (1) and confusing connected branches Feature prototypes of all classes within Cosine similarity of (c); taking the class with the maximum cosine similarity as a sample to be classified Is a final prediction category of (2); If the second-stage accurate judgment condition is not triggered, directly outputting the top-1 candidate category of the preliminary classification result as the final prediction category.

Description

Classification incremental classification method based on union graph modeling and fusion space Technical Field The invention belongs to the technical field of machine learning and computer vision, and particularly relates to a category increment classification method based on a union graph modeling and fusion space. Background Deep learning models have achieved excellent performance in computer vision classification tasks, but traditional models typically rely on batch training of fixed categories, which is difficult to accommodate for the demand for continuous incremental appearance of categories in real-world scenes. Class incremental learning (Class-INCREMENTAL LEARNING, CIL) is used as a key technology for solving the problem, allows the model to learn new Class knowledge step by step, retains the capability of learned classes, and has important value in practical application scenes such as intelligent monitoring, automatic driving, medical diagnosis and the like. In recent years, continuous learning based on a pre-training model has shown remarkable potential in the aspect of high-efficiency self-adaption across task sequences, and becomes a key foundation for relieving the problem of catastrophic forgetting in class incremental learning. To reduce the cost of incremental training, strategies employing lightweight auxiliary modules, particularly Adapter (Adapter) trimming, have gained widespread attention and application. According to the method, only a small number of newly-added adapters are finely adjusted to adapt to newly-added tasks by freezing the parameters of the pre-training backbone network, so that the feature extraction capability of the pre-training model is fully utilized, and the disastrous forgetting risk caused by parameter updating is effectively reduced. Although adapter trimming is prominent in class increment learning, there are significant limitations to existing approaches. Adapters trained on different incremental phases typically focus only on class feature learning of the current phase, and fail to establish an efficient differentiation mechanism for similar classes across phases. Specifically, the class feature prototypes extracted by the corresponding adapter in the previous stage may have a higher similarity in feature space with prototypes extracted by the proprietary adapter for similar classes in the later stage, resulting in difficulty in accurately distinguishing these cross-stage confusable classes by the model at the time of reasoning, thereby severely affecting overall classification performance. This type of similarity information can be used to guide further distinguishable modeling of cross-stage similarity class features. In addition, prior art CN121305230a discloses a small sample class increment classification method based on global and local synergy. The core thinking is that global alignment of visual and text prototypes, local alignment of color and shape attributes, and global and local text prompt learning is guided by knowledge distillation to alleviate small sample overfitting. However, the limitation of this approach is the lack of modeling across stage class confusion-while global and local hint learning is employed in the prior art, the approach focuses on solving the overfitting problem in small sample classification and is not specifically designed for class confusion problems in class increment learning. In particular, between classes in multiple incremental phases, there may be similar or confusing classes that are not effectively distinguished. This may lead to misclassification in the case of reasoning in practice, especially when there is similarity of categories between incremental phases. Therefore, how to structurally model the similarity among the inter-stage categories and pertinently distinguish the potential confusion categories in the reasoning stage under the application scene of the category continuous increment is still a technical problem to be further solved. Disclosure of Invention The invention aims to overcome the defects of the prior art and provide a category increment classification method based on a union graph modeling and fusion space. The aim of the invention can be achieved by the following technical scheme: The invention provides a category increment classification method based on a union graph modeling and fusion space, which comprises the following steps: Constructing a class increment learning task environment, loading a pre-training visual backbone network and freezing parameters of the pre-training visual backbone network, and respectively configuring independent adaptation modules for each increment stage in the visual backbone network; training the adaptation module corresponding to the incremental stage by sequentially utilizing the new type data corresponding to the incremental stage according to a preset incremental stage sequence, fixing parameters of the adaptation module after training, extracting charac