CN-121809602-B - Model updating method and image classification method based on characterization fusion and head training

CN121809602BCN 121809602 BCN121809602 BCN 121809602BCN-121809602-B

Abstract

A model updating method and an image classification method based on characterization fusion and head training relate to the technical field of federal learning and privacy protection, and in particular relate to a local heterogeneous model updating method used in a federal learning framework. The method solves the problems of unreliable aggregation of prototypes and non-uniform decision boundaries caused by inconsistent feature space geometry in heterogeneous federal learning and knowledge transfer depending on a public data set excessively, and simultaneously remarkably reduces communication cost and model structure leakage risk. The method comprises the steps that a server distributes global model parameters to a client, the client generates a category prototype based on a local feature extractor, uploads updated parameters and the prototype after feature fusion is carried out on the category prototype and the global feature extractor, the server aggregates the parameters and the prototype, trains a global pre-measurement head by utilizing the aggregated prototype and issues the global pre-measurement head, and the client updates the local model and carries out image classification according to the updated parameters and the prototype. The invention is suitable for the fields of medical image analysis, industrial vision detection, edge intelligent equipment and the like.

Inventors

XU DAWEI
WU HENG
ZHAO GUOGANG
TONG ZHIPENG
ZHAO JIAN

Assignees

长春大学

Dates

Publication Date: 20260512
Application Date: 20260311

Claims (9)

1. An image classification method based on characterization fusion and head training is characterized in that, image classification is carried out by using a local heterogeneous model obtained by a model updating method based on characterization fusion and head training, and the image classification comprises the following steps: step B1, the client performs normalization, size adjustment and format conversion pretreatment on the image to be classified to obtain a pretreated image; step B2, inputting the preprocessed image into an updated local feature extractor to obtain an image feature representation; step B3, inputting the image characteristic representation into the updated local prediction head to obtain category probability or logits and outputting the category probability or logits; The model updating method comprises the following steps: Step A1, initializing global isomorphic model parameters by a server, wherein the global isomorphic model parameters comprise global feature extractor parameters and global prediction head parameters; A2, the client generates a category prototype based on a local feature extractor in the local heterogeneous model by using local training data, performs feature extraction through the local feature extractor and the initialized global feature extractor parameters respectively, and performs self-adaptive fusion on the two obtained features to obtain fused global feature extractor parameters; Step A3, the client uploads the parameters of the integrated global feature extractor and the category prototype to a server; Step A4, the server aggregates the parameters of the global feature extractor uploaded by all clients to obtain updated parameters of the global feature extractor, and aggregates the category prototypes to obtain global prototypes; Step A5, the server trains the global prediction head parameters by using the global prototype to obtain trained global prediction head parameters; Step A6, the client receives the global feature extractor parameters updated by the server and the trained global pre-measurement head parameters, and is used for updating the local heterogeneous model to obtain an updated local feature extractor and a local pre-measurement head; in step A2, the adaptive fusion includes the following steps: step A211, extracting the characteristics, wherein the client terminal extracts the sample pairs in the local training data during communication Input into global feature extractor and local feature extractor to extract global features And personalized features ; Step A212, double projection mapping, the client will globally characterize And personalized features Respectively input global projection module And local projection module Obtaining projection characteristics ; Step A213, splicing and fusing, namely, the projection characteristics And (3) with Splicing according to feature dimensions to form fused features ; Step A214, double-branch prediction, calculating the total loss of the client ; Step A215, joint optimization, wherein the total loss of the client side Updating global isomorphic models by gradient descent with independent learning rates Local heterogeneous model Global projection module And local projection module 。
2. The method for classifying images based on token fusion and head training according to claim 1, wherein in step A1, the server initializes global isomorphic model parameters by generating initial parameters according to the global isomorphic model by the server And the initial parameters are set The method comprises the steps of storing the data in a server as a global isomorphic model baseline; The server distributes the initialized global isomorphic model parameters to the selected clients as follows: Firstly, the server randomly selects a participation client set from the client complete set according to the set participation proportion : , Wherein the method comprises the steps of Representing the current communication round of time and, In order to set the participation proportion, Representing a subset of the slave corpus The selected number is the participation proportion Sum of total number of total sets A product of (2) is rounded down; then, the server updates the global isomorphic model parameters after the (t-1) th round Distributed to collections Is provided.
3. The method of image classification based on token fusion and head training according to claim 1, wherein in step a211, the global features are extracted And personalized features The method comprises the following steps: Wherein the method comprises the steps of Representing the client serial number and, The global feature extractor is represented by a graph, Representing a local feature extractor; In step A212, the obtained projection features The method comprises the following steps: Global features The dimension is Subspace, individualization features The dimension is The sub-space is defined by a sub-space, Will globally characterize Mapping to dimensions of Is (are) subspaces, i.e Obtaining projection characteristics , Will personalize the features Mapping to dimensions of Is (are) subspaces, i.e ; In step A213, the fused features The method comprises the following steps: Wherein the method comprises the steps of Representing a splice in the feature dimension, As a feature of the post-fusion, ; In step a214, the dual-branch prediction is a global prediction and a personalized prediction; The client terminal fuses the characteristics Input global pre-measurement head Obtaining global prediction : Will personalize the features Inputting local prediction header Obtaining personalized predictions : , The calculation client total loss The method comprises the following steps: , , , For globally predicting True labels in local training data Is added to the system, the loss of (a) is, For individualizing predictions True labels in local training data Is a loss of (2); Is that And Adding the obtained total loss of the client; in step a215, the client total loss Updating global isomorphic models by gradient descent with independent learning rates Local heterogeneous model Global projection module And local projection module The method comprises the following steps: Wherein the method comprises the steps of 、、、 Respectively corresponding to global isomorphic models Local heterogeneous model Global projection module And local projection module Is set up To ensure convergence stability.
4. The method of image classification based on token fusion and head training according to claim 1, wherein in step A2, the generating a class prototype comprises the steps of: Step A221, collecting local features, wherein the client traverses the local training set to belong to the category Is input to a local feature extractor Obtaining a corresponding local feature representation; step A222, generating a category prototype, wherein the client calculates the category based on the local feature representation Corresponding category prototype : , Wherein the method comprises the steps of Representing clients Including all tags as local data subsets of (1) Is a sample of (2) 。
5. The method of claim 1, wherein in step A3, the client uploads the fused global feature extractor parameters and class prototypes to a server as follows: The client side updates the parameters of the global isomorphic model Together with the identification of the client and training round information, the information is uploaded to a server for the server to record the client set participating in the training round ; The client side uploads the category prototype to the server in such a way that the client side uploads the category prototype And class labels therefor All class prototype set of components To the server with the necessary turns or client identifications.
6. The method for classifying images based on feature fusion and head training according to claim 2, wherein in step A4, the server aggregates the parameters of the global feature extractor uploaded by all clients into the updated global isomorphic model parameters by the server Accumulating layer by layer and averaging according to the number of the clients, namely, carrying out the calculation on each layer of parameters of the global isomorphic model Corresponding layer parameters of each client After accumulation, divide by Obtaining updated global isomorphic model parameters Completing the updated global isomorphic model parameters Is a polymer of (a).
7. The method of claim 1, wherein in step A4, the aggregating the class prototypes is: first, for each category Mapping the class prototypes to the shared potential space by a neural feature transformer T to obtain potential prototypes: , Wherein the method comprises the steps of Representing possession categories Is a set of clients of the (c) network, In order to share the dimensions of the potential space, To control parameters of the neural feature transformer T; The potential prototypes are then assembled using hybrid alignment loss Draw to its class centroid The mixed alignment loss combines the numerical value proximity and the direction consistency to finally obtain an aligned class center set Namely global prototypes: Wherein the method comprises the steps of Representation of The norm of the sample is calculated, Is cosine similarity, wherein Scalar is Euclidean inner product For balancing the directional terms with the numerical proximity terms.
8. The method of claim 1, wherein in step A5, the server trains the global pre-measurement head parameters using global prototypes as follows: In the first place In round communication, the server prototypes the global As training samples, corresponding categories are used As a real label, the class center is used Inputting current global pre-measurement head Training: For each category Will be like the center Inputting the global prediction head to obtain prediction output, and minimizing the prediction output and the category Loss function between To update the global pre-header parameters: Wherein the method comprises the steps of In order for the rate of learning to be high, Is shown in the first In-wheel aligned categories Is a potential centroid of (c).
9. The method for classifying images based on token fusion and head training according to claim 1, wherein in step A6, the process of receiving the updated global feature extractor parameters of the server and the trained global pre-prediction head parameters by the client is as follows: the server stores the updated global feature extractor parameters and the trained global pre-measurement head parameters to a parameter warehouse according to the communication round number; The server distributes the updated global feature extractor parameters, the trained global pre-measurement head parameters and training round information to the client according to the client selection result of the next round; the client uses the received updated global feature extractor parameters and the trained global prediction head parameters.

Description

Model updating method and image classification method based on characterization fusion and head training Technical Field The invention relates to the technical field of federal learning and privacy protection, in particular to a local heterogeneous model updating method used in a federal learning framework. Background Federal Learning (FL) is a distributed machine learning paradigm that allows multiple parties to co-train a model without sharing raw data, thereby utilizing scattered data resources while protecting data privacy. Typical FL frameworks often implement co-training by a server distributing a global model, clients training locally and uploading updates, and servers aggregating updates and then continuing the iteration. This approach has privacy preserving advantages since only model parameters are exchanged without exposing the original data. However, this scheme defaults to all clients being the same, while in a real cross-device scenario, the participating devices have significant differences in data distribution, model composition, computational effort, and bandwidth. Especially statistical heterogeneity (non-independent co-distribution) leads to global model bias towards data rich clients, resulting in performance degradation and convergence difficulties. To alleviate the statistical heterogeneity problem, personalized federal learning (pFL) was proposed in an effort to accommodate local data characteristics while retaining synergistic advantages. However, most personalization methods still assume that the clients use the same model architecture and that transmitting complete model parameters or gradients is not only costly to communicate, but also may reveal model structure information. To directly address the challenges of model isomerization and communication privacy, heterogeneous federal learning (HtFL) has emerged as a novel paradigm. HtFL allow clients to use different model architectures and handle heterogeneous data and reduce communication costs and improve performance by sharing knowledge (rather than model parameters) among clients. However, existing methods still face some limitations, such as knowledge distillation methods that send predictions logits as global knowledge over a shared data set, but such methods rely heavily on the quality and availability of the shared data set, whereas non-data-aware distillation methods introduce a shared global equidistant small model to interact with heterogeneous local models, or a shared lightweight class representation (i.e., prototype) as global knowledge, but typically require weighted averaging of client-uploaded samples, which can lead to data distribution leakage. More importantly, the class prototypes extracted by different model architectures are in heterogeneous feature space, have distribution offset and geometric inconsistency, and are unreliable in direct aggregation. In summary, the core problem faced by the current heterogeneous federal learning model training is that knowledge distillation-based technology excessively depends on a common data set, and a prototype sharing scheme of a non-knowledge distillation fraction is unreliable due to geometrical inconsistency of heterogeneous feature space, so that global decision boundary instability is finally caused, and precision improvement in image classification application is limited. Disclosure of Invention The method solves the problems of unreliable aggregation of prototypes, non-uniform decision boundaries and excessive dependence on a public data set for knowledge transfer caused by inconsistent feature space geometry in heterogeneous federal learning, and simultaneously remarkably reduces the communication cost and the model structure leakage risk. The model updating method based on the characterization fusion and the head training comprises the following steps: Step A1, initializing global isomorphic model parameters by a server, wherein the global isomorphic model parameters comprise global feature extractor parameters and global prediction head parameters; A2, the client generates a category prototype based on a local feature extractor in the local heterogeneous model by using local training data, performs feature extraction through the local feature extractor and the initialized global feature extractor parameters respectively, and performs self-adaptive fusion on the two obtained features to obtain fused global feature extractor parameters; Step A3, the client uploads the parameters of the integrated global feature extractor and the category prototype to a server; Step A4, the server aggregates the parameters of the global feature extractor uploaded by all clients to obtain updated parameters of the global feature extractor, and aggregates the category prototypes to obtain global prototypes; Step A5, the server trains the global prediction head parameters by using the global prototype to obtain trained global prediction head parameters; and step A6, the client receive