CN-121998035-A - Federal learning method and system based on many-to-many dynamic similarity aggregation

CN121998035ACN 121998035 ACN121998035 ACN 121998035ACN-121998035-A

Abstract

The invention provides a federal learning method and a federal learning system based on many-to-many dynamic similarity aggregation, which relate to the technical field of federal learning, and the method comprises the steps of randomly issuing a middleware model to a client and acquiring the middleware model after the client is trained and updated; and aggregating the updated middleware models and the collaborative models based on a dynamic aggregation strategy to update the middleware models again and returning to execute the step of randomly issuing the middleware models to the client to iterate the middleware models until the preset iteration times are reached to obtain a target middleware model, and aggregating the target middleware models and the collaborative models corresponding to the target middleware models to obtain a plurality of aggregated middleware models to generate a global model. Therefore, the invention reduces the influence caused by gradient conflict and client drift.

Inventors

LENG QINGMING
Guo Zixie
XU KE
XU LIN
YANG HAIOU

Assignees

九江学院

Dates

Publication Date: 20260508
Application Date: 20251211

Claims (7)

1. The federal learning method based on the many-to-many dynamic similarity aggregation is characterized in that a federal learning model comprises a cloud server and a plurality of clients, the cloud server comprises a plurality of middleware models and a global model, the number of the clients is matched with that of the middleware models, and the method is applied to the cloud server; The method comprises the following steps: Randomly issuing a middleware model to a client and acquiring the middleware model after the client trains and updates; Calculating Euclidean distance and cosine distance of the two updated middleware models, carrying out normalization weighting treatment to obtain similarity of the two updated middleware models, and obtaining a collaboration model of the current updated middleware model based on the similarity; Aggregating the updated middleware model and the collaboration model based on the dynamic aggregation strategy to update the middleware model again and returning to execute the step of randomly issuing the middleware model to the client to iterate the middleware model until the preset iteration times are reached to obtain a target middleware model; And aggregating the collaborative model corresponding to the target middleware model and the target middleware model to obtain a plurality of aggregated middleware models, and generating a global model according to the aggregated middleware models.
2. The many-to-many dynamic similarity aggregation-based federal learning method of claim 1, wherein the collaboration model selects from among the remaining updated middleware models of all updated middleware models other than the current updated middleware model, wherein the selection policy comprises: When the actual iteration times are smaller than the preset iteration times, selecting a middleware model with highest similarity as a cooperation model to complete aggregation; when the actual iteration times are not less than the preset iteration times, selecting a middleware model with the lowest similarity as a cooperation model to complete aggregation; wherein, the expression of the selection strategy is: Wherein CoMSel denotes a selection policy, W i denotes a parameter of the ith middleware model, W j denotes a parameter of the jth middleware model, e denotes the number of training iterations, and λ is a system boundary parameter.
3. The federal learning method based on many-to-many dynamic similarity aggregation according to claim 1, wherein the similarity calculation formula is: Sim(W i ,W j )＝0.5S Euc +0.5S Cos ; Wherein W i represents the parameter of the ith middleware model, W j represents the parameter of the jth middleware model, S Euc represents the similarity between the middleware model parameter W i obtained by Euclidean distance normalization and the middleware model parameter W j , and S Cos represents the similarity between the middleware model parameter W i obtained by cosine distance normalization and the middleware model parameter W j ; Wherein: wherein d Euc represents a Euclidean distance, d Cos represents a cosine distance; Wherein:
4. the federal learning method based on many-to-many dynamic similarity aggregation according to claim 1, wherein in the step of aggregating a collaborative model of a target middleware model and a target middleware model to obtain a plurality of aggregated middleware models, and generating a global model from the plurality of aggregated middleware models, an expression of the global model is: wherein W g represents a global model, K represents the number of middleware models, T represents the preset iteration times, and i represents the number of middleware models.
5. The federal learning method based on the many-to-many dynamic similarity aggregation is characterized in that a federal learning model comprises a cloud server and a plurality of clients, the cloud server comprises a plurality of middleware models and a global model, the number of the clients is matched with that of the middleware models, and the method is applied to the clients; The method comprises the following steps: acquiring a middleware model randomly issued by a cloud server; And updating the middleware model through data set training and uploading the middleware model after training and updating to the cloud server.
6. A federal learning system based on many-to-many dynamic similarity aggregation, applied to a cloud server, the system comprising: The acquisition module is used for randomly issuing the middleware model to the client and acquiring the middleware model after the client trains and updates; The computing module is used for computing Euclidean distance and cosine distance of the two updated middleware models, carrying out normalization weighting processing to obtain similarity of the two updated middleware models and obtaining a collaboration model of the current updated middleware model based on the similarity; the iteration module is used for aggregating the updated middleware model and the collaboration model based on the dynamic aggregation strategy to update the middleware model again and returning to execute the step of randomly issuing the middleware model to the client so as to iterate the middleware model until the preset iteration times are reached to obtain a target middleware model; and the aggregation module is used for aggregating the target middleware model and the cooperative model corresponding to the target middleware model to obtain a plurality of aggregated middleware models, and generating a global model according to the plurality of aggregated middleware models.
7. A federal learning system based on many-to-many dynamic similarity aggregation, for application to a client, the system comprising: the acquisition module is used for acquiring a middleware model randomly issued by the cloud server; and the training module is used for training and updating the middleware model through the data set and uploading the middleware model after training and updating to the cloud server.

Description

Federal learning method and system based on many-to-many dynamic similarity aggregation Technical Field The invention relates to the technical field of federal learning, in particular to a federal learning method and a federal learning system based on many-to-many dynamic similarity aggregation. Background The performance of deep learning models depends largely on large-scale, high-quality data. However, in real-world scenarios, data is often distributed across different devices, organizations, and even geographical locations, and the collection, management, and use of data is facing increasingly stringent legal and ethical constraints due to increasing concerns about user privacy. Therefore, the conventional centralized training paradigm, i.e. a training manner in which all data is collected to a central server for unified training, is severely restricted in practical deployment. In this context, federal learning is becoming a viable solution to the data island and privacy protection problems as an emerging distributed machine learning framework. In the prior art, the federal learning framework mainly uses a one-to-many model, and the framework maintains a unique global model at the cloud end and a plurality of local client models with private data at the local. And after the local model is trained by the local client, uploading the local model to a cloud server for aggregation. When the data heterogeneity of a plurality of clients is large, client drift phenomenon can occur, gradient conflict among models is unavoidable, and the existing method has obvious application limitation. Disclosure of Invention Based on the above, the invention aims to provide a federal learning method and a federal learning system based on many-to-many dynamic similarity aggregation, which are used for solving the limitations of the existing federal learning framework. The invention provides a federal learning method based on many-to-many dynamic similarity aggregation, wherein a federal learning model comprises a cloud server and a plurality of clients, the cloud server comprises a plurality of middleware models and a global model, the number of the clients is matched with that of the middleware models, and the method is applied to the cloud server; The method comprises the following steps: Randomly issuing a middleware model to a client and acquiring the middleware model after the client trains and updates; Calculating Euclidean distance and cosine distance of the two updated middleware models, carrying out normalization weighting treatment to obtain similarity of the two updated middleware models, and obtaining a collaboration model of the current updated middleware model based on the similarity; Aggregating the updated middleware model and the collaboration model based on the dynamic aggregation strategy to update the middleware model again and returning to execute the step of randomly issuing the middleware model to the client to iterate the middleware model until the preset iteration times are reached to obtain a target middleware model; And aggregating the collaborative model corresponding to the target middleware model and the target middleware model to obtain a plurality of aggregated middleware models, and generating a global model according to the aggregated middleware models. In addition, the federal learning method based on the many-to-many dynamic similarity aggregation according to the present invention may further have the following additional technical features: further, the collaboration model selects from the remaining updated middleware models of all updated middleware models except the current updated middleware model, wherein the selection strategy comprises: When the actual iteration times are smaller than the preset iteration times, selecting a middleware model with highest similarity as a cooperation model to complete aggregation; when the actual iteration times are not less than the preset iteration times, selecting a middleware model with the lowest similarity as a cooperation model to complete aggregation; wherein, the expression of the selection strategy is: Wherein CoMSel denotes a selection policy, W i denotes a parameter of the ith middleware model, W j denotes a parameter of the jth middleware model, e denotes the number of training iterations, and λ is a system boundary parameter. Further, the similarity calculation formula is: Sim(Wi,Wj)＝0.5SEuc+0.5SCos; Wherein W i represents the parameter of the ith middleware model, W j represents the parameter of the jth middleware model, S Euc represents the similarity between the middleware model parameter W i obtained by Euclidean distance normalization and the middleware model parameter W j, and S Cos represents the similarity between the middleware model parameter W i obtained by cosine distance normalization and the middleware model parameter W j; Wherein: wherein d Euc represents a Euclidean distance, d Cos represents a cosine distance; Wherein: furt