Search

CN-116070713-B - Method for relieving Non-IID influence based on interpretable federal learning

CN116070713BCN 116070713 BCN116070713 BCN 116070713BCN-116070713-B

Abstract

The invention discloses a method for relieving Non-IID influence based on interpretable federal learning. The invention mainly introduces an explanatory mechanism of the verification set to describe the explanatory results of the verification set sample in the central server based on the influence of the local client update on the learning ability of each class of the aggregation model, and evaluates the explanatory results of each class by using a Structural Similarity Index (SSIM), thereby deducing the clients with unbalanced data. And then, parameters of the clients with unbalanced data are adjusted, the gradient distance between the parameters of the unbalanced client model and the parameters of the global aggregation model of the last round is minimized, and the parameters of the unbalanced client model are corrected through convergence of the gradient distance. Thereby achieving a reduction in the negative effects of data imbalance.

Inventors

  • ZHOU WENJIE
  • LI PIJI
  • LIU ZHE

Assignees

  • 南京航空航天大学

Dates

Publication Date
20260508
Application Date
20221230

Claims (5)

  1. 1. A method for mitigating the effects of Non-IID based on interpretive federal learning, comprising the steps of: (1) Constructing a Non-IID scene, namely distributing data of different numbers and different labels to each client by using Dirichlet distribution aiming at a target data set, and constructing a Non-IID scene; firstly, setting K category labels, wherein the number of clients participating in federal learning is N, samples of each category label are required to be divided into different clients according to different proportions, and the category numbers on the clients are different; next, matrix is arranged A category label distribution matrix; Wherein the row vectors Representing a probability distribution vector of a class c on different clients, each dimension representing a sample division of a c-th class; the probability distribution vector is sampled from the dirichlet distribution: ; ; ; Wherein the method comprises the steps of Is a parameter; (2) A verification set is built, wherein a picture is placed in each category in a central server to serve as the verification set; (3) Selecting clients to perform interpretability, namely taking cosine similarity calculated by a local updating direction of the clients and an updating direction of a previous round of aggregation model as a score, and selecting k clients with low scores to participate in the interpretability; (4) The method comprises the steps of respectively inputting verification sets deployed on a central server into a previous round of aggregation model and a current round of client model for performing interpretable operation, performing interpretability according to a classification result of the model to obtain a highlight feature map, performing similarity calculation on the highlight feature maps of the two models in a one-to-one correspondence manner, wherein a similarity calculation formula is as follows: ; Wherein the method comprises the steps of The coefficients representing the picture x are represented by, K represents a normal amount for enhancing robustness, and if the variation amplitude of various similarities exceeds a set threshold value, the client data is considered to be unbalanced; (5) Dynamically adjusting the parameters of the client model, namely minimizing the gradient distance between the unbalanced client model and the parameters of the previous round of global aggregation model, correcting the parameters of the client model through convergence of the gradient distance, and flexibly adjusting the iteration times of the minimized gradient according to the federal learning aggregation time.
  2. 2. The method for alleviating the effects of Non-IID based on interpretive federal learning according to claim 1, wherein the selecting the client for interpretive according to the score in step (3) includes using the cosine similarity calculated by the local update direction of the client and the update direction of the previous round of aggregation as the score, and the calculation formula is as follows: ; Wherein the method comprises the steps of And (3) with Representing the local update gradient and the global gradient of the i-th client respectively, A score representing the ith client; and arranging the scores in order from small to large, and selecting k clients before score ordering to participate in subsequent interpretable operations.
  3. 3. The method for mitigating the effects of Non-IIDs based on interpretive federal learning of claim 1, wherein selecting clients for interpretive purposes in step (3) according to representative gradients includes taking the differences between the local model and the global model of the client as "representative gradients", "representative gradients" for each client are first calculated, and "representative gradients" are calculated as: Wherein And (3) with The method comprises the steps of respectively carrying out local updating parameters of an ith client and parameters of last global updating, gathering the clients into c classes through hierarchical clustering according to a representative gradient, sequentially extracting m clients participating in the gathering from the c classes, and carrying out subsequent interpretability operation.
  4. 4. The method for reducing the effects of Non-IID based on interpretive federal learning of claim 3, wherein selecting clients for hierarchical clustering in interpretive according to a representative gradient comprises: clustering operation is carried out by using a Ward method, a representative gradient is used as input data of clustering, and a distance matrix is created by using the representative gradient, N clusters are assumed, the square sum of residual Errors (ESS) in each cluster is 0, and the ESS calculation formula is as follows: ; sequentially calculating the ESS difference after combining every two clusters, namely - And finding two clusters with the minimum ESS difference degree after merging, and repeating the process.
  5. 5. The method for mitigating Non-IID effects based on interpretive federal learning of claim 1, wherein dynamically adjusting client model parameters includes inputting a verification set representing imbalance into an imbalance client model, and calculating a gradient distance between each imbalance client model and a previous round of global aggregate model, respectively, the gradient distance calculation formula being as follows: ; Wherein the method comprises the steps of The sample gradient of the kth verification set generated by the local model and the sample gradient of the kth verification set generated by the global model aggregated last time are respectively; the offset parameters of the client model are corrected by narrowing the gradient distance calculated by the above formula, and the number of iterations of the gradient is minimized according to the time flexible adjustment of federal learning aggregation.

Description

Method for relieving Non-IID influence based on interpretable federal learning Technical Field The invention belongs to the field of artificial intelligence safety, and particularly relates to a federal learning method based on interpretability for relieving influence caused by model performance reduction in Non-IID scenes. Background Research shows that with the continuous progress of artificial intelligence technology, the quality of data and the size of data volume become an important factor for restricting the development of an AI model. However, conventional centralized machine learning, which directly collects data of all parties, can cause serious privacy security problems. Thus, the advent of federal learning, which addresses this key problem, can cooperatively train out a shared global machine learning model while maintaining user data dispersion. However, as federal learning applications increase, the scenario facing the federal learning applications is more and more complex, and in real life, as each client has own preference, data of different clients have different characteristics, which can cause problems of reduced precision of federal learning models, slow convergence speed and the like. The conventional solution to the Non-IID problem in federal learning generally starts from two aspects, one is to optimize the weight difference of clients participating in aggregation, and the other is to optimize the selection of clients. However, optimizing the weight difference of the client mostly results in a decrease in convergence rate of the model, and optimizing the selection of the client increases excessive communication overhead and computation overhead. As various artificial intelligence models begin to make decisions instead of humans, it is critical to support the interpretation of the model's output. The initial purpose of the interpretability study is to improve the transparency of the model by constructing an interpretable model or designing an interpretation method, and simultaneously verify and evaluate the reliability and safety of the decision behavior and the decision result of the model, so that the potential safety hazard of the model in practical deployment application is eliminated. Therefore, the method utilizes the interpretability to solve the problem of model precision reduction caused by Non-IID in the federal learning scene in a more reasonable and interpretable way, so that the solution proposed based on the method has more practical significance and application value. Disclosure of Invention In order to solve the technical problems mentioned in the background art, the invention provides a federal learning method based on interpretability, which can effectively relieve the problem of federal learning model accuracy reduction caused by Non-IID scenes. In order to achieve the technical purpose, the technical scheme of the invention is as follows: A method for relieving Non-IID influence based on interpretability federal learning introduces a verification set interpretable mechanism based on the influence of local client update on various learning capabilities of an aggregation model. And the interpretation results of the verification samples are obtained through the descriptive verification, namely the influence of the local client on the global model learning capacity is obtained through the descriptive verification, and then the change condition of the model learning capacity of each type is judged. And evaluating the representative interpretable results of each class by using a Structural Similarity Index (SSIM), so as to infer the clients with unbalanced data. And then the parameters of the clients with unbalanced data are adjusted, and the distances between the parameters of the clients and the parameters of the aggregation model updated last time are reduced by iteration, so that the negative influence caused by unbalanced data is reduced. Further, the method comprises the following steps: (1) Constructing a Non-IID scene, namely distributing data of different numbers and different labels to each client by using Dirichlet distribution aiming at a target data set, and constructing a Non-IID scene; (2) A verification set is built, wherein a picture is placed in each category in a central server to serve as the verification set; (3) Selecting clients to perform interpretability, namely grading according to cosine similarity of an aggregated update direction in a local update direction of the clients, and selecting top-k clients with lower grades to participate in the interpretability; (4) The client with unbalanced data is screened, namely, the difference of the interpretable highlight feature images of the same verification set is calculated by comparing each client model participating in aggregation with the global model of the previous round of aggregation, and the data of the client is considered to be unbalanced if the variation amplitude of various similarities excee