CN-122021774-A - General federal learning reconstruction attack method aiming at disturbance gradient

CN122021774ACN 122021774 ACN122021774 ACN 122021774ACN-122021774-A

Abstract

The invention relates to the technical field of federal learning reconstruction attack, and provides a general federal learning reconstruction attack method aiming at disturbance gradients. The method specifically comprises forward propagation and backward propagation gradient solving, disturbance gradient generation, gradient weight calculation, gradient alignment calculation, feature extraction based on Taylor expansion and feature reconstruction based on double convolution. And aiming at the last full-connection layer of the federal learning model, carrying out forward propagation and backward propagation gradient solving, calculating the weights and alignment amounts of the weight gradient and the bias gradient by only considering non-target classes and a weighted average method, realizing the alignment of the disturbance weight gradient and the disturbance bias gradient based on a first-order Taylor expansion, extracting the features after the alignment, finally inputting the features into a double convolution reconstructor for reconstruction, and recovering to obtain the privacy training data of the client. The invention can effectively cope with complex and diverse disturbance strategies, recover the privacy training data of the client, and realize the universal federal learning reconstruction attack effect.

Inventors

ZHOU HUI
QIN ZHENG
SUN PENG
DENG XIN
ZHANG FEN
QIU JIANHUA

Assignees

湖南大学

Dates

Publication Date: 20260512
Application Date: 20260316

Claims (7)

1. The universal federal learning reconstruction attack method for disturbance gradient is characterized by comprising the following steps: (1) Forward propagation calculation and backward propagation gradient derivation are carried out on the last full-connection layer of the federal learning model, and the obtained weight gradient and bias gradient of the original gradient are used as reference quantity for alignment and feature extraction of the subsequent disturbance gradient; (2) Simulating a gradient disturbance process of a federal learning client, adding disturbance to an original gradient to obtain a disturbance gradient, and extracting disturbance gradient components from the disturbance gradient, wherein the disturbance gradient components comprise disturbance weight gradients and disturbance bias gradients; (3) The obtained disturbance weight gradient and disturbance bias gradient are subjected to weight calculation, and only non-target class gradients and corresponding weights are considered in the weight calculation process in order to avoid self-deviation introduced by using all class gradients; (4) Calculating the alignment of the disturbance weight gradient and the disturbance bias gradient by using a weighted average method; (5) The method comprises the steps of extracting features based on Taylor expansion, constructing a first-order Taylor expansion of disturbance weight gradient and a first-order Taylor expansion of disturbance bias gradient and bias gradient by utilizing the approximate characteristics of the first-order Taylor expansion, dividing the weight gradient and the bias gradient after obtaining two first-order Taylor expansion, and deducing to obtain the input features of a fully connected layer; (6) The method comprises the steps of obtaining input features of a full connection layer, taking the input features as input of a feature reconstructor after the input features of the full connection layer are obtained, reconstructing the features from the features to data, directly taking the obtained features as the input of the reconstructor if the feature reconstructor is trained, reconstructing the features, if the feature reconstructor is not trained, firstly constructing a double convolution conversion sequence comprising residual convolution and transpose convolution, wherein each conversion sequence comprises transpose convolution, residual convolution blocks and an activation function, each residual convolution block comprises convolution, batch normalization and activation functions, residual connection is further arranged between the transpose convolution and the activation functions, allowing operations of the batch normalization and activation functions in the middle to be skipped, retaining input feature information, and reconstructing after training is completed, and recovering to obtain privacy training data.
2. The general federal learning reconstruction attack method for perturbation gradients of claim 1, wherein the forward propagation and backward propagation gradient solution comprises the steps of: for the last fully connected layer (Fully Connected Layer, FC) of the model, its forward propagation process can be expressed as: Wherein the method comprises the steps of , , , The output, weight, input and bias of the full connection layer, respectively, the input features And weight matrix Matrix multiplication is performed, and then offset is added Obtaining the output of the layer ; The back propagation process carries out gradient solving and updating on the weight and the bias parameter, and the original gradient can be obtained through derivation by a chain-type derivative rule Is expressed as: Wherein, the Is the partial derivative of the loss function with respect to the weight matrix, Represents a fully-connected layer of the adhesive, In order to be a mean square error loss, In order to derive the sign of the deviation, Is the full connection layer The output values of the individual neurons are then, Is the sum of the weight matrix Weight vectors corresponding to the neurons; the original gradient The bias gradient expression of (2) is: Wherein, the Is the partial derivative of the loss function with respect to the bias vector, Is the full connection layer And the bias term corresponding to each neuron.
3. The general federal learning reconstruction attack method for perturbation gradients of claim 1, wherein the perturbation gradient generation comprises the steps of: For the original gradient Adding disturbances Obtaining disturbance gradient And respectively obtain disturbance weight gradients Disturbance bias gradient Disturbance weight gradient The expression of (2) is: Wherein, the A weight disturbance term of the corresponding dimension; Corresponding disturbance bias gradient The expression of (2) is: Wherein, the Is a bias perturbation term for the corresponding dimension.
4. The general federal learning reconstruction attack method for perturbation gradients of claim 1, wherein the gradient weight calculation comprises the steps of: Based on the disturbance weight gradient and the disturbance bias gradient of the full connection layer, calculating the gradient weight can obtain: Wherein, the Is a set of all categories; Since the perturbation gradients already contain noise, the direct use of all class gradients introduces self-bias affecting the accuracy of subsequent reconstruction, in order to avoid self-bias during computation, only non-target classes need to be considered The gradient of the non-target class gradient participates in subsequent calculation, and the expression of the non-target class gradient is as follows: Wherein, the For the true target class of the current sample, Is the full connection layer Disturbance weight gradient components corresponding to the non-target classes, Is the full connection layer Disturbance bias gradient components corresponding to the non-target classes; the weight expression corresponding to the non-target class gradient is: Wherein, the Non-target class for full connection layer The weights of the corresponding disturbance weight gradient components, Non-target class for full connection layer The corresponding perturbations bias the weights of the gradient components.
5. The general federal learning reconstruction attack method for perturbation gradients of claim 1, wherein the gradient alignment calculation comprises the steps of: The pair Ji Liangbiao of weight gradients has the expression: Wherein, the Representing the sum range is strictly limited to non-target class sets , Representing for each non-target class Multiplying the gradient of the class by the corresponding weight, then accumulating and summing, Adding the weights of the disturbance weight gradients of all non-target classes, and dividing the numerator denominator by the denominator as a normalization factor to obtain the alignment quantity of the disturbance weight gradients; the pair Ji Liangbiao of bias gradients has the expression: Wherein, the And adding the weights of all non-target disturbance bias gradient components, and dividing the weights by a numerator denominator to obtain the alignment of the disturbance bias gradient.
6. The general federal learning reconstruction attack method for perturbation gradients according to claim 1, wherein the feature extraction based on taylor expansion comprises the steps of: the loss function Regarding parameters The taylor expansion formula of (c) is: Wherein, the Is that At the parameters of The gradient of the light beam, In order to transpose the gradient, Is that At the parameters of A black plug matrix on the substrate, wherein the black plug matrix is arranged on the substrate, Is a parameter Is a minor variable of (2); the above formula is simplified by first order approximation to obtain a loss function Is a first order taylor expansion of (a): based on first-order Taylor expansion deformation A disturbance weight gradient can be obtained And weight gradient Is a first order taylor expansion of (a): Disturbance bias gradient With bias gradient Is a first order taylor expansion of (a): dividing the weight gradient and the bias gradient to obtain the input characteristics of the fully connected layer The expression is: 。
7. The general federal learning reconstruction attack method for perturbation gradients according to claim 1, wherein the feature reconstruction based on double convolution comprises the steps of: After obtaining the input features of the full connection layer, an attacker can select the input features according to the features Performing data reconstruction, if yes, a feature reconstructor After training, the server directly performs privacy data reconstruction, and the expression is: Wherein, the Feature reconstructor for finally reconstructed private data From a conversion sequence If not, firstly constructing a double-convolution conversion sequence for training by the server, wherein the expression of the double-convolution conversion sequence is as follows: wherein each conversion Involving transposed convolution Residual convolution block The function is activated and the residual convolution block contains convolutions Batch normalization And Activating the function, convolving at transpose And Residual connections are also provided between the activation functions, allowing intermediate skipping And And after training, reconstructing based on the input characteristics of the full-connection layer, thereby obtaining privacy training data and completing reconstruction attack.

Description

General federal learning reconstruction attack method aiming at disturbance gradient Technical Field The invention relates to the technical field of federal learning reconstruction attack, in particular to a general federal learning reconstruction attack method aiming at disturbance gradients. Background Federal learning, which is a data-free local distributed collaborative training model, effectively solves the problem of data islanding of traditional centralized training, and is widely applied in privacy sensitive fields such as finance, medical treatment and the like. However, after the local training is completed, the gradient uploaded to the server by the client is vulnerable to reconstruction attacks. An attacker can reconstruct and restore the privacy training data of the client by observing the acquired gradient, so that the client faces serious privacy threat. In view of this, the existing gradient disturbance strategy generates a disturbance gradient by adding noise, sparsification, clipping and other disturbance modes to the original gradient, so as to destroy the mapping relation between the gradient and the privacy training data, and seek to reduce the risk of data leakage. However, the existing reconstruction attack has significant limitations that most of the methods are designed aiming at a specific single disturbance protection method, cannot cope with complex, diverse and combined disturbance strategies, and lacks a universal disturbance reconstruction attack method. How to design a universal federal learning reconstruction attack method aiming at complex and diverse disturbance gradients is a technical problem to be solved in the field. Disclosure of Invention In order to solve the problem that the existing federal learning reconstruction attack method is difficult to cope with complex and various disturbance gradient strategies, the invention initiates a disturbance gradient feature extraction method based on Taylor expansion and a feature reconstructor based on double convolution, and accordingly provides a general federal learning reconstruction attack method. In order to solve the technical problems, the technical scheme of the invention is as follows: the invention provides a general federal learning reconstruction attack method aiming at disturbance gradient, which comprises the following steps: (1) In the training process of the federal learning model, the forward propagation is responsible for finishing the feature mapping and output calculation of the data, and the backward propagation realizes the iterative update of model parameters based on the gradient information of the loss function Wherein,,,The output, weight, input and bias of the full connection layer are respectively adopted, and the mean square error loss is adopted in the back propagation processThe weight and the bias parameters are subjected to gradient solving and updating, and the original gradient can be obtained respectively through derivation by a chain-type derivative ruleWeight gradient of (2)Bias gradient; (2) Disturbance gradient generation, for original gradientAdding disturbancesObtaining disturbance gradientAnd respectively obtain disturbance weight gradientsDisturbance bias gradientTwo disturbance gradient components respectively satisfyingAnd; (3) Gradient weight calculation, disturbance weight gradient to fully connected layerDisturbance bias gradientWeight calculation is carried out to obtainWhereinAs the disturbance gradient already contains noise, the self-deviation needs to be avoided in the calculation process, so that only non-target classes need to be considered for calculationGradient of (2)And corresponding weights; (4) Calculating the alignment of the weight gradients by a weighted average methodAlignment of offset gradients; (5) Feature extraction based on Taylor expansion, loss functionRegarding parametersTaylor expansion formula of (2)WhereinIs thatAt the parameters ofThe gradient of the light beam,In order to transpose the gradient,Is thatAt the parameters ofA black plug matrix on the substrate, wherein the black plug matrix is arranged on the substrate,Is a parameterThe equation is simplified by first order approximation to obtain a first order Taylor expansion of the loss functionBased on first-order Taylor expansionRespectively obtaining disturbance weight gradientsAnd weight gradientFirst order taylor expansion of (a)And perturbing the bias gradientWith bias gradientFirst order taylor expansion of (a)Finally, dividing the weight gradient and the bias gradient, and deducing to obtain the input characteristics of the full-connection layer; (6) Taking the full-connection layer characteristics obtained by the characteristic extraction in the step (5) as the input of a characteristic reconstructor to further obtain a final reconstruction data resultWherein the feature reconstructor is composed ofImplementation of individual conversion sequencesEach of which is converted intoInvolving tra