CN-122027232-A - Federal learning system and method based on optimized multiparty calculation

CN122027232ACN 122027232 ACN122027232 ACN 122027232ACN-122027232-A

Abstract

The invention discloses a federal learning system based on optimized multiparty calculation, which comprises a local processing module, a gradient updating module and a global aggregation gradient, wherein the local processing module is based on global model parameters of a central aggregation platform, trains a local model of each participant platform to obtain a model updating gradient, compresses and safely transforms to obtain a transformed gradient vector, each participant platform generates a random scalar mask and performs encryption operation, and meanwhile generates identity authentication information, the gradient aggregation module is used for verifying the identity of each participant platform through the central aggregation platform, then performs plaintext aggregation on the transformed gradient vector of all participant platforms which pass verification, decrypts the encrypted mask parameters through safe multiparty calculation, then performs safe aggregation, and finally obtains the global aggregation gradient through inverse transformation reconstruction, and the gradient updating module is used for updating the local model through each participant platform by utilizing the global aggregation gradient. The invention ensures the security of data privacy and improves the calculation efficiency of federal learning.

Inventors

ZHAO WEICHENG
CHANG WENTAO
CAO XIN
CAO MAOSEN
PENG WENJIE
WANG WENTAO
WANG LINGSHUANG
QIN YIWEN
HAN ZHI
CHEN JIAYUAN
XIE LI
MA HAOYUAN

Assignees

国网四川省电力公司营销服务中心
四川蜀能电力有限公司检验检测分公司

Dates

Publication Date: 20260512
Application Date: 20260119

Claims (10)

1. A federal learning system based on optimized multiparty computing, comprising: The local processing module is used for training the local model corresponding to each participant platform based on global model parameters of the central aggregation platform to obtain a corresponding model update gradient, compressing the model update gradient of each participant platform to obtain a quantized sparse gradient, performing security transformation on the quantized sparse gradient to obtain a transformed gradient vector, generating a random scalar mask by each participant platform, performing encryption operation to obtain an encryption mask parameter, and generating identity authentication information of each participant platform; The gradient aggregation module is used for carrying out identity verification on each participant platform by using the identity authentication information through the central aggregation platform, carrying out plaintext aggregation on the gradient vectors after the transformation of all participant platforms pass verification to obtain a global aggregation transformation gradient, decrypting encryption mask parameters of all participant platforms pass verification by using secure multiparty calculation, carrying out secure aggregation on the obtained plaintext after decryption to obtain a global mask aggregation gradient, and carrying out inverse transformation reconstruction on the global aggregation transformation gradient and the global mask aggregation gradient to obtain a global aggregation gradient; the gradient updating module is used for updating the local model of the participant platform by utilizing the global aggregation gradient to obtain updated local model parameters.
2. The federal learning system based on optimized multiparty computing of claim 1, wherein training the local model corresponding to each participant platform based on global model parameters of the central aggregation platform to obtain a corresponding model update gradient comprises the following steps: Each participant platform Receiving global model parameters issued by a central aggregation platform, wherein the global model parameters are the current global model of the central aggregation platform Weights of each participant platform Using local data sets Random gradient descent training is carried out on the local model of the self-body to obtain an updated local model: Wherein, the The global model updated for the t-th round of local model of the central aggregation platform, For the local data set of the ith participant platform, Representing a k-round random gradient descent training in the local model, The local model updated for the ith participant platform; Compensating residual errors introduced by sparsification in the t-1 th round of local model updating process: wherein The sparsified residuals representing the t-1 th round local model update, Representing the model update gradient after compensation; thereby calculating the model update gradient: Wherein, the The model update gradient of the local model of the ith participant platform after the t-th round of update is represented.
3. The federal learning system based on optimized multiparty computing of claim 2, wherein the compressing the model update gradient of each participant platform to obtain the quantized sparse gradient comprises the following steps: updating gradients to models Sequentially executing sparsification processing and random quantization processing to generate a compression gradient vector: Model update gradient Performing sparsification treatment to obtain sparse gradients : Wherein K is a preset positive integer superparameter for representing model update gradient The number of elements retained in the sparsification process; representing a sparsity selection algorithm; Further compression processing, for sparse gradients Performing random quantization processing to obtain quantized sparse gradient Wherein quantized sparse gradients for each element j The method comprises the following steps: Wherein, the A sparse gradient to be quantized for element j, For a sparse gradient quantized for element j, For a preset quantization step size, Representing a down-rounding operation, Representing a rounding up operation.
4. The federal learning system based on optimized multiparty computation of claim 3, wherein security transformation of quantized sparse gradients to transformed gradient vectors, each participant platform generating a random scalar mask and performing encryption operations to obtain encryption mask parameters comprises: performing random orthogonal transformation on quantized sparse gradients, and using randomly generated orthogonal matrix Random scalar mask Calculating to obtain a gradient vector after transformation : Wherein, the In order to transform the post-gradient vector, Is randomly generated Is a matrix of orthogonality of the (c), For the quantized sparse gradient, A random scalar mask independently generated for the ith participant platform, An M-dimensional column vector with all elements being 1; Masking random scalar with semantic security encryption function based on discrete logarithm problem Performing encryption operation to obtain encryption mask parameters : Wherein, the Representing a first portion of encryption mask parameters , Representing a shared mask in the encryption process, Representing group elements containing masked plaintext information, Representing a second portion of the encryption mask parameters , An encryption function representing semantic security; each participant platform transforms the transformed gradient vector And encryption mask parameters Uploading to a central aggregation platform.
5. The federal learning system based on optimized multiparty computing of claim 4, wherein each participant platform generates its own identity authentication information, and the specific process of verifying the identity of each participant platform by using the identity authentication information through the central aggregation platform is as follows: computing each participant platform Shared secret value with the central aggregation platform S : Wherein, the For the ith participant platform Is used to store the private key of (a), Representing the public key of the central aggregation platform S, Representing the computation of a shared key, H represents a key derivation hash function, For calculating a shared secret value; shared secret value In combination with the current timestamp Generating a hash function by authenticating a token Generating identity authentication tokens : Wherein, the Representing an authentication token for the ith participant platform, Representing the current timestamp of the ith participant platform at the t-th round of update, Representing that the authentication token generates a hash function; Each participant platform Transform the gradient vector Encryption mask parameters Identity authentication token And a time stamp Merging into unified data packets Uploading to a central aggregation platform: for each data packet received The central aggregation platform utilizes its own private key And each stored participant platform Public key of (a) Calculating each participant platform Is a desired authentication token of (1) : Wherein, the As a private key of the central aggregation platform S, For the ith participant platform Is used to determine the public key of (a), For calculating a shared secret value, The desired shared secret value calculated independently for the central aggregation platform, Representing a shared secret value to be expected With time stamps Forming a new character string by sequentially connecting the character strings end to end; Central aggregation platform versus ith participant platform Identity authentication token of (a) And expecting an authentication token If the same, the ith participant platform Passes the authentication of (a).
6. The federal learning system based on optimized multiparty computation of claim 5, wherein performing plaintext aggregation on transformed gradient vectors of all participant-platforms that pass verification to obtain a global aggregated transform gradient, decrypting encrypted mask parameters of all participant-platforms that pass verification using secure multiparty computation, and performing secure aggregation on decrypted plaintext to obtain a global mask aggregate gradient, performing inverse transformation on the global aggregated transform gradient and the global mask aggregate gradient to reconstruct to obtain a global aggregate gradient comprises the steps of: extracting gradient vectors after transformation for all participant platforms passing verification Weighting plaintext aggregation is carried out according to the aggregation weight of each participant platform to obtain a global aggregation transformation gradient : Wherein, the Representing an ith participant platform Is a weight of aggregation of (1); For encryption mask parameters Decryption using secure multiparty computation, the central aggregation platform uses its own private key Calculation of Wherein Representing the private key of the central aggregation platform, A first portion of the encryption mask parameters is represented, Representing a first portion of encryption mask parameters R i denotes the ith participant platform The random number to be generated is a random number, Representing the shared masking factor used to eliminate randomness during decryption, Representing the first part of the encryption mask parameters by the central aggregation platform using its own private key The power operation is performed so that the power of the light, Representation of Is a specific development of (a); For encryption mask parameters Decrypting to obtain the mask plaintext at the index position : Wherein, the Representing the masking item introduced during the encryption process, Representing the decryption factor; And then the discrete logarithm solving algorithm is used for solving the clear text from the mask In recovering the plaintext : For plaintext Weighted security aggregation to obtain global mask aggregation gradient : Transforming gradients from global aggregation Subtracting the global mask aggregation gradient from Is obtained by: Wherein, the For an M-dimensional column vector with all elements being 1, Is randomly generated Is a matrix of orthogonality; Then performing inverse transformation of random orthogonal transformation, and reconstructing to obtain global aggregation gradient : Wherein, the Is an orthogonal matrix Is a transpose of (a).
7. The federal learning system based on optimized multiparty computing of claim 1, wherein the local model itself is updated by each participant platform using the global aggregation gradient, and the specific process of obtaining updated local model parameters is: Global aggregation gradient to be reconstructed by the central aggregation platform Issued to each participant platform, each participant platform utilizing a global aggregation gradient Updating the corresponding local model to obtain updated local model parameters, wherein the formula is as follows: Wherein, the As a local model parameter for the current t-turn, In order to update the local model parameters, The learning rate of the current t rounds.
8. A federal learning system for neural network models including nonlinear activation functions using optimized multiparty calculations, characterized by: The local processing module is used for training the neural network model corresponding to each participant platform based on the global model parameters of the central aggregation platform to obtain a corresponding model update gradient, compressing the model update gradient of each participant platform to obtain a quantized sparse gradient, performing security transformation on the quantized sparse gradient to obtain a transformed gradient vector, generating a random scalar mask by each participant platform and performing encryption operation to obtain an encryption mask parameter, converting an input value of a nonlinear activation function in the neural network model into a secret shared value, and generating identity authentication information of each participant platform; The gradient aggregation module is used for carrying out identity verification on each participant platform through the central aggregation platform by utilizing the identity authentication information, carrying out plaintext aggregation on the gradient vectors after the transformation of all participant platforms pass verification to obtain a global aggregation transformation gradient, decrypting encryption mask parameters of all participant platforms pass verification by using safe multiparty computation, carrying out safe aggregation on the obtained plaintext after decryption to obtain a global mask aggregation gradient, carrying out inverse transformation reconstruction on the global aggregation transformation gradient and the global mask aggregation gradient to obtain a global aggregation sparse gradient of the neural network model, carrying out polynomial aggregation computation on secret sharing values of a nonlinear activation function by utilizing an optimized multiparty computation protocol and pre-generated auxiliary data to obtain a nonlinear aggregation gradient of the neural network model, and obtaining the global aggregation gradient based on the global aggregation sparse gradient of the neural network model and the nonlinear aggregation gradient of the neural network model; the gradient updating module is used for updating the neural network model by utilizing the global aggregation gradient through each participant platform to obtain updated neural network model parameters.
9. The federal learning system for neural network models incorporating nonlinear activation functions, employing optimized multiparty computation of claim 8, wherein the specific process of deriving the global aggregation gradient comprises: generating and distributing in secret sharing manner a set of multivariate Beaver triples related to the highest power K to be solved as helper data for all participant platforms in advance The auxiliary data form is: Wherein, the For K independent random values Is a shared set of secrets of (c), A shared set of secrets multiplied by two for any two random values, A shared set of secrets multiplied by any j random values, Secret sharing for the total product of K random values; i.e. auxiliary data Secret sharing including random values by K independence And secret sharing of all possible cross product terms composed of random values; input values for nonlinear activation functions Each participant platform utilizes a pre-shared random value For input values Masking to calculate the difference Secret shared value of (2) : Wherein, the Intermediate data representing the nonlinear activation function processing for the input value; Is a random value; in a unique communication round, all participant platforms simultaneously disclose and reconstruct the plaintext result of the difference: ; Each participant platform based on the same difference value as disclosed Pre-shared assistance data Locally computing all power terms in parallel by linear operation Secret shared value of (2), wherein Input value representing nonlinear activation function Secret sharing results to the K-th power; Each participant platform is based on secret sharing value of each power term and preset polynomial coefficient And performing linear combination to obtain a secret sharing value of the nonlinear activation function calculation result: Aiming at the secret sharing value of the nonlinear activation function, the optimized multipartite computing protocol and the pre-generated auxiliary data are utilized to execute polynomial aggregation computation based on secret sharing to obtain a nonlinear aggregation gradient, the global aggregation sparse gradient of the neural network model is directly used for global model updating, and the nonlinear aggregation gradient is used for influencing the model updating gradient generated by neural network model training through the nonlinear activation function, so that the global aggregation gradient is obtained.
10. The federal learning method based on optimized multiparty calculation is characterized by comprising the following steps: Training the local model corresponding to each participant platform based on global model parameters of a central aggregation platform to obtain corresponding model update gradients, compressing the model update gradients of each participant platform to obtain quantized sparse gradients, performing security transformation on the quantized sparse gradients to obtain transformed gradient vectors, generating random scalar masks by each participant platform, performing encryption operation to obtain encryption mask parameters, and generating identity authentication information of each participant platform; Carrying out identity verification on each participant platform by using the identity authentication information through a central aggregation platform, carrying out clear text aggregation on the gradient vectors after the transformation of all participant platforms pass verification to obtain a global aggregation transformation gradient, decrypting encryption mask parameters of all participant platforms passing verification by using secure multiparty calculation, carrying out secure aggregation on the clear text obtained after decryption to obtain a global mask aggregation gradient, and carrying out inverse transformation reconstruction on the global aggregation transformation gradient and the global mask aggregation gradient to obtain a global aggregation gradient; And updating the local model by each participant platform by utilizing the global aggregation gradient to obtain updated local model parameters.

Description

Federal learning system and method based on optimized multiparty calculation Technical Field The invention relates to the technical field of multiparty safety calculation, in particular to a federal learning system and a federal learning method based on optimized multiparty calculation. Background Federal learning is used as a distributed machine learning paradigm, which allows multiple participant platforms to jointly train a model under the condition of not sharing local original data, and can solve the problem of 'data island' in a mode of not sharing the original data, but model parameters or gradient information exchanged in the federal learning training process can be used for deducing sensitive data of the participant platforms, so that the prior art usually introduces secure multiparty computing to protect gradient confidentiality, for example, the local gradient is split into secret shares through an arithmetic secret sharing technology and then security aggregation is carried out, however, the federal learning scheme based on the secure multiparty computing has significant performance bottlenecks, multiple rounds of communication interaction are needed when the secure multiparty computing is used for realizing nonlinear operations such as multiplication, the training process is time-consuming in an environment with high network delay, and nonlinear activation functions widely used in a complex model are needed to be realized through computation intensive polynomial approximation under the secure multiparty computing framework, further aggravated in computation and communication delay, and the secure multiparty computing framework is not complete, and lacks authentication on participant identities, so that the system is easy to fight against attack such as model throwing or data pollution, and part of scheme is easy to solve the security risk of a single point of a trusted party. Disclosure of Invention The invention aims to provide a federal learning system and a federal learning method based on optimized multiparty calculation, the invention obviously improves the calculation efficiency and the safety of federal learning while guaranteeing the data privacy safety. To achieve the object, the invention provides a federal learning system based on optimized multiparty computation, which comprises: The local processing module is used for training the local model corresponding to each participant platform based on global model parameters of the central aggregation platform to obtain a corresponding model update gradient, compressing the model update gradient of each participant platform to obtain a quantized sparse gradient, performing security transformation on the quantized sparse gradient to obtain a transformed gradient vector, generating a random scalar mask by each participant platform, performing encryption operation to obtain an encryption mask parameter, and generating identity authentication information of each participant platform; The gradient aggregation module is used for carrying out identity verification on each participant platform by using the identity authentication information through the central aggregation platform, carrying out plaintext aggregation on the gradient vectors after the transformation of all participant platforms pass verification to obtain a global aggregation transformation gradient, decrypting encryption mask parameters of all participant platforms pass verification by using secure multiparty calculation, carrying out secure aggregation on the obtained plaintext after decryption to obtain a global mask aggregation gradient, and carrying out inverse transformation reconstruction on the global aggregation transformation gradient and the global mask aggregation gradient to obtain a global aggregation gradient; the gradient updating module is used for updating the local model of the participant platform by utilizing the global aggregation gradient to obtain updated local model parameters. Preferably, based on global model parameters of the central aggregation platform, training the local model corresponding to each participant platform to obtain a corresponding model update gradient comprises the following specific processes: Each participant platform Receiving global model parameters issued by a central aggregation platform, wherein the global model parameters are the current global model of the central aggregation platformWeights of each participant platformUsing local data setsRandom gradient descent training is carried out on the local model of the self-body to obtain an updated local model: Wherein, the The global model updated for the t-th round of local model of the central aggregation platform,For the local data set of the ith participant platform,Representing a k-round random gradient descent training in the local model,The local model updated for the ith participant platform; Compensating residual errors introduced by sparsification in the t-1 th round of l