Search

CN-122027193-A - Privacy protection collaborative learning robustness enhancement method based on zero knowledge proof

CN122027193ACN 122027193 ACN122027193 ACN 122027193ACN-122027193-A

Abstract

The invention relates to a privacy protection collaborative learning robustness enhancement method based on zero knowledge proof, belonging to the technical field of computer machine learning. Aiming at the technical pain points that the existing collaborative learning is difficult to consider the privacy protection of data and models and is easy to be attacked by model poisoning, the invention realizes the robustness enhancement through multi-equipment collaboration. The data owner and the model owner firstly complete key initialization and data set encryption promise, a forward propagation dividing linear/nonlinear calculation layer designs a verifiable protocol based on homomorphic encryption and zero knowledge proof to realize encryption calculation and authenticity verification, a reverse propagation cooperative calculation gradient guarantees gradient credibility through a secret sharing and proof mechanism, and model updating aggregates effective gradients through threshold decryption to filter malicious submission. The invention can resist the poisoning attack of the malicious data owners, the accuracy of the model is reduced by less than or equal to 4% under various attacks, the success rate of the gradient matching attack is nearly random, and the training precision is maintained while the privacy is protected.

Inventors

  • SHEN MENG
  • YU XIAOCHANG
  • PENG BOHAN
  • SU YUE
  • ZHU LIEHUANG

Assignees

  • 北京理工大学

Dates

Publication Date
20260512
Application Date
20251223

Claims (9)

  1. 1. The privacy protection collaborative learning robustness enhancement method based on zero knowledge proof is applied to a computer system for collaborative training of a machine learning model, and the computer system comprises a plurality of local computing devices of data owners, an aggregation server of the model owners and a key management device of a trusted authority, and is characterized in that: Step 1, initializing a system and configuring safety parameters, and completing key generation and encryption submission of a data set; The key initialization comprises the steps that a data owner generates a personal key pair through a Paillier password system of a local computing device, a trusted authority generates a global key pair through a key management device based on Threshold Paillier password systems, and the threshold value of the global key pair is set to be the number K+1 of the data owners; Step 1.2, data set promise, namely, after each data owner encrypts a local data set through a personal public key, submitting the local data set to an aggregation server and promises to participate in calculation by using the encrypted data set; the data provider always uses the promised data set to calculate in the whole system execution process; step2, safe forward propagation calculation; Based on the hardware adaptation characteristic of Paillier homomorphic encryption, encryption calculation and verification of neural network forward propagation are completed cooperatively between an aggregation server and local computing equipment, the propagation process is divided into a linear calculation layer and a nonlinear calculation layer, and calculation and verification of encrypted data are realized through a differential security protocol; The linear calculation layer performs calculation through Paillier homomorphism, and generates disturbance ciphertext by combining a secret sharing mechanism and interacts with the disturbance ciphertext; step 2.2, the nonlinear calculation layer designs a verifiable safety comparison protocol VSCP and a verifiable safety multiplication protocol VSMP based on a zero knowledge proof related sub-protocol; step 2.3, realizing encryption calculation and verification of a ReLU function and a Softmax function and a maximum pooling layer based on VSCP and VSMP; Step 3, secure back propagation calculation, namely, the encryption calculation of gradients ∇ X, ∇ b and ∇ W is completed cooperatively by the aggregation server and the local computing equipment, wherein ∇ W is verified cooperatively by a secret sharing technology and zero knowledge proof; And 4, updating the model, wherein the aggregation server discards the gradient which does not pass through verification, aggregates the effective gradient through a Threshold Paillier threshold decryption mechanism, updates the global model and iterates until convergence, and stops collaborative training at the moment and outputs a machine learning model which is finally trained.
  2. 2. The method for enhancing robustness of privacy-preserving collaborative learning based on zero-knowledge proof according to claim 1, wherein in step 1.1, the private key of the private key pair is stored in a secure storage area of the local computing device and is not transmitted externally, the global key pair comprises a global public key GPK, a group of private keys GSKk and a corresponding verification key GVKk, K is 1, k+1, and K is an integer of value [1, k+1], and the key management device is distributed to each participant device through a secure communication channel.
  3. 3. The privacy-preserving collaborative learning robustness enhancement method based on zero-knowledge proof as set forth in claim 1, wherein in step 2.1, the linear computation layer includes a convolution layer, a full-connection layer and an average pooling layer, the aggregation server performs ciphertext matrix operation on the encrypted input, generates a random mask construction disturbance ciphertext and sends the disturbance ciphertext to the data owner, the data owner obtains a partial intermediate result after decryption, and the aggregation server retains another partial intermediate result including the mask; Wherein, the convolution layer and the full connection layer are calculated as follows: Ciphertext matrix operation, namely directly executing [ WX ] calculation on encryption input [ X ] by a model owner, and realizing linear transformation of a weight matrix W and encryption characteristics by utilizing addition homomorphism of Paillier, wherein [ WX ] represents an intermediate result; Secret sharing mechanism by which the model owner calculates intermediate results Generating a random mask s by the model owner and constructing a disturbance ciphertext Sent to the data owner, and the data owner decrypts and obtains Model owner reservation B represents a random mask vector locally held by a model owner and is used for hiding a real intermediate result, so that privacy security in the gradient calculation process is ensured; And (3) average pooling layer calculation, namely directly executing homomorphic average calculation on the encryption feature map by a model owner, wherein the calculation result is also through the secret sharing mechanism.
  4. 4. The privacy-preserving collaborative learning robustness enhancement method based on zero-knowledge proof as set forth in claim 1, wherein in step 2.2, verifiable security comparison protocol VSCP includes a noise injection stage and a difference calculation verification stage, a model owner adds random noise to generate blind ciphertext, the data owner decrypts the calculated difference and submits the proof, and the model owner determines the comparison result after verification; In the noise injection stage, a model owner adds random noise [ s1] and random noise [ s2] to input ciphertext [ x ] and input ciphertext [ y ] respectively to generate blind ciphertext [ x ̃ ] = [ x+s1], [ ỹ ] = [ y+s2] and sends the blind ciphertext to a data owner; Verification of the Difference calculation data owner decryption calculation Submitting [ d ] and SubProof/OriginProof, and after verifying the validity of the verification, determining whether x > y is true or not by comparing the relation between d and s1-s 2; ; The verifiable safety multiplication protocol VSMP comprises a blind transmission stage and a multiplication verification stage, and the real product is recovered by removing noise through noise injection, product calculation and zero knowledge proof; In the blind transmission stage, the model owner generates noise [ s1, s2] for input ciphertext [ x ] and [ y ] respectively, and constructs [ x ̃ ] = [ x+s1], [ ỹ ] = [ y+s2] and sends the generated noise to the data owner; The multiplication verification stage comprises the steps of decrypting and calculating m= (x+s1) (y+s2) by a data owner for constructing multiplication evidence, submitting [ m ] and MulProof, removing noise influence after verification by a model owner, and recovering a real product : Wherein, the M represents the product intermediate result after adding noise.
  5. 5. The privacy-preserving collaborative learning robustness enhancement method based on zero-knowledge proof as set forth in claim 1, wherein in step 2.3, division verification of a Softmax function is converted into multiplication verification, and the exponent operation result and the operation relation of a numerator denominator are verified through the zero-knowledge proof; ReLU function computation is centered on a comparison between the input ciphertext [ x ] and 0, with Relu (x) equal to x if x >0, otherwise Relu (x) equal to 0; Constructing two auxiliary functions To verify Relu (Z): Wherein Z represents the linear transformation output of the current layer before activation, and w represents the weight vector corresponding to the layer; then, for all , , , , : Wherein, the The weight matrix of the first layer is represented, n represents the dimension of the input of the first layer, z represents the linear output obtained by multiplying the weight vector by the input vector, and x represents the corresponding input vector; the server obtains the ciphertext through homomorphic operation 、 : Model owner and data owner comparison through VSCP And (3) with Outputting the ReLU activation result for all the values )ε If (3) Then [ ]=[ ], =1, Otherwise [ ]=[ ], =0, Wherein, Respectively representing the ith sample at the jth neuron via (-) And The two obtained candidate linear outputs are used for realizing the activation judgment of the ReLU through comparison under the dense state; Softmax function calculation: The Softmax function is calculated as sigma ,σ Representing the output probability of the ith component of the input vector normalized by Softmax, Representing the ith component Then, the data owner calculates the output of the exponential function and transmits the ciphertext result to the model owner; To verify whether the data owner submits a forgery Zero knowledge proof ZKP and inadvertent transmission OT to identify counterfeit calculations: through cross validation To verify different pairs of Representing vectors obtained by performing exponential operation on the plaintext input x element by element; For any pair , A first set of exponent input vectors representing the submission of the data owner, Representing the corresponding second index input vector, the model owner is verified by the following steps : For each pair of Data owner calculation Sum [ Respective proofs of use AddProof and OriginProof; 、 representing two plaintext components in the pair of inputs, respectively; The model owner accepts k pairs of the model through OT and verifies that for each pair of the model owner's acceptance Model owner calculation ; Model owners announce accepted pairs; Model owner and data owner co-execution MulProof, allowing model owners to verify data owner submissions And Is indeed equal to the product of ; If the verification passes, the model owner obtains the rank of [ x ] through VSCP and then checks using VSCP Is a sequence of (2); converting verification of the division contained by the Softmax function into verification of the multiplication, where the model owner is known Known to the data owner The method comprises the following steps: data owner calculates denominator And molecules Satisfies the following conditions Then The ith output being a Softmax function, data owner send ]、[ To the model owner; a base representing natural logarithms; Representing the ith output probability of the scaled Softmax; representing molecules to be processed Dividing the remainder obtained by the denominator x; model owner performs homomorphic addition to obtain encryption denominator Performing homomorphic scalar multiplication to obtain a numerator ; Model owner and data owner operations VSCP verify that the satisfaction is met ; If verification passes, model owner calculation The model owner and data owner execute VSMP to verify whether it is satisfied ; Maximum pooling layer calculation: The core of the max-pooling layer is to perform ciphertext comparison by performing VSCP.
  6. 6. The zero-knowledge proof-based privacy-preserving collaborative learning robustness enhancement method of claim 1, wherein in step 3, the aggregation server-directed, local computing device cooperatively performs cryptographic computations of gradients ∇ X, ∇ b, and ∇ W during a back propagation phase: Step 3.1, model owner independent calculations ∇ X and ∇ b; Calculation of Model owner based on , wherein, Is the gradient of the upper layer and the gradient of the lower layer, Is a known plaintext weight matrix; Due to Is plaintext, the model owner can directly execute ciphertext-plaintext Wen Chengfa to obtain encryption ; Calculation of Model owner based on Wherein Is a constant, so the model owner can directly calculate ciphertext ; Step 3.2 model owner and data owner collaborative computing ; Due to The method involves multiplication of two ciphertexts, and a model owner cannot directly calculate the two ciphertexts; Model owner calculation Representing the result of the intermediate product constructed by the model owner in a dense state; model owner generates random number s, calculates And is sent to the data owner of the data, Representing a noise adding result obtained after the model owner adds the random number s to m; Decryption by the data owner And calculate The data owner will And Sent to the model owner (encrypted by ThPai) and appended with a proof MulProof that n is EquProof, prove decryption and re-encryption Sent with the model owner Consistent; Representing randomized ciphertext transmitted by a data owner to a model owner Decrypting and re-encrypting to obtain a ciphertext form; The model owner verifies the proof submitted by the data owner and if passed, calculates Representing the original product ciphertext form recovered by the model owner after removal of the random number s.
  7. 7. The privacy-preserving collaborative learning robustness enhancement method based on zero-knowledge proof as set forth in claim 1, wherein in step 4, each data owner generates a partial decryption value and a decryption proof based on the aggregation gradient, the aggregation server verifies the partial decryption result to reconstruct a global gradient, updates the model by W t+1 = Wt- η ∇ W, where η is a learning rate, Representing global model parameters obtained by training of the t+1 th round, Representing global model parameters at the time of the t-th training.
  8. 8. The privacy-preserving collaborative learning robustness enhancement method based on zero-knowledge proof of claim 1, wherein in step 2.3, the ReLU function is calculated by constructing filter+ and filter-auxiliary functions, candidate linear outputs are obtained by means of a key Wen Tongtai operation, and the activation result is determined by comparing the candidate output sizes through VSCP.
  9. 9. The privacy-preserving collaborative learning robustness enhancement method based on zero-knowledge proof of claim 1 wherein in step 1.2, the data provider owns a data set Comprising a feature matrix , And a label matrix , ; The real number domain is represented by the number, Representing feature dimensions, data provider using its personal public key Encrypting data Then send the encrypted data set [ , To the server, As a matrix of features, Is a tag matrix.

Description

Privacy protection collaborative learning robustness enhancement method based on zero knowledge proof Technical Field The invention relates to a privacy protection collaborative learning robustness enhancement method based on zero knowledge proof, which aims to verify the calculation result of a data owner by introducing a zero knowledge proof technology and enhance the capability of a privacy protection collaborative learning system for resisting poisoning attack, and belongs to the technical field of computer machine learning. Background In recent years, collaborative learning (Collaborative Learning) has become an important paradigm in the field of machine learning, with the core that a model Owner (model Owner del own) is able to train a high-quality model by using data of a plurality of data owners (Data Owners, data owners es). Typical applications of collaborative learning include disease diagnosis, where medical institutions co-train diagnostic models by using geographically distributed hospital data. By utilizing large-scale and diversified data sets, collaborative learning can significantly improve model accuracy. With the increasing attention paid to the data privacy and model privacy, how to protect the rights and interests of all parties in collaborative learning becomes a key technical problem to be solved urgently. In collaborative learning, the core of data privacy is to ensure that training data of a data owner is not compromised, preventing the model owner from reconstructing the original data through model updates (e.g., gradient matching attacks). At the same time, model privacy is aimed at protecting the exclusive ownership of the trained model by the model owner. Traditional federal learning (FEDERATED LEARNING, FL) is a widely adopted collaborative learning paradigm whose basic flow is for data owners to train models locally, with model owners responsible for aggregate updates. However, the model distribution mechanism of FL has an inherent drawback in that the model owner needs to distribute the global model to the data owners, which may lead to privacy leakage of the model. To solve the above problems, researchers have recently proposed a collaborative learning framework based on secure multiparty computing (Secure Multiparty Computation, MPC). These frameworks typically rely on multiple non-collusion servers to ensure data privacy and model privacy. However, there is a key assumption that servers cannot collude with each other. Once two or more servers collude, the security of the protocol will be completely destroyed. In addition, some schemes (e.g., pencil) avoid server collusion assumptions, but require that the data owner must be Semi-honest (Semi-honest), i.e., strictly adhere to the protocol specifications. In practical applications, however, malicious data owners may destroy the model performance by submitting counterfeit model updates (e.g., model poisoning attacks), resulting in significant degradation of model accuracy. Therefore, existing collaborative learning systems typically achieve privacy protection of data and models at the expense of being unable to verify data owners' updates, with a significant decrease in model accuracy in the face of model poisoning attacks by malicious data owners. How to design a collaborative learning system capable of resisting model poisoning attack of malicious data owners and guaranteeing model and data privacy at the same time becomes a key technical problem to be solved urgently. Disclosure of Invention The invention aims to improve the robustness of the existing privacy protection collaborative learning framework, and creatively provides a privacy protection collaborative learning robustness enhancing method based on zero knowledge proof. According to the method, from the perspective of secure multiparty calculation, by introducing a verifiable encryption calculation protocol, the calculation authenticity of a data owner is effectively ensured, a malicious participant is prevented from destroying a training process through forging update, the robustness of a collaborative learning framework is improved, and meanwhile, the data privacy and the model privacy are ensured. The innovation point of the invention comprises the steps of deeply analyzing the security threat of model poisoning attack in collaborative learning, providing a verifiable encryption protocol aiming at nonlinear calculation, and combining Paillier homomorphic encryption and zero knowledge proof technology to realize privacy protection and authenticity verification of gradient calculation. Through the dynamic noise injection and removal mechanism, the privacy of the model is protected, and meanwhile, the calculation process of a data owner is ensured to be verified, so that fake update submitted maliciously is effectively filtered, and the training precision of the model is maintained. In order to achieve the above purpose, the present invention is realized by the fo