CN-116306990-B - Model processing method, device and equipment

CN116306990BCN 116306990 BCN116306990 BCN 116306990BCN-116306990-B

Abstract

The embodiment of the specification discloses a processing method, a device and equipment of a model, wherein the method comprises the steps of receiving a target model issued by a server, obtaining sample data used for training the target model, carrying out model training on the target model based on the sample data, determining gradient information corresponding to the target model after model training, carrying out symbolizing processing on the gradient information corresponding to the target model to obtain symbol gradient information corresponding to the target model, carrying out sampling processing on the symbol gradient information based on a preset probability distribution rule, carrying out coding processing on the sampled symbol gradient information based on the probability distribution rule to obtain differential privacy gradient information, and sending the differential privacy gradient information to the server, wherein the differential privacy gradient information is used for triggering the server to update the target model stored in the server based on the differential privacy gradient information.

Inventors

SHEN SHUHENG
ZHANG CHANGHAO
FU XINYI
WANG WEIQIANG

Assignees

支付宝(杭州)信息技术有限公司

Dates

Publication Date: 20260505
Application Date: 20230310

Claims (16)

1. A method of processing a model, the method comprising: Receiving a target model issued by a server; acquiring sample data for training the target model, carrying out model training on the target model based on the sample data, and determining gradient information corresponding to the target model after model training; Symbolizing the gradient information corresponding to the target model to obtain symbol gradient information corresponding to the target model, sampling the symbol gradient information based on a preset probability distribution rule, and carrying out coding processing on the sampled symbol gradient information by adopting a random response mechanism based on the probability distribution rule to obtain differential privacy gradient information, wherein the symbolizing processing is to compress the value of each dimension in the gradient information from a floating point type value to a designated symbol, and optimize the target model by using the symbol gradient information containing the designated symbol; and sending the differential privacy gradient information to the server, wherein the differential privacy gradient information is used for triggering the server to update the target model stored in the server based on the differential privacy gradient information.
2. The method of claim 1, wherein the model training the target model based on the sample data, determining gradient information corresponding to the target model after model training, comprises: model training is carried out on the target model based on the sample data, and initial gradient information corresponding to the target model is obtained; And acquiring historical gradient information of the target model, and determining gradient information corresponding to the target model through an adaptive optimization algorithm based on the historical gradient information and the initial gradient information, wherein the adaptive optimization algorithm comprises an Adam algorithm or Adagrad algorithm.
3. The method according to claim 2, wherein the determining gradient information corresponding to the target model by an adaptive optimization algorithm based on the historical gradient information and the initial gradient information includes: Determining first-order momentum and second-order momentum corresponding to the historical gradient information based on the historical gradient information; And determining first-order momentum and second-order momentum corresponding to the initial gradient information based on the first-order momentum and second-order momentum corresponding to the historical gradient information and the initial gradient information corresponding to the target model, and determining gradient information corresponding to the target model based on the first-order momentum and second-order momentum corresponding to the initial gradient information.
4. The method of claim 3, wherein the determining the first and second order momentums corresponding to the initial gradient information based on the first and second order momentums corresponding to the historical gradient information and the initial gradient information corresponding to the target model comprises: Determining first-order momentum corresponding to the initial gradient information based on the first-order momentum corresponding to the historical gradient information and the initial gradient information corresponding to the target model; and determining the second-order momentum corresponding to the initial gradient information based on the second-order momentum corresponding to the historical gradient information and the first-order momentum corresponding to the initial gradient information.
5. The method according to any one of claims 1-4, wherein the symbolizing the gradient information corresponding to the target model to obtain symbolized gradient information corresponding to the target model includes: And setting the gradient information of which the value in each axis is larger than a preset threshold value as a first symbol, and setting the gradient information of which the value in each axis is smaller than the preset threshold value as a second symbol, so as to obtain the symbol gradient information of which the value in each axis is different from the first symbol.
6. The method of claim 5, wherein the predetermined threshold is 0, the first symbol is +1, and the second symbol is-1.
7. The method of claim 6, wherein the sampling the symbol gradient information based on a preset probability distribution rule, and encoding the sampled symbol gradient information by using a random response mechanism based on the probability distribution rule to obtain differential privacy gradient information, comprises: sampling the symbol gradient information based on a preset probability distribution rule to obtain sampled symbol gradient information formed by a plurality of sampling points; For each sampling point in the sampled symbol gradient information, the following processing is performed to obtain differential privacy gradient information: if a first probability of two different probabilities contained in the probability distribution rule corresponding to the sampling point is determined based on the probability distribution rule, and the sampling point is the second symbol, deleting the second symbol of the sampling point; And if the probability distribution rule is based on which the sampling point corresponds to the second probability of two different probabilities contained in the probability distribution rule, re-sampling the sampling point, and re-assigning the symbol gradient information of the re-sampled sampling point according to the corresponding probability contained in the probability distribution rule.
8. The method of claim 7, the probability distribution rule comprising a bernoulli distribution, the first probability being a probability corresponding to 0 in the bernoulli distribution, the second probability being a probability corresponding to 1 in the bernoulli distribution, the reassigning sign gradient information of resampled sample points according to respective probabilities contained in the probability distribution rule comprising: setting the symbol gradient information of the resampled sampling point to 0 if the resampled sampling point corresponds to a first probability; and if the resampled sampling point corresponds to the second probability, setting the symbol gradient information of the resampled sampling point to be 1.
9. The method of claim 1, the sample data being historical transaction data between different users, the target model being a risk prevention model for a preset risk, the preset risk including one or more of fraud risk, illegal transaction risk.
10. The method according to claim 1, wherein the target model is a model issued when the server updates the target model stored in the server through the differential privacy gradient information sent by the received terminal device and determines that the updated target model does not meet a preset convergence condition.
11. A method of processing a model, the method comprising: Receiving differential privacy gradient information of a target model uploaded by terminal equipment, wherein the differential privacy gradient information is obtained by the terminal equipment through model training on the basis of acquired sample data for training the target model, determining gradient information corresponding to the target model after model training, carrying out symbolizing processing on the gradient information corresponding to the target model to obtain symbol gradient information corresponding to the target model, carrying out sampling processing on the symbol gradient information on the basis of a preset probability distribution rule, and carrying out coding processing on the sampled symbol gradient information by adopting a random response mechanism on the basis of the probability distribution rule, wherein the symbolizing processing is the information obtained by compressing the numerical value of each dimension in the gradient information from a floating point type numerical value to a designated symbol, and carrying out optimizing processing on the target model by using the symbol gradient information containing the designated symbol; Performing aggregation treatment on the received differential privacy gradient information to obtain aggregated differential privacy gradient information; Decoding the aggregated differential privacy gradient information to obtain corresponding target symbol gradient information; and updating the target model based on the target symbol gradient information to obtain an updated target model.
12. The method of claim 11, wherein the probability distribution rule includes two different probabilities, and the decoding the aggregated differential privacy gradient information to obtain corresponding target symbol gradient information includes: Obtaining a corresponding calculation result through a preset algorithm based on the aggregated differential privacy gradient information, the quantity of the differential privacy gradient information subjected to aggregation treatment and two different probabilities contained in the probability distribution rule; And determining the target symbol gradient information based on the obtained calculation result.
13. A device for processing a model, the device comprising: the model receiving module is used for receiving the target model issued by the server; The gradient acquisition module is used for acquiring sample data for training the target model, carrying out model training on the target model based on the sample data and determining gradient information corresponding to the target model after model training; The differential privacy module is used for carrying out symbolization processing on gradient information corresponding to the target model to obtain symbol gradient information corresponding to the target model, carrying out sampling processing on the symbol gradient information based on a preset probability distribution rule, carrying out coding processing on the sampled symbol gradient information by adopting a random response mechanism based on the probability distribution rule to obtain differential privacy gradient information, wherein the symbolization processing is carried out by compressing the numerical value of each dimension in the gradient information from a floating point type numerical value to a designated symbol, and carrying out optimization processing on the target model by using the symbol gradient information containing the designated symbol; And the gradient sending module is used for sending the differential privacy gradient information to the server, and the differential privacy gradient information is used for triggering the server to update the target model stored in the server based on the differential privacy gradient information.
14. A device for processing a model, the device comprising: The differential gradient receiving module is used for receiving differential privacy gradient information of a target model uploaded by terminal equipment, the differential privacy gradient information is obtained by the terminal equipment through model training on the basis of acquired sample data for training the target model, determining gradient information corresponding to the target model after model training, carrying out symbolizing processing on the gradient information corresponding to the target model to obtain symbol gradient information corresponding to the target model, carrying out sampling processing on the symbol gradient information on the basis of a preset probability distribution rule, carrying out coding processing on the sampled symbol gradient information by adopting a random response mechanism on the basis of the probability distribution rule, and carrying out optimizing processing on the target model by using the symbol gradient information containing the appointed symbol; The aggregation module is used for carrying out aggregation treatment on the received differential privacy gradient information to obtain aggregated differential privacy gradient information; the decoding module is used for decoding the aggregated differential privacy gradient information to obtain corresponding target symbol gradient information; And the model updating module is used for updating the target model based on the target symbol gradient information to obtain an updated target model.
15. A processing apparatus of a model, the processing apparatus of the model comprising: processor, and A memory arranged to store computer executable instructions that, when executed, cause the processor to: Receiving a target model issued by a server; acquiring sample data for training the target model, carrying out model training on the target model based on the sample data, and determining gradient information corresponding to the target model after model training; Symbolizing the gradient information corresponding to the target model to obtain symbol gradient information corresponding to the target model, sampling the symbol gradient information based on a preset probability distribution rule, and carrying out coding processing on the sampled symbol gradient information by adopting a random response mechanism based on the probability distribution rule to obtain differential privacy gradient information, wherein the symbolizing processing is to compress the value of each dimension in the gradient information from a floating point type value to a designated symbol, and optimize the target model by using the symbol gradient information containing the designated symbol; and sending the differential privacy gradient information to the server, wherein the differential privacy gradient information is used for triggering the server to update the target model stored in the server based on the differential privacy gradient information.
16. A processing apparatus of a model, the processing apparatus of the model comprising: processor, and A memory arranged to store computer executable instructions that, when executed, cause the processor to: Receiving differential privacy gradient information of a target model uploaded by terminal equipment, wherein the differential privacy gradient information is obtained by the terminal equipment through model training on the basis of acquired sample data for training the target model, determining gradient information corresponding to the target model after model training, carrying out symbolizing processing on the gradient information corresponding to the target model to obtain symbol gradient information corresponding to the target model, carrying out sampling processing on the symbol gradient information on the basis of a preset probability distribution rule, and carrying out coding processing on the sampled symbol gradient information by adopting a random response mechanism on the basis of the probability distribution rule, wherein the symbolizing processing is the information obtained by compressing the numerical value of each dimension in the gradient information from a floating point type numerical value to a designated symbol, and carrying out optimizing processing on the target model by using the symbol gradient information containing the designated symbol; Performing aggregation treatment on the received differential privacy gradient information to obtain aggregated differential privacy gradient information; Decoding the aggregated differential privacy gradient information to obtain corresponding target symbol gradient information; and updating the target model based on the target symbol gradient information to obtain an updated target model.

Description

Model processing method, device and equipment Technical Field The present document relates to the field of computer technologies, and in particular, to a method, an apparatus, and a device for processing a model. Background The way in which user data is used to train a model on a cloud server and to use the model for servicing users is already quite common in current internet services, however, this way necessarily relies on the collection and analysis of user privacy data by the cloud server. At present, laws and regulations related to privacy security of user data are mature, and security of the user to the privacy data is paid more attention to, so that a paradigm of data analysis processing at a server side is not suitable for current data processing requirements. Aiming at the situation, a distributed machine learning technology, namely federal learning, is provided, most of calculation processes and data can be kept locally for users through federal learning, gradient information required by model updating is sent to a cloud server instead of original user data, and the user data is available and invisible. However, when the gradient information is uploaded to the server, intermediate gradient information is included, an attacker can analyze through the intermediate gradient information and restore a part of original data, so that private data is revealed. Therefore, a technical scheme with better privacy protection capability and better communication overhead is needed to be provided. Disclosure of Invention The embodiment of the specification aims to provide a technical scheme with better privacy protection capability and better communication overhead. In order to achieve the above technical solution, the embodiments of the present specification are implemented as follows: The embodiment of the specification provides a method for processing a model, which comprises the steps of receiving a target model issued by a server. Sample data for training the target model is obtained, model training is carried out on the target model based on the sample data, and gradient information corresponding to the target model after model training is determined. Symbolizing the gradient information corresponding to the target model to obtain the symbol gradient information corresponding to the target model, sampling the symbol gradient information based on a preset probability distribution rule, and encoding the sampled symbol gradient information based on the probability distribution rule to obtain differential privacy gradient information. And sending the differential privacy gradient information to the server, wherein the differential privacy gradient information is used for triggering the server to update the target model stored in the server based on the differential privacy gradient information. The method for processing the model comprises the steps of receiving differential privacy gradient information of a target model uploaded by terminal equipment, wherein the differential privacy gradient information is obtained by the terminal equipment through model training on the basis of acquired sample data for training the target model, determining gradient information corresponding to the target model after model training, carrying out symbolizing processing on the gradient information corresponding to the target model to obtain symbol gradient information corresponding to the target model, carrying out sampling processing on the symbol gradient information on the basis of a preset probability distribution rule, and carrying out coding processing on the sampled symbol gradient information on the basis of the probability distribution rule. And carrying out aggregation treatment on the received differential privacy gradient information to obtain aggregated differential privacy gradient information. And decoding the aggregated differential privacy gradient information to obtain corresponding target symbol gradient information. And updating the target model based on the target symbol gradient information to obtain an updated target model. The device for processing the model provided by the embodiment of the specification comprises a model receiving module for receiving the target model issued by the server. The gradient acquisition module is used for acquiring sample data for training the target model, carrying out model training on the target model based on the sample data, and determining gradient information corresponding to the target model after model training. And the differential privacy module is used for carrying out symbolization processing on the gradient information corresponding to the target model to obtain symbol gradient information corresponding to the target model, carrying out sampling processing on the symbol gradient information based on a preset probability distribution rule, and carrying out coding processing on the sampled symbol gradient information based on the probability distribution rule to