CN-122020448-A - Privacy training data leakage risk detection method for sequence recommendation system

CN122020448ACN 122020448 ACN122020448 ACN 122020448ACN-122020448-A

Abstract

The invention discloses a privacy training data leakage risk detection method for a sequence recommendation system, and belongs to the technical fields of a sequence recommendation model and privacy protection. Aiming at the problems that the sequence recommendation model has a privacy training data leakage risk and the risk is difficult to evaluate accurately in the Top-K label black box scene by the existing method, the method comprises the steps of firstly obtaining a target model and an auxiliary data set, dividing the target model into the target model and the auxiliary data set, training a shadow model, obtaining a proxy model integration through knowledge distillation, extracting comparison performance characteristics to generate a training sample, training a two-class attack model, and finally evaluating the privacy leakage risk of the target data set by using the training sample. The method is mainly used for detecting the privacy training data leakage risk of the sequence recommendation model, and is suitable for third party model security audit and privacy compliance inspection scenes.

Inventors

SHI YUAN
WU JUNJIE
ZHU TIANYU
HAN XIAO
LI QIUTONG

Assignees

北京航空航天大学

Dates

Publication Date: 20260512
Application Date: 20251224

Claims (10)

1. The privacy training data leakage risk detection method for the sequence recommendation system is characterized by comprising the following steps of: S1, acquiring a target sequence recommendation model to be detected and an auxiliary data set conforming to an input and output format of the model, and dividing the auxiliary data set into a shadow data set and a proxy data set; S2, training a shadow model by utilizing a shadow data set, wherein the architecture of the shadow model is the same as or similar to that of a target sequence recommendation model; s3, training to generate a proxy model integration by using a shadow model as a teacher model and utilizing a knowledge distillation method and adjusting distillation weight alpha based on a proxy data set, wherein the proxy model integration comprises a plurality of proxy models with different prediction capacities; S4, integrating each user interaction sequence in the shadow data set by using a shadow model and an agent model to generate a Top-K recommendation list, and calculating recommendation performance indexes of the models on the user interaction sequences; S5, extracting comparison performance characteristics, wherein the comparison performance characteristics are differences of recommended performance indexes of each agent model in the integration of the shadow model and the agent model; S6, generating a training sample set based on the shadow data set, wherein the training sample set comprises comparison performance characteristics and corresponding member or non-member labels; S7, training a two-class attack model by using the training sample set; And S8, acquiring a target data set, adopting the same method as the S3, taking a target sequence recommendation model as a teacher model, generating a proxy model integration corresponding to the target sequence recommendation model based on proxy data set training, replacing a shadow model with the target sequence recommendation model according to S4 and S5 for each user interaction sequence to be evaluated in the target data set, extracting the contrast performance characteristics between the target sequence recommendation model and the proxy model integration thereof, inputting the contrast performance characteristics into the two-class attack model trained in the S7, and outputting the probability that the user interaction sequence belongs to training data members so as to evaluate the privacy leakage risk.
2. The privacy training data leakage risk detection method for a sequence recommendation system according to claim 1, wherein the distillation weight alpha in S3 is in a value range of 0 to 1, and is used for controlling the balance between the proxy model in simulating the behavior of a target sequence recommendation model and learning a real label, wherein the proxy model only learns from the real label when alpha=0, completely simulates the target sequence recommendation model when alpha=1, and trains by combining two supervision signals when alpha epsilon (0, 1).
3. The privacy training data leakage risk detection method for a sequence recommendation system according to claim 1, wherein the recommendation performance index in S4 is ndcg@k.
4. The privacy training data leakage risk detection method for the sequence recommendation system according to claim 1, wherein in S6, the generation mode of the training sample set comprises the steps of extracting all prefix subsequences from user interaction sequences of the shadow data set, extracting comparison performance characteristics of each prefix subsequence, carrying out average aggregation on the comparison performance characteristics of all prefix subsequences of each user interaction sequence to generate an aggregation comparison performance characteristic of each user interaction sequence, dividing the shadow data set into a training subset and a testing subset according to proportion, marking the aggregation comparison performance characteristics of the user interaction sequences in the training subset as members, marking the aggregation comparison performance characteristics of the user interaction sequences in the testing subset as non-members, and marking the aggregation comparison performance characteristics of the user interaction sequences in the testing subset as 0.
5. The method for detecting risk of leakage of privacy training data for a sequence recommendation system according to claim 1, wherein in S7, the classification attack model is one of logistic regression, XGBoost or random forest.
6. The method for detecting the risk of leakage of the privacy training data for the sequence recommendation system according to claim 1 is characterized in that in S8, a plurality of risk levels are set according to the member probabilities output by the two-class attack model, the higher the probability is, the higher the privacy leakage risk is, wherein the probability E [0,0.25] is a level 1 risk, the probability E (0.25, 0.5) is a level 2 risk, the probability E (0.5, 0.75) is a level 3 risk, the probability E (0.75,0.9) is a level 4 risk, and the probability E (0.9, 1.0) is a level 5 risk.
7. The method for detecting risk of leakage of privacy training data for a sequence recommendation system according to claim 1, further comprising creating a synthetic sequence by means of autoregressive generating capability of the target sequence recommendation model when auxiliary data cannot be acquired, replacing the auxiliary data set with the synthetic sequence for training integration of the shadow model and the proxy model, wherein the creating the synthetic sequence comprises sampling a random article as a starting mark, inputting the random article as a starting mark to the target sequence recommendation model to acquire a recommendation list, randomly sampling an article from the recommendation list and adding the article to the sequence, and iteratively repeating the process to generate the synthetic sequence which is considered reasonable by the target sequence recommendation model.
8. An electronic device comprising a processor, a memory and a program stored on the memory and executable on the processor, the program when executed by the processor implementing the steps of the privacy training data leakage risk detection method for a sequence recommendation system as claimed in any one of claims 1 to 6.
9. A computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, and the computer program when executed by a processor implements the steps of the privacy training data leakage risk detection method for a sequence recommendation system according to any one of claims 1 to 6.
10. Computer program product comprising computer instructions which, when executed by a processor, implement the steps of a privacy training data leakage risk detection method for a sequence oriented recommendation system according to any of claims 1 to 6.

Description

Privacy training data leakage risk detection method for sequence recommendation system Technical Field The invention relates to the technical field of sequence recommendation models and privacy protection. More particularly, the invention relates to a privacy training data leakage risk detection method for a sequence recommendation system. Background With the development of deep learning technology, the sequence recommendation system is widely applied to digital platforms such as electronic commerce, video streaming media, social media and the like. These systems capture the dynamic interest changes of the user by analyzing their historical interaction sequences, thereby predicting the merchandise or content that the user may be interested in the future. Deep sequence models such as architectures based on cyclic neural networks (RNNs), convolutional Neural Networks (CNNs), and convertors have become the dominant paradigm of sequence recommendations, enabling efficient modeling of long-range timing dependencies. However, the sequence recommendation model presents a serious risk of privacy disclosure during the training process. Membership inference attack (Membership INFERENCE ATTACK, MIA) is one of the most typical threats, and an attacker determines whether a specific user interaction sequence is used for model training by analyzing the recommended output of the model, so that sensitive behavior information of the user, such as browsing history, purchase records, demographics and the like, can be revealed. Such attacks not only threaten personal privacy, but may also be used for identity theft or targeted utilization, particularly in the context of increasingly stringent data protection regulations (e.g., GDPR), privacy risk assessment of sequence recommendation models. Member leakage is typically associated with model overfitting-the model of the excessive memory training data can produce differences in behavior that can be exploited across the member and non-member data. Early MIA research mainly aims at traditional machine learning tasks such as image classification and the like, and can be classified into white-box attacks (accessible model parameters and architecture) and black-box attacks (only capable of inquiring model output) according to the access rights of attackers. In black box scenarios, an attacker typically approximates the behavior of the target model with a confidence score, entropy, loss function, etc. statistical signals, or by a shadow model. However, MIA for sequence recommendation systems presents unique challenges. First, the sequence recommendation system typically only exposes the Top-K recommendation list and does not provide a confidence score, forming a "Top-K tag black box" scenario, which defeats the traditional confidence-based attack approach. Second, the inputs to the sequence recommendation system are discrete commodity ID sequences rather than continuous inputs, making it difficult to detect decision boundaries by controllable perturbations. Third, unlike the transduction recommendation system, the sequential recommendation system provides personalized recommendation to all users, so that recommendation mode differences between members and non-members become smaller. The existing MIA method for the recommendation system has the following limitations: 1. Methods based on recommended pattern differences (e.g., biased-MIA, DL-MIA) do not work well in a sequence recommendation system because sequence recommendations personalize both members and non-members, lacking significant pattern differences. 2. Heuristic methods (such as GAP) infer only from the prediction accuracy of the target model, ignoring the prediction difficulty difference of different sequences, i.e. easily predicted sequences can be correctly predicted whether they are members or not, and difficult sequences can be mispredicted, resulting in systematic misclassification. 3. The model extraction method (e.g., ME-MIA) uses a proxy model to approximate a target model, but features extracted by knowledge distillation cannot accurately reserve discrimination signals for distinguishing members from non-members, and limit the effectiveness thereof. Therefore, development of a detection method capable of effectively identifying privacy leakage risk of a sequence recommendation model in a black box scene is needed, and the influence of sequence prediction difficulty is explicitly considered while the information limit that only a Top-K list can be acquired is overcome. Disclosure of Invention An object of the present invention is to provide a method for detecting risk of leakage of privacy training data for a serial recommendation system, so as to at least solve the above-mentioned problems. In order to achieve the objects and other advantages of the present invention, there is provided a privacy training data leakage risk detection method for a sequence recommendation system, including: s1, acquiring a target sequen