JP-7857244-B2 - Learning device, data generation device, learning method, and learning program

Inventors

• 披田野 清良
• 福島 和英

Assignees

• ＫＤＤＩ株式会社

Dates

Publication Date

20260512

Application Date

20230315

Claims (6)

1. A generator that outputs synthetic data similar to the training data based on noise, An encoder that obtains feature representations from the data augmentation of the training data or the data augmentation of the synthesized data, A first header that transforms the feature representations corresponding to the aforementioned training data into input to a first loss function for distinguishing between data with sensitive attributes and data with non-sensitive attributes using supervised controlled learning, A second header that converts the feature representations corresponding to the training data and the composite data into inputs to a second loss function for distinguishing between the training data and the composite data using supervised comparative learning, A third header that converts the feature representations corresponding to the training data and the synthesized data into inputs to a third loss function for mutual identification, A generative adversarial network is constructed that includes a fourth header which converts feature representations of data with sensitive attributes and data with non-sensitive attributes from the aforementioned synthesized data into inputs to a fourth loss function for mutual identification. A first update unit that repeatedly updates the first header, the second header, and the encoder based on the gradients of the first and second loss functions, and updates the third header based on the gradient of the third loss function, A second update unit repeats the process of updating the generator based on the update process performed by the first update unit and the third loss function, which is used to make it impossible to distinguish between the training data and the synthesized data. A third update unit repeats the update process performed by the second update unit and the process of updating the fourth header based on the fourth loss function, The system includes an update unit, a fourth update unit that corresponds to the update process performed by the third update unit, and a fourth loss function that repeats the process of updating the generator based on a loss function that makes it impossible to distinguish between data with sensitive attributes and data with non-sensitive attributes, The first loss function is a learning device in which the contribution to the loss is adjusted according to the number of data points for each type of data, including data with sensitive attributes and data with non-sensitive attributes.
2. The learning device according to claim 1, wherein the first and second headers convert the feature representations of the target into vectors for calculating the cosine similarity between data.
3. The learning apparatus according to claim 2, wherein the first loss function is adjusted as the average value of the cosine similarity within the sets of data having sensitive attributes and data having non-sensitive attributes, respectively.
4. The learning device according to claim 1, wherein the third and fourth headers are converted into estimated values for identifying the feature representation of the target.
5. A generator that outputs synthetic data similar to the training data based on noise, An encoder that obtains feature representations from the data augmentation of the training data or the data augmentation of the synthesized data, A first header that transforms the feature representations corresponding to the aforementioned training data into input to a first loss function for distinguishing between data with sensitive attributes and data with non-sensitive attributes using supervised controlled learning, A second header that converts the feature representations corresponding to the training data and the composite data into inputs to a second loss function for distinguishing between the training data and the composite data using supervised comparative learning, A third header that converts the feature representations corresponding to the training data and the synthesized data into inputs to a third loss function for mutual identification, A computer that has constructed a generative adversarial network including a fourth header that converts feature representations corresponding to sensitive and non-sensitive attributes of the synthesized data into inputs to a fourth loss function for mutual identification, A first update step that repeats the process of updating the first header, the second header and the encoder based on the gradients of the first loss function and the second loss function, and the process of updating the third header based on the gradient of the third loss function, A second update step repeats the process of updating the generator based on the update process of the first update step and the third loss function, which is used to make it impossible to distinguish between the training data and the synthesized data. A third update step which repeats the update process performed in the second update step and the process of updating the fourth header based on the fourth loss function, The update process performed in the third update step and the fourth update step, which repeats the process of updating the generator based on a loss function that corresponds to the fourth loss function and makes it impossible to distinguish between data with sensitive attributes and data with non-sensitive attributes, are executed. The first loss function is a learning method in which the contribution to the loss is adjusted according to the number of data points for each type of data, including data with sensitive attributes and data with non-sensitive attributes.
6. A learning program for causing a computer to function as a learning device according to any one of claims 1 to 4.

Description

This invention relates to a fair data generation method using a generative adversarial network. Conventionally, with the aim of achieving fair decision-making by AI (Artificial Intelligence), there have been attempts to train a generator using generative adversarial networks (AGNs) so that the distribution of data with sensitive attributes (e.g., image data of women) and data with non-sensitive attributes (other image data) become identical, i.e., indistinguishable. The goal is to construct a fair AI (e.g., a classifier) using the synthetic data output by the generator (see, for example, Non-Patent Document 1). If a classifier is constructed using fair synthetic data as training data, this classifier can perform inferences regardless of sensitive attributes. However, it is known that when an AI is constructed with fairness in mind, membership estimation attacks that infer training data from its output become easier (see, for example, Non-Patent Document 2). In this case, it has been pointed out that the probability of success of an attack against training data with specific attributes increases significantly. Furthermore, while Non-Patent Literature 2 analyzes the privacy risk of impartial AI targeting classifiers, it is known that membership estimation attacks can also be applied to generative adversarial networks. In a membership estimation attack against a generative adversarial network, the training data used to train the generator is estimated from the similarity between the synthesized data output by the generator trained using the generative adversarial network and the given data. Countermeasures against membership estimation attacks on generative adversarial networks include methods using differential privacy and a method called PrivGAN (see, for example, Non-Patent Document 3), which divides the training data set into multiple datasets and generalizes the generator by imposing constraints so that it does not depend on individual datasets. Xu et al., FairGAN: Fairness-aware Generative Adversarial Networks, IEEE Big Data 2018.Chang et al., On the Privacy Risks of Algorithmic Fairness, EuroSP 2021.Mukherjee et al., PrivGAN: Protecting GANs from Membership Inference Attacks at Low Cost, PETS 2021.Jeong et al., Training GANs with Stronger Augmentations via Contrastive Discriminator, ICLR 2021. This figure shows the functional configuration of the data generation device in the embodiment.This is a first flowchart showing the processing algorithm in the learning phase of the data generation device in the embodiment.This is a second flowchart showing the processing algorithm in the learning phase of the data generation device in the embodiment. An example of an embodiment of the present invention will be described below. The data generation device of this embodiment applies supervised controlled learning instead of the adversarial generative network using unsupervised controlled learning described in Non-Patent Document 4. The data generation device generates fair, privacy-conscious synthetic data that is not similar to the training data alone by diversifying the training data with both sensitive and non-sensitive attributes using data augmentation and supervised controlled learning. Figure 1 is a diagram showing the functional configuration of the data generation device 1 (learning device) in this embodiment. The data generation device 1 is an information processing device (computer) equipped with a control unit 10 and a storage unit 20, as well as various input/output interfaces. The control unit 10 controls the entire data generation device 1, and realizes the various functions in this embodiment by appropriately reading and executing various programs stored in the storage unit 20. The control unit 10 may be a CPU. The storage unit 20 is a storage area for various programs and various data that enable the hardware group to function as a data generation device 1, and may be ROM, RAM, flash memory, or a hard disk drive (HDD). Specifically, the memory unit 20 stores programs (learning program, data generation program) for causing the control unit 10 to execute each of the functions of this embodiment, as well as parameters of the learning model that constitutes the generative adversarial network, training data, and the like. The control unit 10 includes a generator G, an encoder D, a first header h1, a second header h2, a third header h3, and a fourth header h4 as multiple learning models based on various parameters stored in the memory unit 20, a first update unit 11, a second update unit 12, a third update unit 13, and a fourth update unit 14 for training these learning models, and an output unit 15 that outputs synthesized data using the trained models. The data generation device 1 operates these functional units to learn the generator G (learning phase), and then uses the learned generator G to output fair, privacy-conscious synthesized data (generation phase). The control unit 10 constitutes a generat