KR-102964363-B1 - System for federated learning based on local model optimization, apparatus therefor, and method therefor

Abstract

A method for federated learning for optimizing local models of a federated learning system includes the steps of: a federated server receiving a plurality of local models from a plurality of local devices; the federated server generating a global model based on the plurality of local models; the federated server transmitting the global model to the plurality of local devices; and one of the plurality of local devices generating an ensemble model by combining the local model and the global model.

Inventors

• 박현희
• 김형빈
• 유철우
• 김용호

Assignees

• 명지대학교 산학협력단

Dates

Publication Date

20260512

Application Date

20221230

Claims (20)

1. In a method for federated learning for local model optimization of a federated learning system, A step in which a federated server receives multiple local models from multiple local devices; A step in which the aforementioned federated server generates a global model based on the aforementioned plurality of local models; The step of the federated server transmitting the global model to the plurality of local devices; and A step in which one of the multiple local devices combines a local model and a global model to generate an ensemble model; Includes, The step of generating the above ensemble model is The above local device Characterized by generating ensemble parameters of the ensemble model by combining local parameters of the local model of the previous communication round and global parameters of the global model of the current communication round. Methods for combined learning.
2. In paragraph 1, The step of generating the above ensemble model is The above local device mathematical formula The above ensemble model is generated according to, The above o is a local model, and The above g is a global model, and The above ε is an ensemble model, and The above i is the index of the local device, and The above τ is the index of the communication round, and The above μ is characterized as being an iteration index of a local model. Methods for combined learning.
3. In paragraph 1, The step of the local device updating the local model with the ensemble model; and A step in which the local device learns the updated local model using local data; Characterized by further including Methods for combined learning.
4. delete
5. In paragraph 1, The step of generating the above global model is The aforementioned alliance server mathematical formula The above global model is generated according to, The above o is a local model, and The above g is a global model, and The above i is the index of the local device, and The above is local data of the i-th local device, and The above N is local data of the entire local device, and The above τ is the index of the communication round, and The above μ is characterized as being an iteration index of a local model. Methods for combined learning.
6. In a method for federated learning for optimizing a local model of a local device, A step in which the learning unit generates a local model through learning using local data; A step in which the integration unit provides the above local model to the federated server; The above-mentioned linkage unit receives a global model generated by combining a plurality of local models of a plurality of different local devices from the above-mentioned federation server; and A step in which the learning unit combines the global model and the local model to generate an ensemble model; Includes, The step of generating the above ensemble model is The above learning unit is characterized by generating ensemble parameters of the ensemble model by combining local parameters of the local model of the previous communication round and global parameters of the global model of the current communication round. Methods for combined learning.
7. In paragraph 6, The step of generating the above ensemble model is The above learning unit mathematical formula The above ensemble model is generated according to, The above o is a local model, and The above g is a global model, and The above ε is an ensemble model, and The above i is the index of the local device, and The above τ is the index of the communication round, and The above μ is characterized as being an iteration index of a local model. Methods for combined learning.
8. In paragraph 6, A step in which the learning unit updates a local model with the ensemble model; A step in which the above learning unit learns the updated local model using local data; Characterized by further including Methods for combined learning.
9. delete
10. In paragraph 6, The step of generating the above ensemble model is The above learning unit prepares evaluation data including an evaluation input vector and a label corresponding to the evaluation input vector from local data; A step in which the learning unit calculates a loss representing the difference between the output vector corresponding to the evaluation input vector of each of the local model and the global model and the label using the evaluation data; A step in which the above learning unit derives the inverse of the loss of each of the global model and the local model as the model weight of each of the global model and the local model; A step in which the learning unit applies the model weights to the global parameters and local parameters to derive the weighted average of the global parameters of the global model and the local parameters of the local model as the ensemble parameters of the ensemble model; Characterized by including Methods for combined learning.
11. In a federated learning system for federated learning for local model optimization, When receiving multiple local models from multiple local devices, A global model is created based on the above multiple local models, and A federated server that transmits the global model to the plurality of local devices; and One of the plurality of local devices that generates an ensemble model by combining its own local model and the global model; Includes, The above local device Characterized by generating ensemble parameters of the ensemble model by combining local parameters of the local model of the previous communication round and global parameters of the global model of the current communication round. Federated learning system.
12. In Paragraph 11, The above local device mathematical formula The above ensemble model is generated according to, The above o is a local model, and The above g is a global model, and The above ε is an ensemble model, and The above i is the index of the local device, and The above τ is the index of the communication round, and The above μ is characterized as being an iteration index of a local model. Federated learning system.
13. In Paragraph 11, The above local device Update the local model with the above ensemble model, and Characterized by training the updated local model using local data Federated learning system.
14. delete
15. In Paragraph 11, The above-mentioned alliance server is mathematical formula The above global model is generated according to, The above o is a local model, and The above g is a global model, and The above i is the index of the local device, and The above is local data of the i-th local device, and The above N is local data of the entire local device, and The above τ is the index of the communication round, and The above μ is characterized as being an iteration index of a local model. Federated learning system.
16. In a local device for federated learning for local model optimization, Provide the local model generated through training using local data to the federated server, and An interlocking unit that receives a global model generated by combining multiple local models of multiple different local devices from the aforementioned federated server; and A learning unit that generates an ensemble model by combining the above global model and the above local model; Includes, The above learning unit Characterized by generating ensemble parameters of the ensemble model by combining local parameters of the local model of the previous communication round and global parameters of the global model of the current communication round. Local device.
17. In Paragraph 16, The above learning unit mathematical formula The above ensemble model is generated according to, The above o is a local model, and The above g is a global model, and The above ε is an ensemble model, and The above i is the index of the local device, and The above τ is the index of the communication round, and The above μ is characterized as being an iteration index of a local model. Local device.
18. In Paragraph 16, The above learning unit Update the local model with the above ensemble model, and Characterized by training the updated local model using local data Local device.
19. delete
20. In Paragraph 16, The above learning unit Evaluation data is prepared from local data, including an evaluation input vector and a label corresponding to the evaluation input vector, and Using the above evaluation data, calculate a loss representing the difference between the output vector corresponding to the above evaluation input vector of each of the local model and the global model and the above label, and The inverse of the loss of the global model and the local model, respectively, is derived as the model weight of the global model and the local model, respectively, and Characterized by applying the above model weights to global parameters and local parameters to derive the weighted average of the global parameters of the global model and the local parameters of the local model as the ensemble parameters of the ensemble model. Local device.

Description

System for federated learning based on local model optimization, apparatus therefor, and method therefor The present invention relates to federated learning, and more specifically, to a system for federated learning based on local model optimization, an apparatus for the same, and a method for the same. Until now, distributed data has been collected on a central server and processed all at once using AI algorithms (e.g., Centralized Learning). However, collecting and analyzing scattered data in one place entails many constraints, such as the interests of participants, efficiency, analysis time, and cost. By utilizing federated learning, distributed local data can be processed independently on individual devices holding the data without being collected in one place, while achieving an effect similar to processing the entire data at once. FIG. 1 is a diagram illustrating the configuration of a system for federated learning for local model optimization according to an embodiment of the present invention. FIG. 2 is a diagram illustrating the configuration of a device for federated learning for local model optimization according to an embodiment of the present invention. FIG. 3 is a flowchart illustrating a method for federated learning for local model optimization according to an embodiment of the present invention. FIG. 4 is a flowchart illustrating a method for generating an ensemble model according to another embodiment of the present invention. Figure 5 is a diagram illustrating the effects of a method for federated learning for local model optimization according to an embodiment of the present invention. FIG. 6 is a drawing showing a computing device according to an embodiment of the present invention. Before the detailed description of the present invention, it should be understood that the embodiments described in this specification and the configurations illustrated in the drawings are merely the most preferred embodiments of the present invention and do not represent all of the technical ideas of the present invention, and that various equivalents and modifications that can replace them may exist at the time of filing this application. Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that identical components in the accompanying drawings are indicated by the same reference numerals whenever possible. Furthermore, detailed descriptions of known functions and configurations that could obscure the essence of the invention will be omitted. For the same reason, some components in the accompanying drawings may be exaggerated, omitted, or schematically depicted, and the size of each component does not entirely reflect its actual size. In addition, the terms and words used in the specification and claims described below should not be interpreted as being limited to their ordinary or dictionary meanings, but should be interpreted in a meaning and concept consistent with the technical spirit of the invention, based on the principle that the inventor can appropriately define the concept of the terms to best describe his invention. First, a system for federated learning for local model optimization according to an embodiment of the present invention will be described. FIG. 1 is a diagram illustrating the configuration of a system for federated learning for local model optimization according to an embodiment of the present invention. FIG. 2 is a diagram illustrating the configuration of a device for federated learning for local model optimization according to an embodiment of the present invention. Referring to FIG. 1, a system for federated learning for local model optimization according to an embodiment of the present invention includes a federated server (10) and at least one local device (20). In particular, the local device (20) includes a learning unit (100) and a linkage unit (200). Both the federated server (10) and the local device (20) perform computing operations and are devices capable of wired or wireless communication. The federated server (10) and the local device (20) can communicate with each other through a network. Each learning unit (100) of a plurality of local devices (20) can collect unique data that other local devices (20) cannot collect in the region where it is located, i.e., the local area, and thereby can have unique local data that is distinguished from other local devices (20). This local data can be training data. This training data includes an input vector and a label (Ground-Truth), which is a target value corresponding to the input vector. Each learning unit (100) of a plurality of local devices (20) derives a local model having local parameters through machine learning/deep learning using the local data. Here, the local parameters can be the weights, thresholds, boundaries, gradients, etc. of the local model. Each linkage unit (200) of a plurality of local devices (20) transmits the ge