KR-20260064325-A - Federated learning device based on flat minima searching and method thereof

Abstract

The present invention relates to a federated learning apparatus and method based on minimum uniformity searching that can converge to the minimum uniformity in the global model by searching for the minimum uniformity between a global model and a local model, and can improve the accuracy of the global model by reflecting the characteristics of multiple local models. The minimum uniform searching-based federated learning device of the present invention comprises a server that learns a global model of federated learning and a client that learns a local model of federated learning.

Inventors

• 윤성환
• 이태환

Assignees

• 울산과학기술원

Dates

Publication Date

20260507

Application Date

20241031

Claims (20)

1. A server that learns a global model of federated learning; and A minimum uniform searching-based federated learning device comprising a client that learns a local model of the federated learning above.
2. In Article 1, A minimum uniform searching-based federated learning device characterized by including a global disturbance model calculation unit that calculates a global disturbance model based on the global model in the above server.
3. In Paragraph 2, A minimum uniform searching-based federated learning device characterized by including: a server that transmits the global model and the global disturbance model to the client to request local model learning; and a local model learning request transmission unit.
4. In Paragraph 3, A minimum uniform searching-based federated learning device characterized by including a local model learning result receiving unit that receives local model learning results from the client, wherein the server above includes:
5. In Paragraph 4, A minimum uniform searching-based federated learning device characterized by including a global parameter update unit that updates global parameters based on the local model training results of the server.
6. In Paragraph 5, A minimum uniform searching-based federated learning device characterized by including: a local model learning request receiving unit in which the client receives the global model and the global disturbance model from the server and receives a request for the learning of the local model.
7. In Paragraph 6, A minimum uniform searching-based federated learning device characterized by including a local model initialization unit that initializes the local model based on the global model in the above client.
8. In Article 7, A minimum uniform searching-based federated learning device characterized by including a local parameter update unit that updates local parameters in the above client.
9. In Paragraph 8, A minimum uniform searching-based federated learning device characterized by including a model difference calculation unit that calculates the difference between the global model and the local model in the above client.
10. In Article 9, A minimum uniform searching-based federated learning device characterized by including a local model learning result transmission unit that transmits the learning result of the local model to the server, wherein the above client is a local model learning result transmission unit.
11. In Paragraph 8, A minimum uniform searching-based federated learning device characterized by including a local parameter update unit that includes a local model gradient calculation module for deriving the gradient of the local model.
12. In Paragraph 11, A minimum uniform searching-based federated learning device characterized by including a local parameter update unit that includes a local disturbance model calculation module that derives a local disturbance model based on the local model.
13. In Paragraph 12, A minimum uniform searching-based federated learning device characterized by including: an interpolation model generation module that generates an interpolated model by interpolating the global disturbance model and the local disturbance model, wherein the local parameter update unit described above comprises the above-mentioned global disturbance model and the local disturbance model.
14. In Paragraph 13, A minimum uniform searching-based federated learning device characterized by including a local parameter update unit that updates the local model based on the local model and the interpolation model.
15. In Article 10, A minimum uniform searching-based federated learning device characterized by including a global parameter update unit that performs an average of the differences between the global model and a plurality of local models.
16. In Paragraph 15, A minimum uniform searching-based federated learning device characterized by including a global parameter update unit that performs an average of the distances between the global model and the local models based on the difference between the global model and a plurality of local models.
17. In Paragraph 16, A minimum uniform searching-based federated learning device characterized by including a global parameter update unit that updates the global model based on the average of the differences between the global model and a plurality of local models.
18. In Paragraph 17, A minimum uniform searching-based federated learning device characterized by including a global parameter update unit comprising a model-to-model distance average binarization module that binarizes the average of the distances between the global model and the local model.
19. In Paragraph 18, A minimum uniform searching-based federated learning device characterized by including: a global parameter update unit comprising a model distance window average module that performs a window average on the result binarized by the model distance average binarization module.
20. A server training phase for training a global model of federated learning on the server; and A minimum uniform searching-based federated learning method comprising: a client learning step for learning a local model of the federated learning at the client.

Description

Federated learning device based on flat minima searching and method thereof The present invention relates to a federated learning apparatus and method based on flat minima searching, and more specifically, to searching for flat minima between a global model and a local model. Furthermore, the present invention relates to a federated learning apparatus and method based on flat minima searching capable of performing flat minima searching on a global model. Federated Learning (FL) is receiving significant attention as a core framework that enables decentralized learning across a vast number of distributed clients while preserving data privacy. The core of Federated Learning (FL) is to store local data on the client side and communicate gradients or model parameters between the server and the client, while direct access to the server's local data is prohibited. Nevertheless, as diversity or heterogeneity in data distribution among clients remains an unresolved challenge that hinders the successful aggregation of global model parameters, leading to performance degradation and inhibiting model convergence, efforts to overcome this have continued. For example, Korean Patent Publication No. 10-2024-0119732 discloses a federated learning and apparatus that receives weights from multiple clients, uses a dequantizer to change the intrinsic precision corresponding to each of the weights to a reference precision, determines masks corresponding to each of the weights based on the intrinsic precision, merges the weights changed to the reference precision based on the masks to determine an integrated weight, quantizes the weight to the intrinsic precision, and transmits it to multiple clients. However, even in this case, there is a disadvantage in that the results of flat minima searching converge heterogeneously between the local and global. FIG. 1 is a schematic diagram showing a federated learning device based on flat minima searching according to one embodiment of the present invention. Figure 2 is a schematic diagram showing the server of Figure 1 in detail. Figure 3 is a schematic diagram showing the client of Figure 1 in detail. Figure 4 is a schematic diagram showing the local parameter update section of Figure 3 in detail. Figure 5 is a schematic diagram showing the global parameter update section of Figure 2 in detail. Figure 6 is a diagram showing the operation of the client of Figure 1. FIG. 7 is a flowchart illustrating a federated learning method based on minimum uniform searching according to an embodiment of the present invention. Figure 8 is a flowchart showing the server learning step of Figure 7 in detail. Figure 9 is a flowchart showing the client learning steps of Figure 7 in detail. Figure 10 is a flowchart showing the local parameter update step of Figure 9 in detail. Figure 11 is a flowchart showing the global parameter update step of Figure 8 in detail. Hereinafter, specific embodiments for carrying out the present invention will be described with reference to the attached drawings. In describing the present invention, terms such as first, second, etc. may be used to describe various components, but the components may not be limited by the terms. The terms are intended solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component. When it is described that a component is connected to or coupled with another component, it may be directly connected to or coupled with that other component, but it can also be understood that there may be other components in between. The terms used in this specification are used merely to describe specific embodiments and are not intended to limit the invention. Singular expressions may include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as "comprising" or "having" are intended to indicate the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not excluding in advance the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof. In addition, the shapes and sizes of elements in the drawings may be exaggerated for clearer explanation. Hereinafter, a minimum uniform searching-based federated learning device and method according to the present invention will be described in detail with reference to the attached drawings. FIG. 1 is a schematic diagram showing a flat minima searching-based federated learning device according to one embodiment of the present invention, and FIGS. 2 to 6 are detailed schematic diagrams and drawings for explaining FIG. 1 in detail. Hereinafter, a federated learning device based on minimum uniform searching according to an embo