CN-122029549-A - Method and apparatus for federal learning

CN122029549ACN 122029549 ACN122029549 ACN 122029549ACN-122029549-A

Abstract

Broadly, embodiments of the present disclosure provide a method for performing federal learning on progressively available data that is suitable for implementation by resource-constrained devices such as smartphones and edge devices. A method includes obtaining an instruction to begin a FL run, determining a pattern to be used during the FL run, wherein the pattern is one of a streaming pattern involving sub-period training and a non-streaming pattern involving period training, training a local ML model stored at a client device using the determined pattern, the local ML model corresponding to a global ML model, and transmitting a plurality of updated parameters of the local ML model when training is complete.

Inventors

LI RUI
S. Venice
Y.D.Quan
J. Fernandez Marquez
K. Mishenko
A. Kulis

Assignees

三星电子株式会社

Dates

Publication Date: 20260512
Application Date: 20240412
Priority Date: 20231205

Claims (15)

1. A computer-implemented method for adaptive federal learning FL of a global machine learning ML model at a client device, the method comprising: Obtaining an instruction for starting FL round; Determining a pattern to be used during the FL round, wherein the pattern is one of a streaming pattern involving sub-period training and a non-streaming pattern involving period training; training a local ML model stored at the client device using the determined pattern, the local ML model corresponding to the global ML model, and When training is complete, multiple updated parameters of the local ML model are sent.
2. The method of claim 1, further comprising: obtaining current parameters of the local ML model at the beginning of the FL run, wherein the current parameters are obtained from the global ML model, and The local ML model is modified using the obtained current parameters.
3. The method of claim 1 or 2, wherein determining the pattern to be used during the FL round is based on a current communication network speed of the client device and a size of the local ML model.
4. The method of any of claims 1-3, wherein obtaining the instructions to begin the FL round comprises obtaining a predefined FL round duration and a predefined minimum number of model updates to be performed within the FL round duration.
5. The method of any of claims 1-4, wherein determining a pattern to be used during a FL round comprises: determining a number of updates that the client device is capable of performing during the FL round; comparing the obtained predefined minimum number of model updates to be performed with the determined number of updates, and The streaming mode is determined to be used in response to the obtained predefined minimum number of model updates to be performed being less than the determined number of updates that the client device is capable of performing.
6. The method of any of claims 1-5, wherein determining the number of updates that the client device can perform during the FL round is based on: an arrival rate of a data item at a client device, wherein the data item is used to train a local ML model; predefined FL run duration, and Batch size of a batch of data items used to train a local ML model.
7. The method of any of claims 1 to 6, further comprising determining the batch size based on available memory of the client device for training of the local ML model and the size of the local ML model.
8. The method of any of claims 1-7, wherein determining a pattern to be used during a FL round comprises determining that a streaming pattern is to be used if: The size of the local ML model is equal to the predefined size; the current communication network speed is determined to be equal to the threshold speed, and The predefined minimum number of model updates obtained to be performed is less than the determined number of updates.
9. The method of any of claims 1-8, wherein determining a pattern to be used during the FL round comprises determining that a streaming pattern is to be used if: The size of the local ML model is greater than the predefined size multiplied by the second scaling factor; the current communication network speed is determined to be equal to the threshold speed, and The predefined minimum number of model updates obtained to be performed is less than the determined number of updates divided by the second scaling factor.
10. The method of any of claims 1-9, wherein determining a pattern to be used during a FL round comprises determining that a streaming pattern is to be used if: The size of the local ML model is equal to the predefined size; The current communication network speed is determined to be less than the threshold speed multiplied by the first scaling factor, and The predefined minimum number of model updates obtained to be performed is less than the determined number of updates divided by the first scaling factor.
11. The method of any of claims 1-10, wherein determining a pattern to be used during a FL round comprises determining that a streaming pattern is to be used if: The size of the local ML model is greater than a predefined size m multiplied by a second scaling factor; The current communication network speed is determined to be less than the threshold speed multiplied by the first scaling factor, and The predefined minimum number of model updates obtained to be performed is less than the determined number of updates divided by the second scaling factor divided by the first scaling factor.
12. The method of any of claims 1 to 11, wherein when the non-streaming mode is used, transmitting the plurality of updated parameters of the local ML model includes selecting a subset of the plurality of updated parameters and compressing the selected subset prior to transmitting.
13. The method of any one of claims 1 to 12, further comprising: Data items are obtained that are used in training the local ML model during the FL round.
14. A client device for adaptive federal learning FL for a global machine learning ML model, the client device comprising: At least one processor coupled to the memory, the at least one processor configured to: obtaining an instruction for starting FL round from a server; Determining a pattern to be used during the FL round, wherein the pattern is one of a streaming pattern involving sub-period training and a non-streaming pattern involving period training; training a local ML model stored at the client device using the determined pattern, the local ML model corresponding to the global ML model, and When training is complete, the multiple updated parameters of the local ML model are sent to the server.
15. A computer-readable medium configured to store instructions that, when executed by at least one processor, cause the at least one processor to perform the method of any one of claims 1 to 13.

Description

Method and apparatus for federal learning Technical Field The present disclosure relates generally to a method and apparatus for federal learning. In particular, the present disclosure provides a federal learning process that is suitable for implementation using resource-constrained devices such as smartphones and edge devices. Background In recent years, federal Learning (FL) has become an area of intense research due to its privacy preserving nature and the ability to train ML models with the computing power of millions of user devices. Federal learning is a mature machine learning technique in which a global ML model (typically stored on a server) is trained via multiple independent training sessions performed by local or client devices using their own data sets. Each local/client device is provided with a copy or version of the global ML model for local training. Each client device sends updated parameters of the locally trained model to the server for aggregation. The updated parameters may be weights and/or deviations of the ML model (or its neural network). Federal learning involves multiple rounds or iterations of local training and aggregation. To ensure good performance of the final global ML model, federal learning may rely on an iterative process, where each interaction between a client device and a server is referred to as a federal learning round (round). Each round or iteration involves sending the current global model to the client devices, performing local training of the model on those client devices to generate a set of potential model updates at each client device, and then aggregating and processing those local updates into a single global update and applying it to the global model. The related work of FL is widely focused on the scenario where the client data (i.e., the data items located on the client device) is easily used for training, i.e., the data distribution is known in advance, and the entire data set can be stored in the memory of the client device. However, in real world deployments, data is more likely to be generated instantaneously, the data distribution may change over time, and the client device has limited memory capacity. These aspects have not been partially resolved until recently under an oversimplified setting that does not take into account memory constraints and upstream communication costs of the mobile device. Furthermore, upcoming ML architectures such as transformers have recently paved the way in applications such as computer vision, speech recognition, robotic control, and natural language processing. Nevertheless, such models inherently have significantly greater memory requirements during training. This fact prevents or hinders memory-constrained client devices from performing local training, which in turn prevents deployment of the transformer model in FL settings. The present disclosure has provided a need for an improved federal learning process that can be implemented on resource-constrained devices. Disclosure of Invention [ Solution to the problem ] According to an embodiment of the present disclosure, a computer-implemented method for adaptive federal learning FL of a global machine learning ML model at a client device is provided. The method may include obtaining an instruction to begin a FL round. The method may include determining a pattern to use during the FL round, wherein the pattern is one of a streaming pattern (STREAMING MODE) involving sub-period (epoch) training and a non-streaming pattern involving period training. The method may include training a local ML model stored at the client device using the determined pattern, the local ML model corresponding to the global ML model. The method may include sending a plurality of updated parameters of the local ML model upon completion of the training. According to an embodiment of the present disclosure, a client device for adaptive federal learning FL for a global machine learning ML model is provided. The client device may include at least one processor coupled to the memory for obtaining instructions from the server to begin a FL round. The client device may include at least one processor coupled to the memory for determining a pattern to use during the FL round, wherein the pattern is one of a streaming pattern involving sub-period training and a non-streaming pattern involving period training. The client device may include at least one processor coupled to the memory for training a local ML model stored at the client device using the determined pattern, the local ML model corresponding to the global ML model. The client device may include at least one processor coupled to the memory for transmitting the plurality of updated parameters of the local ML model to the server upon completion of the training. According to an embodiment of the present disclosure, a computer-readable storage medium configured to store instructions is provided. The instructions, when executed by the at least one p