CN-121996728-A - Client and sample joint optimization method for federal learning of industrial Internet of things

CN121996728ACN 121996728 ACN121996728 ACN 121996728ACN-121996728-A

Abstract

The invention discloses a client and sample joint optimization method for federal learning of an industrial Internet of things, which comprises the following steps of S1, constructing a cloud-edge-end architecture, S2, initializing a cloud server, S3, executing important sense data shuffling by a terminal device, S4, screening out clients participating in federal learning training by an edge server, S5, executing grouping importance sampling by selected client devices, S6, executing local model training, uploading updated data to the edge server, S7, executing synchronous aggregation by the edge server, uploading the updated data to the cloud server, S8, executing global aggregation by the cloud server, and issuing updating to the edge server for synchronization, S9, repeating the steps S3-S8 until the model converges or reaches a preset training round. The invention can obviously improve training efficiency, convergence speed and prediction precision in a resource constraint scene.

Inventors

CAI HUI
CHEN JIAJIN
Sheng Biyun
ZHOU JIAN
XIAO FU

Assignees

南京邮电大学

Dates

Publication Date: 20260508
Application Date: 20260126

Claims (8)

1. The client and sample joint optimization method for the industrial Internet of things federal learning is characterized by comprising the following steps of: S1, constructing a cloud-edge-end architecture consisting of a cloud server, an edge server and IIoT terminal equipment, S2, initializing global model parameters by a cloud server And calculate the true global data distribution Complete system initialization and distribute global data Global model parameters Down-set to each of the edge servers, S3, the terminal equipment receives dynamic stream data in real time and executes important sense data shuffling, wherein the important sense data shuffling is used for determining whether to replace new data with old data based on an important value of a sample and equipment capacity, S4, screening out the data meeting the constraint condition by the edge server and enabling the aggregate data distribution to be closest to the constraint condition As a client for participating in federal learning training in the current round, S5, the selected client device performs grouping importance sampling on the local data set, generates small batch data required by training, S6, the selected client device performs local model training and uploads the client local model update data to the edge server, S7, the edge server executes synchronous aggregation, uploads the aggregated model update data to the cloud server, S8, the cloud server executes global aggregation, updates the global model, transmits the updated global model parameters to the edge server for synchronization, S9, repeating the steps S3-S8 until the model converges or reaches a preset training round.
2. The client and sample joint optimization method for federal learning of industrial internet of things according to claim 1, wherein step S3 comprises the steps of: S31, quantizing based on the feedforward loss value of the current global model, calculating the importance value of new inflow data, simultaneously calling the importance value of the existing sample locally cached by the equipment, S32, comparing the importance value of the new data with the importance value of the cached samples, and if the importance of the new data is higher than the importance of part of the samples in the cache, not exceeding the storage capacity of the equipment by the total data after replacement Deleting the low importance sample in the buffer memory, storing new data in the buffer memory, updating the local data set of the equipment and the data volume column vector of the corresponding class label of the newly stored data And upload the data volume column vector To an edge server.
3. The client and sample joint optimization method for federal learning of industrial internet of things as set forth in claim 1, wherein step S4 includes the steps of: S41, each edge server receives data volume column vectors transmitted by each device in the group And generating a category data volume matrix of the belonging group, wherein the formula is as follows: s42, the edge server calculates whether each device meets the selection condition based on the 0-1 knapsack optimization model, and the optimization objective function is as follows: Wherein The representative device selects a binary vector while satisfying the constraint that the number of devices selected per group is L, corresponding to the formula: where K is the total number of devices in the group, The element values referring to K rows and 1 column are all transposed versions of the 1 column vector.
4. The client and sample joint optimization method for federal learning of industrial internet of things as set forth in claim 1, wherein step S5 includes the steps of: s51, the selected client device sets the local data Divided into d balanced and mutually exclusive groups of samples, denoted Sg, S52, determining a sample group with importance to be updated in the round based on a polling mechanism, recalculating importance values of the samples of the group, multiplexing historical importance results of the rest groups, S53, calculating the selection probability of each sample group based on an exponential weighting function, and preferentially selecting the sample group with the latest updated importance, wherein the formula is as follows: Wherein the method comprises the steps of For the number of iterations of the sample set i from the last update, beta is a negative scaling factor, S54, sampling according to the importance ratio of the samples in the selected sample group, preferentially selecting the samples with the high importance ratio, and generating the number scale of the samples Is used for completing sample screening.
5. The client and sample joint optimization system for federal learning of the industrial Internet of things is characterized by comprising a cloud-side-end architecture consisting of a cloud server, an edge server and IIoT terminal equipment, wherein a federal learning module is arranged on the basis of the cloud-side-end architecture, and information is transmitted through a communication module; The cloud server of the system is provided with a global model, and the global model is responsible for executing system initialization and global aggregation calculation and transmitting global model parameters and real global data distribution to all edge servers through a communication module; The edge server of the system is provided with a training client selection module and an aggregation module, wherein the training client selection module selects the aggregation data distribution closest to the real global data distribution The L equipment is used as a client side which participates in federal learning training in the round; the aggregation module executes synchronous aggregation and uploads the aggregated model update data to the cloud server; The system comprises a terminal device, a data shuffling module, an importance sampling module and a local model training module, wherein the terminal device is provided with the data shuffling module, the importance sampling module and the local model training module, the data shuffling module determines whether to replace old data with new data based on a sample importance value and device capacity, the importance sampling module performs grouping importance sampling on selected devices to generate small batch data required by training, and the local model training module performs client local model training and uploads client local model updating data to an edge server.
6. The client and sample joint optimization system for federal learning of industrial internet of things according to claim 5, wherein the importance sampling module updates the importance value of the samples based on a polling mechanism and preferentially selects the most recently updated importance sample group when sampling, and then samples the most recently updated importance sample group according to the importance ratio of the samples in the selected sample group to generate a number scale of Is used for completing sample sampling.
7. The industrial Internet of things federal learning client and sample joint optimization device is characterized by comprising a memory, a processor and an industrial Internet of things federal learning client and sample joint optimization program which is stored in the memory and can run on the processor, wherein the industrial Internet of things federal learning client and sample joint optimization program is configured with a client and sample joint optimization method for realizing the industrial Internet of things federal learning as claimed in claim 1.
8. The storage medium is characterized in that the storage medium is stored with an industrial internet of things federal learning client and sample joint optimization program, and the industrial internet of things federal learning client and sample joint optimization program, when executed, realizes the industrial internet of things federal learning client and sample joint optimization method as claimed in claim 1.

Description

Client and sample joint optimization method for federal learning of industrial Internet of things Technical Field The invention relates to the technical field of federation learning and industrial Internet of things intersection, in particular to a federation learning optimization method suitable for resource constraint industrial Internet of things scenes. Background Along with the rapid development of the industrial Internet of things, mass data are generated daily by fixed equipment (such as a monitoring camera, a temperature and humidity sensor and the like) and mobile equipment (such as a patrol unmanned aerial vehicle, a logistics robot and the like) in a factory, and the data contain key information such as equipment running state, production efficiency and the like, so that the mass data are core basis of industrial intelligent decision. However, data is generally stored in a terminal device in a scattered manner, and relates to production privacy and business confidentiality, and the data is directly uploaded to a cloud for centralized machine learning training, so that data leakage risk exists, and federal learning becomes a core paradigm of privacy protection model training in an industrial internet of things scene, wherein the paradigm realizes distributed collaborative training while guaranteeing data privacy through a process of 'local training of the terminal device-updating only model-cloud aggregation optimization'. However, when the existing federal learning technology is applied to an industrial internet of things scene, three key challenges are faced, so that the training efficiency is low and the model performance is poor: a. the problem of data heterogeneity is that the industrial Internet of things equipment has significant heterogeneity in quantity, type and distribution due to deployment scenes and function positioning differences, and the assumption of independent same distribution (i.i.d.) of traditional machine learning is violated. In the prior art, a random client selection strategy is adopted, and equipment with data distribution deviating from global average is selected easily, so that the global model training accuracy is reduced. B. the problem of equipment storage constraint is that most industrial Internet of things terminal equipment is limited in storage capacity and cannot bear a large amount of local data, but the existing federal learning method generally assumes that a client has sufficient storage capacity, and is not considered to limit sample selection by storage constraint, so that low-quality redundant samples are easy to occupy storage space and influence training effectiveness. C. The dynamic stream data adaptation problem is that industrial Internet of things equipment data is dynamically generated in a stream form, data distribution continuously changes along with time, a training sample cannot be updated in real time by a traditional static data processing method, and therefore a model is difficult to adapt to new data characteristics and generalization capability is attenuated. In addition, the existing federal learning sample selection depends on random sampling, and the phenomenon of 'low importance sample oversampling' is easy to occur along with training iteration, and the low importance samples contribute little to model parameter updating, so that redundant calculation overhead is increased, model overfitting can be caused, and training efficiency and final performance are further reduced. In summary, the prior art cannot meet the requirements of federal learning on high efficiency, accuracy and resource adaptation in the industrial internet of things scene, and a targeted optimization scheme is needed. The foregoing is provided merely for the purpose of facilitating understanding of the technical solutions of the present invention and is not intended to represent an admission that the foregoing is prior art. Disclosure of Invention The invention aims to solve the problems and provide the client and sample joint optimization method for the federal learning of the industrial Internet of things, which effectively relieves the model performance attenuation caused by data heterogeneity, reduces the equipment calculation power consumption and improves the training efficiency, the convergence speed and the prediction precision on the basis of guaranteeing the data privacy of equipment. In order to achieve the above object, the technical scheme of the present invention is as follows: the client and sample joint optimization method for the federal learning of the industrial Internet of things comprises the following steps: S1, constructing a cloud-edge-end architecture consisting of a cloud server, an edge server and IIoT terminal equipment, S2, initializing global model parameters by a cloud serverAnd calculate the true global data distributionComplete system initialization and distribute global dataGlobal model parametersDown-set to each of the edge serve