CN-122021810-A - Training method for distributed deployment of neural network

CN122021810ACN 122021810 ACN122021810 ACN 122021810ACN-122021810-A

Abstract

The invention discloses a training method of a distributed deployment neural network, which comprises the steps of adding a communication mathematical model between every two computing nodes in a neural network architecture corresponding to the distributed deployment neural network, wherein the input of the communication mathematical model is middle data output by the previous computing node, the output of the communication mathematical model is lossy middle data, executing a forward propagation computing process, computing a loss function, executing a reverse computing process, computing the gradient of a neural network parameter and a communication system parameter, and updating the neural network parameter and the communication system parameter according to the gradient. The loss function includes an error function reflecting the accuracy of the final calculation result and a communication cost function reflecting the communication cost. The invention can reduce the communication cost to the maximum extent, and the distributed deployment neural network with single training can be widely applied to the real scenes with various complex channel environments.

Inventors

MIAO FENG
WANG CONG
LIANG SHIJUN
Yang Zaizheng

Assignees

南京大学

Dates

Publication Date: 20260512
Application Date: 20260210

Claims (10)

1. A training method for distributed deployment of a neural network, comprising: According to the wireless transmission data and the communication system parameters, establishing a relation between the bit error rate and the communication system parameters to obtain a communication mathematical model; In a neural network architecture corresponding to the distributed deployment neural network, adding the communication mathematical model between every two computing nodes; Performing a forward propagation calculation process on the neural network architecture to calculate a loss function, wherein the loss function comprises an error function reflecting the accuracy of a final calculation result and a communication cost function reflecting the communication cost; according to the loss function, obtaining gradients of the loss function on the neural network parameters and the communication system parameters respectively through back propagation calculation; Updating the neural network parameters according to the gradient of the loss function to the neural network parameters, and updating the communication system parameters according to the gradient of the loss function to the communication system parameters; and repeating the training process after updating the neural network parameters and the communication system parameters.
2. The method for training a distributed deployment neural network of claim 1, wherein the forward propagation calculation process comprises: performing forward propagation computation of the neural network prior to the communication mathematical model; Obtaining the bit error rate of a channel through the communication mathematical model according to the target channel condition and the communication system parameter, and taking the bit error rate as the turnover probability of each bit in the communication code element; The bit data is randomly turned over by taking the communication code element as a basic unit according to the turning probability of each bit in the communication code element to obtain lossy bit data, wherein the bit data is obtained by converting intermediate data output by a previous computing node; and converting the lossy bit data into lossy intermediate data, and completing the forward propagation calculation of the neural network after the communication mathematical model according to the lossy intermediate data.
3. The method for training a distributed deployment neural network according to claim 1, wherein the communication mathematical model uses a neural network to establish a relationship between the bit error rate and the communication system parameters according to the wireless transmission data and the communication system parameters.
4. The training method of the distributed deployment neural network according to claim 1, wherein the obtaining the bit error rate of the channel through the communication mathematical model according to the target channel condition and the communication system parameter comprises the steps of equalizing the bit error rates of all bits in a single code element under the fixed channel condition of flat fading, adjusting the bit error rate of all bits in the single code element according to the frequency component under the fixed channel condition of frequency selective fading, and adjusting the bit error rate of lower bits to be larger than the bit error rate of higher bits when a high-order quadrature amplitude modulation scheme is used.
5. The method for training a distributed deployment neural network according to claim 1, wherein the bit error rate corresponding to the basic channel condition is calculated under the mobile communication condition, and the bit error rate is obtained by randomly selecting and combining the basic channel conditions in the training iteration process.
6. The method for training a distributed deployment neural network of claim 1, wherein the error function is a multi-class cross entropy function.
7. The method for training a distributed deployment neural network according to claim 1, wherein the communication cost is determined according to at least one of communication system hardware accuracy, complexity of a channel coding and decoding algorithm, and cyclic prefix length against intersymbol interference.
8. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the training method of the distributed deployment neural network of any one of claims 1 to 7 when the computer program is executed.
9. A computer readable storage medium having stored thereon a computer program, which when executed by a processor implements the training method of distributed deployment of a neural network according to any of claims 1 to 7.
10. A computer program product comprising a computer program and/or instructions which, when executed by a processor, implements the training method of distributed deployment of a neural network according to any one of claims 1 to 7.

Description

Training method for distributed deployment of neural network Technical Field The invention relates to a neural network training method, in particular to a training method for distributed deployment of a neural network. Background Distributed deployment of a neural network refers to assigning computing tasks of different parts in the neural network model to different computing devices, and the devices communicate and cooperate to complete the reasoning computation process of the whole neural network model. In the context of terminal device reasoning, the communication process is often referred to as wireless communication. The neural network reasoning process distributed on a plurality of devices is divided into the following steps of 1. The calculation process of each device is that each device inputs the acquired input data into a part of the neural network model responsible for the device, completes the calculation process and generates output data. The input data here may be raw input data local to the device or intermediate data from other devices, each device calculating completed output data to be sent by wireless communication to the other devices in need thereof. 2. And the wireless communication process among the devices is that the intermediate data of the neural network is transmitted among the devices through the wireless communication process. The wireless communication process is affected by a number of factors including wireless channel characteristics, wireless modulation-demodulation scheme, and the like. In current distributed deployment neural network research and application, it is often required that the wireless communication process is lossless, that is, the received data of the receiving end is completely consistent with the data of the transmitting end. But real wireless communication is not an ideal data transmission procedure. Because electromagnetic waves carrying information are interfered and attenuated in the transmission process, errors can occur in data restored according to the received electromagnetic waves, and the traditional distributed deployment neural network reasoning system needs to pay high communication cost to ensure the nondestructive performance of data transmission, so that the cost of the distributed deployment neural network in a communication part is difficult to reduce. How to train the distributed deployment neural network, so that the whole distributed deployment neural network can accept a certain degree of lossy communication, thereby reducing the cost is a technical problem which needs to be solved. In the prior art, a neural network model is trained by adopting a back propagation algorithm, but the accuracy of the reasoning result of the neural network model trained by the technical scheme is easy to be interfered by transmission errors, and the error rate of data transmission is still required to be reduced by higher communication cost. Secondly, the wireless channel characteristics are affected by factors such as channel environment, relative motion between the transmitting end and the receiving end, and the like, and are time-varying and not fixed hardware. When channel characteristics change, a distributed deployment neural network with good inference accuracy under the prior fixed channel conditions may perform poorly. In addition, the existing method for reducing the communication cost often depends on debugging experience, and the corresponding lowest communication cost under the error rate which can be tolerated by the distributed deployment neural network cannot be found. Disclosure of Invention Aiming at the problems, the invention provides the training method of the distributed deployment neural network, which can reduce the communication cost to the maximum extent, and the distributed deployment neural network after single training can be widely applied to real scenes with various complex channel environments. The technical scheme adopted by the invention is a training method for distributed deployment of a neural network, which comprises the following steps: According to the wireless transmission data and the communication system parameters, establishing a relation between the bit error rate and the communication system parameters to obtain a communication mathematical model; In a neural network architecture corresponding to the distributed deployment neural network, adding the communication mathematical model between every two computing nodes; Performing a forward propagation calculation process on the neural network architecture to calculate a loss function, wherein the loss function comprises an error function reflecting the accuracy of a final calculation result and a communication cost function reflecting the communication cost; according to the loss function, obtaining gradients of the loss function on the neural network parameters and the communication system parameters respectively through back propagation calculation; Updating