CN-122027620-A - File transmission method and related device

CN122027620ACN 122027620 ACN122027620 ACN 122027620ACN-122027620-A

Abstract

The application discloses a file transmission method and a related device, which acquire network parameter information of a target transmission link, wherein the target transmission link is a transmission link between an initial node and a target node. And predicting through a bandwidth prediction model according to the network parameter information to obtain a predicted bandwidth value of the target transmission link. And determining a target block value of the target file according to the predicted bandwidth value of the target transmission link, wherein the target block value is used for identifying the size of a target block, and the target block is at least one of a plurality of blocks in the target file. And transmitting the target block from the initial node to the target node according to the target block value of the target file. Therefore, the long tail effect that the overall progress is tired due to too slow transmission of a few blocks in a fixed block dividing mode is relieved, and the transmission efficiency is improved.

Inventors

LIU FENG

Assignees

北京奇艺世纪科技有限公司

Dates

Publication Date: 20260512
Application Date: 20260212

Claims (10)

1. A method of file transfer, the method comprising: acquiring network parameter information of a target transmission link, wherein the target transmission link is a transmission link between an initial node and a target node, the initial node is used for transmitting a target file, the target node is used for receiving the target file, and the network parameter information is used for identifying network conditions of the target transmission link; according to the network parameter information, predicting through a bandwidth prediction model to obtain a predicted bandwidth value of the target transmission link, wherein the bandwidth prediction model is used for predicting predicted bandwidth values respectively corresponding to different network parameter information; Determining a target block value of a target file according to the predicted bandwidth value of the target transmission link, wherein the target block value is used for identifying the size of a target block, and the target block is at least one of a plurality of blocks in the target file; and transmitting the target block from the initial node to a target node according to the target block value of the target file.
2. The method according to claim 1, wherein if the network parameter information is time sequence type data, the bandwidth prediction model includes a first hidden layer and a second hidden layer, the first hidden layer includes a dimension that is a first dimension, the second hidden layer includes a dimension that is a second dimension, the first dimension is greater than the second dimension, the dimension is used to represent a size of the encoded data input to the hidden layer, and the predicting, according to the network parameter information, by the bandwidth prediction model, the predicted bandwidth value of the target transmission link includes: dividing the network parameter information into first network parameter information and second network parameter information, wherein the time step corresponding to the first network parameter information is longer than the time step corresponding to the second network parameter information; According to the first network parameter information, predicting through the first hidden layer to obtain a first predicted bandwidth value of the target transmission link, wherein the first predicted bandwidth value is used for identifying a predicted bandwidth value corresponding to the network parameter information of a long time step; according to the second network parameter information, predicting through the second hidden layer to obtain a second predicted bandwidth value of the target transmission link, wherein the second predicted bandwidth value is used for identifying a predicted bandwidth value corresponding to the network parameter information of a short time step; and fusing the first predicted bandwidth value and the second predicted bandwidth value to determine the predicted bandwidth value of the target transmission link.
3. The method of claim 1, wherein determining the target chunk value of the target file based on the predicted bandwidth value of the target transmission link comprises: Acquiring a safety coefficient, wherein the safety coefficient is used for correcting the size of the target block value, and the safety coefficient is smaller than 1; determining an initial target block value of the target file according to the predicted bandwidth value of the target transmission link; and determining the target block value of the target file according to the product of the safety coefficient and the initial target block value.
4. A method according to claim 3, characterized in that the method further comprises: Acquiring an actual throughput deviation value, wherein the actual throughput deviation value is the difference between actual throughput and predicted throughput, the predicted throughput is determined by inputting the predicted bandwidth value into a throughput prediction model, and the throughput prediction model is used for predicting throughput corresponding to different bandwidth values respectively; And if the actual throughput deviation value is larger than a first preset threshold value, adjusting the safety coefficient, wherein the safety coefficient is increased if the actual throughput is larger than the predicted throughput, and the safety coefficient is decreased if the actual throughput is smaller than the predicted throughput.
5. The method according to claim 4, wherein the method further comprises: If the actual throughput deviation value is larger than the second preset threshold value, the super-parameters of the bandwidth prediction model are adjusted to obtain an updated bandwidth prediction model, wherein the updated bandwidth prediction model is the bandwidth prediction model obtained after the super-parameters are adjusted, and the second preset threshold value is larger than the first preset threshold value; And predicting through an updated bandwidth prediction model according to the network parameter information to obtain an updated predicted bandwidth value of the target transmission link.
6. The method of claim 1, wherein if the target node is configured to verify a target hash value of the target file, the target file generates the target hash value from a hash tree, the method further comprises: generating a hash value of a first node of the target file through a first hash encryption algorithm; Generating a hash value of a second node of the target file through a second hash encryption algorithm, wherein the level of the first node is higher than that of the second node, the security of the first hash encryption algorithm is higher than that of the second hash encryption algorithm, the encryption and decryption speed of the second hash encryption algorithm is higher than that of the first hash encryption algorithm, and the level is used for identifying the position of the node in the hash tree.
7. The method of claim 1, wherein determining the target chunk value of the target file based on the predicted bandwidth value of the target transmission link comprises: and determining the target block value of the target file according to the maximum transmission unit of the target transmission link, the round trip delay of the target transmission link and the predicted bandwidth value of the target transmission link.
8. A document transfer apparatus, the apparatus comprising: an acquisition unit for acquiring network parameter information of a target transmission link, the target transmission link being a transmission link between an initial node and a target node, the initial node is used for transmitting a target file, the target node is used for receiving the target file, and the network parameter information is used for identifying the network condition of the target transmission link; the prediction unit is used for predicting according to the network parameter information through a bandwidth prediction model to obtain a predicted bandwidth value of the target transmission link, wherein the bandwidth prediction model is used for predicting the predicted bandwidth values respectively corresponding to different network parameter information; The determining unit is used for determining a target block value of a target file according to the predicted bandwidth value of the target transmission link, wherein the target block value is used for identifying the size of a target block, and the target block is at least one of a plurality of blocks in the target file; and the transmission unit is used for transmitting the target block from the initial node to a target node according to the target block value of the target file.
9. A computer device, the computer device comprising a processor and a memory: the memory is used for storing a computer program and transmitting the computer program to the processor; the processor is configured to perform the method of any of claims 1-7 according to the computer program.
10. A computer readable storage medium, characterized in that the computer readable storage medium is for storing a computer program for executing the method of any one of claims 1-7.

Description

File transmission method and related device Technical Field The present invention relates to the field of computer technologies, and in particular, to a file transmission method and a related device. Background With the development of the fields of cloud computing, video distribution service and the like, large file transmission demands such as high-definition video, software mirror images, log data, AI training data sets and the like are increasing across regions and operators. Such files are typically bulky, frequently transported, and have high requirements for real-time and integrity of the transport. In the related art, a file is generally divided into a plurality of blocks for transmission. However, the insufficient block size may result in long tail effects for file transfers. In particular, the block size directly affects the transmission delay and retransmission costs of individual blocks. If the block is too large, once the block is lost in a high packet loss or high delay transmission link, a large amount of data needs to be retransmitted, so that the block is not completed for a long time, and the overall progress is tired. If the block is too small, the protocol overhead is too high, the effective throughput is reduced, and part of the blocks are slowed down due to scheduling or buffering limitation. Thus, unreasonable chunk sizes can exacerbate the imbalance in completion time between chunks, resulting in lower file transfer efficiency. Disclosure of Invention In view of the above problems, the present application provides a file transfer method and related device, which are used for improving file transfer efficiency. Based on the above, the application discloses the following technical scheme: In a first aspect, an embodiment of the present application provides a file transmission method, where the method includes: acquiring network parameter information of a target transmission link, wherein the target transmission link is a transmission link between an initial node and a target node, the initial node is used for transmitting a target file, the target node is used for receiving the target file, and the network parameter information is used for identifying network conditions of the target transmission link; according to the network parameter information, predicting through a bandwidth prediction model to obtain a predicted bandwidth value of the target transmission link, wherein the bandwidth prediction model is used for predicting predicted bandwidth values respectively corresponding to different network parameter information; Determining a target block value of a target file according to the predicted bandwidth value of the target transmission link, wherein the target block value is used for identifying the size of a target block, and the target block is at least one of a plurality of blocks in the target file; and transmitting the target block from the initial node to a target node according to the target block value of the target file. In a second aspect, an embodiment of the present application provides a file transfer device, including: an acquisition unit for acquiring network parameter information of a target transmission link, the target transmission link being a transmission link between an initial node and a target node, the initial node is used for transmitting a target file, the target node is used for receiving the target file, and the network parameter information is used for identifying the network condition of the target transmission link; the prediction unit is used for predicting according to the network parameter information through a bandwidth prediction model to obtain a predicted bandwidth value of the target transmission link, wherein the bandwidth prediction model is used for predicting the predicted bandwidth values respectively corresponding to different network parameter information; The determining unit is used for determining a target block value of a target file according to the predicted bandwidth value of the target transmission link, wherein the target block value is used for identifying the size of a target block, and the target block is at least one of a plurality of blocks in the target file; and the transmission unit is used for transmitting the target block from the initial node to a target node according to the target block value of the target file. In a third aspect, embodiments of the present application provide a computer device comprising a processor and a memory: the memory is used for storing a computer program and transmitting the computer program to the processor; The processor is configured to perform the method according to the first aspect above according to the computer program. In a fourth aspect, embodiments of the present application provide a computer readable storage medium for storing a computer program for performing the method of the first aspect above. In a fifth aspect, embodiments of the present application provide a computer pro