CN-122018782-A - File distribution method and related device based on file storage cluster

CN122018782ACN 122018782 ACN122018782 ACN 122018782ACN-122018782-A

Abstract

The application discloses a file distribution method based on a file storage cluster and a related device, wherein the file distribution method comprises the steps of responding to a distribution request of a current file to be distributed, dividing each distributed file stored in a storage device into a static file and a dynamic file according to the stability degree of the storage capacity of each distributed file stored in the storage device, counting the current storage capacity of each static file stored in the storage device to obtain the static storage capacity, counting the future storage capacity of each dynamic file stored in the storage device to obtain the dynamic storage capacity, subtracting the static storage capacity and the dynamic storage capacity by the total storage capacity of the storage device to obtain the available storage capacity of the storage device, and distributing the current file to be distributed to the storage device with the available storage capacity meeting the storage capacity condition. By means of the scheme, the storage space utilization rate distribution balancing degree of each storage device can be improved.

Inventors

HE FAN
LI QINGQING
ZHU LINHAO
HUANG LE
WANG RULEI
YE QINGHAN
SHI HUIHUA
ZHOU MINGWEI

Assignees

浙江大华技术股份有限公司

Dates

Publication Date: 20260512
Application Date: 20251216

Claims (10)

1. A method for distributing files based on a file storage cluster, wherein the file storage cluster comprises a plurality of storage devices, comprising: responding to the receiving of an allocation request about a current file to be allocated, dividing each allocated file stored in a storage device into a static file and a dynamic file according to the stability degree of the storage capacity of each allocated file stored in the storage device for each storage device, wherein the stability degree corresponding to the static file is higher than that corresponding to the dynamic file; counting the current storage capacity of each static file stored in the storage device to obtain a static storage capacity, and counting the future storage capacity of each dynamic file stored in the storage device to obtain a dynamic storage capacity; Subtracting the static storage capacity and the dynamic storage capacity from the total storage capacity of the storage device to obtain the available storage capacity of the storage device; and distributing the current file to be distributed to a storage device with the available storage capacity meeting the storage capacity condition for storage.
2. The method of claim 1, wherein the future storage capacity of the allocated file is obtained during allocation of the allocated file to the storage device as a history of files to be allocated, the step of obtaining the future storage capacity of the allocated file comprising: In the process of distributing the distributed files to the corresponding storage devices as historical files to be distributed, dividing each static file stored in each storage device into a plurality of static file sets according to the file attribute of each static file stored in each storage device, wherein one static file set belongs to one candidate static category; Counting the storage capacity of each static file in a static file set belonging to each candidate static category aiming at each candidate static category to obtain the reference storage capacity of the candidate static category; Determining a target static category to which the allocated file belongs from the plurality of candidate static categories according to the file attribute of the allocated file; and taking the reference storage capacity corresponding to the target static category as the future storage capacity of the allocated file.
3. The method according to claim 2, wherein the counting the storage capacity of each static file in the set of static files belonging to the candidate static category to obtain the reference storage capacity of the candidate static category includes: And acquiring a central tendency statistical value of the storage capacity of each static file belonging to the candidate static category as a reference storage capacity of the candidate static category.
4. The method according to claim 2, wherein the dividing each static file stored in each storage device into a plurality of static file sets according to file attributes of each static file stored in each storage device comprises: Fusing the current storage capacity, the file name and the file content field name of the static file aiming at each static file stored in each storage device to obtain a feature vector of the static file; and dividing each static file stored in each storage device into a plurality of static file sets according to the similarity between the feature vectors.
5. The method of claim 1, wherein said counting the current storage capacity of each of said static files stored in said storage device to obtain a static storage capacity, and counting the future storage capacity of each of said dynamic files stored in said storage device to obtain a dynamic storage capacity, comprises: adding the current storage capacities of the static files stored in the storage device to obtain static storage capacities, and adding the future storage capacities of the dynamic files stored in the storage device to obtain dynamic storage capacities; And/or dividing each allocated file stored in the storage device into a static file and a dynamic file according to the stability degree of the storage capacity of each allocated file stored in the storage device, including: Judging whether the time difference between the last writing time and the current time of the distributed files is larger than a preset time difference threshold value or not according to each distributed file stored in the storage device; determining that the allocated file is the static file in response to being greater than the preset time difference threshold; and determining the distributed file as the dynamic file in response to the fact that the distributed file is not larger than the preset time difference threshold.
6. The method of claim 1, wherein the storage capacity condition is that the available storage capacity is maximum, and/or, Before dividing each allocated file stored in the storage device into a static file and a dynamic file according to the stability degree of the storage capacity of each allocated file stored in the storage device, the method further comprises: Determining an allocated file with the current storage capacity larger than a storage capacity threshold value from the allocated files stored in the storage device as a reference file; The dividing each allocated file stored in the storage device into a static file and a dynamic file according to the stability degree of the storage capacity of each allocated file stored in the storage device includes: For each storage device, dividing each reference file in the storage device into the static file and the dynamic file according to the stability degree of the storage capacity of each reference file in the storage device.
7. A file distribution system comprising a file storage cluster and file distribution means for performing the method of any of claims 1-6.
8. An electronic device comprising a memory and a processor for executing program instructions stored in the memory to implement the method of any one of claims 1-6.
9. A computer readable storage medium, characterized in that it has stored thereon program instructions, which when executed by a processor, implement the method of any of claims 1-6.
10. A computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the method of any of claims 1-6.

Description

File distribution method and related device based on file storage cluster Technical Field The present application relates to the field of computer storage technologies, and in particular, to a file allocation method and related devices based on a file storage cluster. Background The file allocation based on the file storage clusters aims at allocating the files to be allocated to one storage device in the file storage clusters for storage. In order to maximize the storage capacity of the file storage clusters, it is necessary to ensure that the storage space usage of each storage device in the file storage clusters is balanced. However, the file allocation method in the related art is adopted to perform file allocation, and the distribution of the storage space usage of each storage device is not sufficiently balanced. Disclosure of Invention The application provides a file distribution method based on a file storage cluster and a related device, which can solve the problem of improving the equilibrium degree of the storage space utilization rate distribution of each storage device. The application provides a file distribution method based on a file storage cluster, which comprises a plurality of storage devices, wherein the file storage cluster comprises the steps of responding to a distribution request of a current file to be distributed, dividing each distributed file stored in the storage device into a static file and a dynamic file according to the stability degree of the storage capacity of each distributed file stored in the storage device, wherein the stability degree corresponding to the static file is higher than that corresponding to the dynamic file, counting the current storage capacity of each static file stored in the storage device to obtain the static storage capacity, counting the future storage capacity of each dynamic file stored in the storage device to obtain the dynamic storage capacity, subtracting the static storage capacity and the dynamic storage capacity by utilizing the total storage capacity of the storage device to obtain the available storage capacity of the storage device, and distributing the current file to the storage device with the available storage capacity meeting the storage capacity condition. The application provides a file distribution device which comprises a division module, a statistics module, a subtraction module and a distribution module. The method comprises the steps of receiving an allocation request about a current file to be allocated, dividing each allocated file stored in a storage device into a static file and a dynamic file according to the stability degree of the storage capacity of each allocated file stored in the storage device for each storage device, wherein the stability degree corresponding to the static file is higher than that corresponding to the dynamic file, counting the current storage capacity of each static file stored in the storage device to obtain the static storage capacity, counting the future storage capacity of each dynamic file stored in the storage device to obtain the dynamic storage capacity, subtracting the static storage capacity and the dynamic storage capacity by using the total storage capacity of the storage device to obtain the available storage capacity of the storage device, and allocating the current file to the storage device with the available storage capacity meeting the storage capacity condition by using the allocation module to store. The application provides a file distribution system which comprises a file storage cluster and a file distribution device, wherein the file distribution device is used for realizing the method. The application provides an electronic device comprising a memory and a processor for executing program instructions stored in the memory to implement the above method. The present application provides a computer readable storage medium having stored thereon program instructions which, when executed by a processor, implement the above-described method. The present application provides a computer program product comprising a computer program which, when executed by a processor, implements the above method. According to the scheme, the distributed files stored in the storage device are divided into the static files and the dynamic files, and then the available storage capacity of the storage device is calculated by utilizing the current storage capacity of the static files and the future storage capacity of the dynamic files. The stability of the storage capacity of the static file is higher than that of the storage capacity of the dynamic file, the future storage capacity is a predicted value, the distortion risk is high, the global performance is strong, the current storage capacity is a determined value, the distortion risk is not high, the global performance is weak, therefore, when the available storage capacity of the storage device is calculated, the current storage capacity of the