Search

CN-122019485-A - File allocation method, apparatus, storage medium, and program product

CN122019485ACN 122019485 ACN122019485 ACN 122019485ACN-122019485-A

Abstract

The embodiment of the application provides a file distribution method, equipment, a storage medium and a program product, and relates to the field of distribution. The method comprises the steps of obtaining a file to be stored, determining an allocation unit of the file to be stored according to the file type of the file to be stored, dividing the file type based on the size of the file to be stored, selecting a block as the allocation unit of the file to be stored when the size of the file to be stored is smaller than a preset threshold value, selecting a section as the allocation unit of the file to be stored when the size of the file to be stored is larger than or equal to the preset threshold value, and allocating a target storage area for the file to be stored in a storage space according to the allocation unit of the file to be stored. The method of the application increases the distribution flexibility of file storage and improves the overall access performance and the system response speed of different types of files.

Inventors

  • SHAO XIN
  • SHAN CHUNXIN

Assignees

  • 中电科金仓(北京)科技股份有限公司

Dates

Publication Date
20260512
Application Date
20251229

Claims (10)

  1. 1. A method for allocating files, applied to a resource allocation system, wherein the resource allocation system includes a resource group, the method comprising: Acquiring a file to be stored; Determining an allocation unit of a file to be stored according to the file type of the file to be stored, wherein the file type is divided based on the size of the file to be stored, and when the size of the file to be stored is smaller than a preset threshold value, the file type is a small file, and a block is selected as the allocation unit of the file to be stored; and distributing a target storage area for the file to be stored in the storage space according to the distribution unit of the file to be stored.
  2. 2. The method of claim 1, wherein in the case that the file to be stored is a large file, the index structure corresponding to the file to be stored is a three-layer index structure, the three-layer index structure comprises a first index node layer, an indirect block layer and a data layer, the first index node layer is used for storing file basic information and an indirect block pointer, the indirect block layer is used for indicating a data layer pointer, and the data layer is used for actually storing the file; And under the condition that the file to be stored is a small file, the index structure corresponding to the file to be stored is a two-layer index structure, wherein the two-layer index structure comprises a second index node layer and the data layer, and the second index node layer is used for storing file basic information and a data layer pointer.
  3. 3. The method of claim 2, wherein the storage space is divided into a plurality of resource groups, each of the resource groups is configured with a block bitmap and an interval bitmap, each bit in the block bitmap corresponds to a block in the resource group and is used for marking a use state of the block, each two consecutive bits in the interval bitmap corresponds to one interval, wherein a first bit indicates whether the corresponding interval is allocated, and a second bit indicates a file type of a file using the corresponding interval.
  4. 4. A method according to claim 3, wherein the storage space is a three-tier storage hierarchy, the first tier is a file system hierarchy for indicating a resource group identifier, the second tier is a resource group hierarchy for recording metadata, a block bitmap, an interval, and inter-cell file source data of a resource group, and the third tier is an interval tier for actually storing data.
  5. 5. The method according to any one of claims 1 to 4, wherein the allocating a target storage area for the file to be stored in a storage space according to an allocation unit of the file to be stored includes: If the allocation unit of the file to be stored is a block, judging whether the residual space of the storage area allocated for the small file currently meets the storage requirement of the file to be stored; under the condition that the residual space of the storage area allocated for the small file currently meets the storage requirement of the file to be stored, a new block is allocated for the file to be stored in the current storage area; And under the condition that the residual space of the storage area currently allocated for the small file does not meet the storage requirement of the file to be stored, allocating a new small file storage area, and allocating a new block for the file to be stored in the new small file storage area.
  6. 6. The method according to any one of claims 1 to 4, wherein the allocating a target storage area for the file to be stored in a storage space according to an allocation unit of the file to be stored includes: If the allocation unit of the file to be stored is an interval, judging whether the residual space of the storage interval currently allocated to the large file meets the storage requirement of the file to be stored; Under the condition that the residual space of the storage interval currently allocated to the large file meets the storage requirement of the file to be stored, allocating space for the file to be stored in the current storage interval; And under the condition that the residual space of the storage interval currently allocated to the large file does not meet the storage requirement of the file to be stored, allocating a new large file storage interval, and under the condition that the new large file storage interval is continuous with the current large file storage interval, merging the new large file storage interval with the current large file storage interval to obtain a merging interval, and allocating space for the file to be stored based on the merging interval.
  7. 7. The method according to any one of claims 1-4, further comprising, after the obtaining the file to be stored: Judging whether the file to be stored is likely to be expanded into a large file or not according to the file extension name, the file creation attribute and the user predefined label of the file to be stored; and if the file to be stored is judged to be possibly expanded into a large file, performing pre-allocation on the file to be stored, wherein the pre-allocation takes an interval as a unit.
  8. 8. An electronic device comprising a processor and a memory communicatively coupled to the processor; The memory stores computer-executable instructions; the processor executes computer-executable instructions stored in the memory to implement the method of any one of claims 1 to 7.
  9. 9. A computer readable storage medium having stored therein computer executable instructions which when executed by a processor are adapted to carry out the method of any one of claims 1 to 7.
  10. 10. A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1 to 7.

Description

File allocation method, apparatus, storage medium, and program product Technical Field The present application relates to the field of distribution, and in particular, to a method, apparatus, storage medium, and program product for file allocation. Background In large-scale distributed storage systems, the file system needs to support efficient storage of both massive small files and oversized files. The former requires high storage density and low fragmentation, and the latter relies on rapid allocation of contiguous space to ensure read-write performance. Therefore, there is a need for a space allocation method that can dynamically adapt to different file types. In the current related implementation, the cluster file system space allocation method mainly adopts allocation units with fixed sizes. For example, a block-based allocation policy divides storage space into blocks of uniform size and files are allocated by the number of blocks. In metadata management, the allocation status of a block is typically recorded using a centralized or distributed index structure (e.g., B-tree, hash table). These methods typically rely on a predefined allocation granularity, track free space and used space through metadata tables or bitmaps, and coordinate allocation operations through a lock mechanism in a multi-node environment. However, the above implementation has difficulty in satisfying the requirements of different file types. Disclosure of Invention The application provides a file allocation method, equipment, a storage medium and a program product, which are used for solving the technical problem of space allocation of different file types. In a first aspect, the present application provides a file allocation method, applied to a resource allocation system, where the resource allocation system includes a resource group, the method includes: Acquiring a file to be stored; determining an allocation unit of a file to be stored according to the file type of the file to be stored, wherein the file type is divided based on the size of the file to be stored, and when the size of the file to be stored is smaller than a preset threshold value, the file type is a small file, and a block is selected as the allocation unit of the file to be stored; and distributing a target storage area for the file to be stored in the storage space according to the distribution unit of the file to be stored. In this embodiment, by dynamically selecting an adapted allocation unit (block or interval) for a small file and a large file according to the comparison between the file size and a preset threshold, the efficiency contradiction generated when the conventional single allocation granularity faces to the mixed file size is fundamentally solved. The method improves the utilization rate of the storage space (aiming at small files) and improves the storage and access performance of large files (aiming at large files), thereby optimizing the resource allocation efficiency and the I/O throughput of the storage system as a whole. In a possible implementation manner, in the case that the file to be stored is a large file, the index structure corresponding to the file to be stored is a three-layer index structure, and the three-layer index structure comprises a first index node layer, an indirect block layer and a data layer; And under the condition that the file to be stored is a small file, the index structure corresponding to the file to be stored is a two-layer index structure, wherein the two-layer index structure comprises a second index node layer and a data layer, and the second index node layer is used for storing file basic information and a data layer pointer. In this implementation, by constructing three layers of indirect indexes for large files and two layers of direct indexes for small files, an optimal balance of metadata management complexity and file access efficiency is achieved. The method reduces the index addressing overhead when the small files are accessed frequently while ensuring the expandability of mass data storage of the large files, thereby improving the response performance of the storage system under the mixed file load from the metadata layer. In one possible implementation, the storage space is divided into a plurality of resource groups, each resource group is configured with a block bitmap and an interval bitmap, each bit in the block bitmap corresponds to a block in the resource group and is used for marking the use state of the block, and each two continuous bits in the interval bitmap corresponds to one interval, wherein the first bit indicates whether the corresponding interval is allocated or not, and the second bit indicates the file type of a file using the corresponding interval. In the implementation mode, the storage space is logically partitioned, and the state tracking is performed by adopting a double-layer bitmap, so that the modularization and the efficient management and control of the storage