Search

CN-116226041-B - File read/write method, device and equipment in distributed file system

CN116226041BCN 116226041 BCN116226041 BCN 116226041BCN-116226041-B

Abstract

The invention provides a file reading/writing method, device and equipment in a distributed file system, which are used for solving the technical problem of low file reading/writing performance caused by mutually exclusive competition of snapshot locks in the distributed file system. Under the condition that snapshot processing is not carried out, the common read/write IO only needs to apply for the read/write lock of the upper-level catalogue of the read/write file, so that the mutually exclusive lock competition and IO time delay of file read/write under the catalogue related to the snapshot processing are effectively reduced, meanwhile, in order to ensure the reliability of snapshot service, a pre-check precheck process is added in a snapshot configuration processing flow, and snapshot processing is started under the condition that the read/write request of the snaplock read/write lock without a non-full path is ensured through the pre-check process. The invention can reduce the mutual exclusion competition of file reading and writing to the snapshot lock in the MDS, improve the file reading/writing performance of the distributed file system and ensure the reliability of snapshot configuration.

Inventors

  • YAO QIANQIAN

Assignees

  • 新华三信息技术有限公司

Dates

Publication Date
20260505
Application Date
20221222

Claims (9)

  1. 1. A method for reading/writing files in a distributed file system, the method comprising: during the snapshot processing of the snapshot-free client, only the non-full-path snapshot read/write lock of the upper-level directory of the read/write file is applied to the file read/write IO under the root directory; during the snapshot processing of the snapshot client, the full-path snapshot read/write lock is required to be applied to the file read/write IO under the root directory; before executing snapshot processing, the snapshot client needs to wait for all read/write IO (input/output) processing of acquiring non-full path snapshot read/write locks to be completed, and then starts to execute snapshot processing; the file read/write IO acquires whether snapshot processing is currently performed or not through a snapshot locking mark, the snapshot locking mark is set to be true by the snapshot client before snapshot processing is performed, is set to be false after snapshot processing is performed, and is a global variable.
  2. 2. The method of claim 1, wherein the step of determining the position of the substrate comprises, The file read/write IO informs that the read/write IO applied to the non-full path snapshot read/write lock is executing currently through a non-snapshot request count, wherein the non-snapshot request count is a global variable; The file read/write IO increases the non-snapshot request count by one after applying for a successful non-full path snapshot read/write lock, and decreases the non-snapshot request count by one after releasing the non-full path snapshot read/write lock.
  3. 3. The method of claim 1, wherein the method for restarting the snapshot processing step after all read/write IOs that acquire the non-full path snapshot read/write lock are processed is: Before executing snapshot preparation, the snapshot client sets a snapshot locking mark by initiating a pre-check flow to inform file read/write IO that snapshot processing is currently performed, and waits for the read/write IO currently applied to the non-full path snapshot read/write lock to be executed through non-snapshot request counting.
  4. 4. A method according to claim 3, wherein the pre-inspection procedure comprises: the snapshot client initiates a pre-check flow before executing snapshot preparation, wherein the pre-check flow comprises: the snapshot client informs a snapshot server to initiate a pre-checking step; the snapshot server side informs all snapshot clients in the distributed file system to execute pre-checking; after receiving a notification of executing the pre-check, the snapshot client updates the snapshot locking mark to be true; The snapshot client circularly checks whether the non-snapshot request count is 0, and feeds back a pre-check response to the snapshot server when the non-snapshot request count is 0; and when the snapshot service side receives the pre-checking responses of all the snapshot clients, notifying the snapshot client side initiating the pre-checking step to execute the snapshot preparation step.
  5. 5. A method according to claim 3, characterized in that the method further comprises: and after receiving the notification of resetting the snapshot tag sent by the snapshot server, the snapshot client updates the snapshot locking mark to be false so as to inform the completion of the file read/write IO snapshot processing flow.
  6. 6. A file read/write apparatus in a distributed file system, the apparatus comprising: The first lock request module is used for only applying for the non-full path snapshot read/write lock of the upper-level catalogue of the read/write file to the file read/write IO under the root catalogue during the snapshot processing of the snapshot-free client; The system comprises a snapshot client, a second lock request module, a global variable and a global variable, wherein the snapshot client is used for executing snapshot processing, and the second lock request module is used for applying full-path snapshot read/write lock to file read/write IO under a root directory during the snapshot processing of the snapshot client, the file read/write IO acquires whether snapshot processing is currently performed or not through a snapshot locking mark, the snapshot locking mark is set to be true by the snapshot client before the snapshot processing is executed, and is set to be false after the snapshot processing is executed, and the snapshot locking mark is the global variable.
  7. 7. A snapshot processing device in a distributed file system, the device being applied to a snapshot client, the device comprising: the pre-checking module is used for setting a snapshot locking mark by initiating a pre-checking flow before the snapshot client performs snapshot preparation so as to inform file read/write IO that snapshot processing is currently performed, and waiting for the completion of read/write IO execution of a non-full path snapshot read/write lock currently applied by non-snapshot request counting; And the global mark updating module is used for updating the snapshot locking mark to be false after the snapshot client receives the notification of resetting the snapshot tag sent by the snapshot server so as to inform the completion of the file read/write IO snapshot processing flow.
  8. 8. An electronic device is characterized by comprising a processor, a communication interface, a storage medium and a communication bus, wherein the processor, the communication interface and the storage medium are communicated with each other through the communication bus; a storage medium storing a computer program; a processor for carrying out the method steps of any one of claims 1-5 when executing a computer program stored on a storage medium.
  9. 9. A storage medium having stored thereon a computer program which, when executed by a processor, implements the method of any of claims 1 to 5.

Description

File read/write method, device and equipment in distributed file system Technical Field The present invention relates to the field of communications and cloud computing technologies, and in particular, to a method, an apparatus, and a device for reading/writing files in a distributed file system. Background Currently, many manufacturers are gradually adopting full flash storage, for example, using Solid state disks (Solid STATE DISK or Solid STATE DRIVE, SSD) or other flash media connected by non-volatile memory host controller interface specifications (Non Volatile Memory Express, NVMe) in the storage subsystem instead of conventional hard disks (HARD DISK DRIVE, HDD). Therefore, in the full flash memory version, the underlying disk is no longer a performance bottleneck, and the file system client is changed from kernel mode cephfs to user mode libcephfs, nfsd (basic NFS daemon) is also changed from kernel mode nfsd to user mode ganesha-nfsd, so that the problem that the whole node crashes (crash) due to the software bug (bug) in kernel mode in the past can be reduced, but the performance in user mode is reduced compared with kernel mode. A snapshot may be considered a copy or replica of the data reproduction. For a file system, a file system snapshot is an instant copy of the file system that contains all the information of the file system at the time of snapshot generation, and is itself a complete usable copy. The snapshot configuration can be generally performed through the cluster management page, and parameters such as a snapshot catalogue, a snapshot name, expiration time and the like can be configured. The Ceph file system (CEPH FILE SYSTEM, cephfs) needs to acquire a read lock of snaplock, which is a full-path distributed snapshot lock, in related read/write Input/Output (IO) processes, and needs to acquire snaplock a write lock in related snapshot configuration processes in order to ensure reliability of snapshot characteristics. Under the current metadata service (METADATA SERVICE, MDS) multithreading architecture, flows involving snaplock state machine changes are protected by mutex locks. Under the application scene that the small files are more in covering writing, the competition of related mutual exclusion locks in the MDS is large, so that the time delay of IO flow is large, most threads are in an idle waiting state, and the performance is greatly affected. Disclosure of Invention In view of the above, the present invention provides a method, an apparatus and a device for reading/writing files in a distributed file system, which are used for solving the technical problem of low file reading/writing performance caused by mutually exclusive competition of snapshot locks in the distributed file system. Based on an aspect of the embodiments of the present invention, the present invention provides a file read/write method in a distributed file system, where the method includes: during the snapshot processing of the snapshot-free client, only the non-full-path snapshot read/write lock of the upper-level directory of the read/write file is applied to the file read/write IO under the root directory; during the snapshot processing of the snapshot client, the full-path snapshot read/write lock is required to be applied to the file read/write IO under the root directory; before the snapshot client executes the snapshot processing, the snapshot processing is started again after all read/write IO (input/output) of the read/write lock for which the non-full path snapshot is acquired is waited for to be processed. Further, the file read/write IO obtains whether snapshot processing is currently performed or not through a snapshot locking mark (snaplocking), the snapshot locking mark (snaplocking) is set to true (true) by the snapshot client before snapshot processing is performed, is set to false (false) after snapshot processing is performed, and the snapshot locking mark (snaplocking) is a global variable. Further, the file read/write IO informs that there is currently a read/write IO applying to a non-full path snapshot read/write lock executing through a non-snapshot request count (nosnapreq), the non-snapshot request count (nosnapreq) is a global variable; The file read/write IO increments the non-snapshot request count (nosnapreq) after applying for a successful non-full path snapshot read/write lock, decrements the non-snapshot request count (nosnapreq) after releasing the non-full path snapshot read/write lock. Further, the method for waiting for the completion of all read/write IOs that acquire the non-full path snapshot read/write lock and then starting to execute the snapshot processing step is as follows: Before executing snapshot preparation (do_preparation), the snapshot client sets a snapshot locking mark (snaplocking) by initiating a pre-check (precheck) process to inform file read/write IO that snapshot processing is currently performed, and waits for the read/write IO currently applied t