Search

CN-121979443-A - Cold and hot hierarchical storage and SLC bypass management method based on storage-level memory SSD and ZNS SSD

CN121979443ACN 121979443 ACN121979443 ACN 121979443ACN-121979443-A

Abstract

The invention relates to the technical field of data storage, and provides a cold and hot hierarchical storage and SLC bypass management method based on storage-level memory SSD and ZNS SSD, which comprises the following steps of 1, system initialization and heterogeneous hierarchical mapping; step 2, receiving a request and reading and writing shunting, when an application layer initiates a key value operation request, firstly judging the request type by a system, if the request is a writing request, entering step 3, if the request is a reading request, entering step 7, step 3, data writing and solidifying, step 4, temperature judging and SLC bypass judging, step 5, overlapping detection and target level judging, step 6, executing heterogeneous medium writing, step 7, when the read request is processed, searching from high to low according to the medium performance, realizing reading optimization of physical isolation, and step 8, ending, and returning an operation result to the application layer. The invention can better perform cold and hot hierarchical storage and SLC bypass management.

Inventors

  • XIE PING
  • YUE JIE

Assignees

  • 青海师范大学

Dates

Publication Date
20260505
Application Date
20251209

Claims (7)

  1. 1. A cold and hot hierarchical storage and SLC bypass management method based on storage-level memory SSD and ZNS SSD is characterized by comprising the following steps: step 1, system initialization and heterogeneous level mapping; The load store engine RocksDB is configured to initialize ZenFS the file system to identify partition characteristics of the ZNS SSD while establishing the following physically isolated hierarchical mapping rules: 1.1 Extremely hot layer mapping, namely forcedly pointing a Level-0 layer file path and a Level-1 layer file path of a log structure merge Tree LSM-Tree to a mounting point of a storage class memory SSD; 1.2 Mapping a file storage target of a Level-2 layer to an SLC cache region of a zoned naming space solid state disk ZNS SSD; 1.3 Mapping the Level-3 layer and the file storage targets below the Level to the QLC main storage area of the ZNS SSD; step 2, receiving a request and reading and writing shunting; When the application layer initiates a key value operation request, the system firstly judges the request type, if yes, the system enters the step 3, and if yes, the system enters the step 7; step 3, data writing and solidification; step 4, judging the temperature and judging the SLC bypass; when the Level-1 layer data volume reaches a threshold value and needs to sink, executing bypass judgment logic, wherein the bypass judgment logic comprises the following steps: 4.1 Reading metadata of SSTable file to be compressed, obtaining creation time stamp, and calculating data cooling degree ; 4.2 If the cold and hot threshold values are compared If the file is larger than a preset cold data threshold value or the file is marked as coming from the large-batch data import, judging the file as cold data, otherwise, judging the file as hot data; Step 5, overlapping detection and target level judgment; 5.1 If the data to be compressed in the Level-1 layer is judged to be hot data or cold data but files with overlapped key value ranges exist in the Level-2 layer, the mapping rule of the step 1 is followed, and the compression target Level is set to be Level-2; 5.2 If the data to be compressed in the Level-1 layer is judged to be cold data and the key value range of the file is not overlapped in the Level-2 layer, activating an SLC bypass strategy, and redirecting the compression target Level to the Level-3; Step 6, executing heterogeneous medium writing; the compression thread performs writing according to the determined target level, rocksDB sends a writing request with different life cycle prompts to the bottom layer ZenFS; Step 7, when a read request is processed, the read optimization of physical isolation is realized according to cascade searching from high to low of medium performance; And 8, ending, and returning an operation result to the application layer.
  2. 2. The method for cold and hot hierarchical storage and SLC bypass management based on storage class memory SSD and ZNS SSD of claim 1, wherein in step 1, storage class memory SSD is Optane SSD.
  3. 3. The method for cold and hot hierarchical storage and SLC bypass management based on storage class memory SSD and ZNS SSD of claim 2, wherein step 3 comprises the steps of: 3.1 Data is first written into the RocksDB memory table; 3.2 When the memory table is fully written, triggering a disk brushing operation to generate an SSTable file of the ordered character string table, and physically writing Optane SSD; 3.3 When the number of the Level-0 files reaches a threshold, triggering compression, and completing the merging and sorting from the Level-0 layer to the Level-1 layer in Optane SSD.
  4. 4. The method for cold and hot hierarchical storage and SLC bypass management based on storage class memory SSD and ZNS SSD of claim 3, wherein in 4.1), The calculation formula is as follows: 。
  5. 5. The method for cold and hot hierarchical storage and SLC bypass management based on storage class memory SSD and ZNS SSD of claim 4, wherein in step 6, specifically: 6.1 If the target is Level-2 layer, rocksDB sends a short life cycle prompt, zenFS writes it into the SLC buffer area of the ZNS SSD; 6.2 If the target is Level-3 layer, rocksDB sends a very long life cycle hint, zenFS writes it directly into the QLC main memory area of the ZNS SSD.
  6. 6. The method for cold and hot hierarchical storage and SLC bypass management based on storage class memory SSD and ZNS SSD of claim 5, wherein step 7 comprises: First-level inquiry, namely inquiring the memory; second-Level inquiry, namely inquiring a Level-0 layer and a Level-1 layer if the object is missed; three-Level inquiry, namely inquiring a Level-2 layer if a miss occurs; four-Level inquiry, namely inquiring the Level-3 layer and the lower layers if the inquiry is not hit yet.
  7. 7. The method for cold and hot hierarchical storage and SLC bypass management based on storage class memory SSD and ZNS SSD of claim 6, further comprising passive migration of SLC to QLC: For Level-2 data written into the SLC buffer, the data is migrated to the QLC main memory area only when SLC space is insufficient or after the next compression becomes cold.

Description

Cold and hot hierarchical storage and SLC bypass management method based on storage-level memory SSD and ZNS SSD Technical Field The invention relates to the technical field of data storage, in particular to a cold and hot hierarchical storage and SLC bypass management method based on storage-level memory SSD and ZNS SSD. Background With the explosion of data intensive applications, storage engines (e.g., rocksDB) based on LSM-Tree (log structured merge Tree) are widely used. ZNS (Zoned Namespace) SSDs are of interest because they eliminate write amplification caused by Garbage Collection (GC) inside the device. However, the existing "RocksDB +zns" architecture still faces the following challenges: 1. The lack of a read path to take full advantage of the media differences is that the mixed storage of extremely hot data and cold data on a QLC (quad-level cell) can result in significant read latency. 2. SLC cache bottlenecks and wear-and-tear existing layering schemes typically brush data blindly from memory into SLC (Single level cell) areas and then sink to QLCs. When dealing with high frequency compression (Compaction) at the L0 (Level-0)/L1 (Level-1) Level, SLC is subject to tremendous write pressure, is prone to become a performance bottleneck and quickly consumes erase and write life. 3. The read-write interference is serious, namely, although the data volume of SSTable files of L0 and L1 levels is not large, the I/O activity of the SSTable files is extremely severe due to the overlapping of Key ranges (L0) and frequent merging and sequencing, and the read performance of bottom data is seriously interfered, so that long-tail delay is increased. 4. Cold data contaminates SLC-not all sinking data is hot data. Forcing cold data to be written to the SLC not only wastes valuable SLC space, but also adds additional write amplification from SLC migration to QLC. Disclosure of Invention The invention provides a cold and hot hierarchical storage and SLC bypass management method based on storage-level memory SSD and ZNS SSD, which can absorb extremely hot flow by utilizing low-delay characteristics of L0/L1 level solidification of LSM-Tree on storage-level memory SSD media, and simultaneously designs SLC bypass strategy to enable cold data to directly skip SLC to enter QLC in Compaction (compaction/merging and finishing operation) processes, thereby improving system read-write performance, reducing SLC abrasion and prolonging equipment service life. The invention relates to a cold and hot hierarchical storage and SLC bypass management method based on storage-level memory SSD and ZNS SSD, which comprises the following steps: step 1, system initialization and heterogeneous level mapping; The load store engine RocksDB is configured to initialize ZenFS the file system to identify partition characteristics of the ZNS SSD while establishing the following physically isolated hierarchical mapping rules: 1.1 Extremely hot layer mapping, namely forcedly pointing a Level-0 layer file path and a Level-1 layer file path of a log structure merge Tree LSM-Tree to a mounting point of a storage class memory SSD; 1.2 Mapping a file storage target of a Level-2 layer to an SLC cache region of a zoned naming space solid state disk ZNS SSD; 1.3 Mapping the Level-3 layer and the file storage targets below the Level to the QLC main storage area of the ZNS SSD; step 2, receiving a request and reading and writing shunting; When the application layer initiates a key value operation request, the system firstly judges the request type, if yes, the system enters the step 3, and if yes, the system enters the step 7; step 3, data writing and solidification; step 4, judging the temperature and judging the SLC bypass; when the Level-1 layer data volume reaches a threshold value and needs to sink, executing bypass judgment logic, wherein the bypass judgment logic comprises the following steps: 4.1 Reading metadata of SSTable file to be compressed, obtaining creation time stamp, and calculating data cooling degree ; 4.2 If the cold and hot threshold values are comparedIf the file is larger than a preset cold data threshold value or the file is marked as coming from the large-batch data import, judging the file as cold data, otherwise, judging the file as hot data; Step 5, overlapping detection and target level judgment; 5.1 If the data to be compressed in the Level-1 layer is judged to be hot data or cold data but files with overlapped key value ranges exist in the Level-2 layer, the mapping rule of the step 1 is followed, and the compression target Level is set to be Level-2; 5.2 If the data to be compressed in the Level-1 layer is judged to be cold data and the key value range of the file is not overlapped in the Level-2 layer, activating an SLC bypass strategy, and redirecting the compression target Level to the Level-3; Step 6, executing heterogeneous medium writing; the compression thread performs writing according to the determined target lev