Search

CN-122018773-A - Data processing method and system

CN122018773ACN 122018773 ACN122018773 ACN 122018773ACN-122018773-A

Abstract

The application provides a data processing method and a data processing system, which can be applied to the technical field of storage. The method is applied to a data processing system, the system comprises a plurality of stripes, data blocks and check blocks in a first stripe in the plurality of stripes are provided by a plurality of first hard disks, and a first control disk in the plurality of first hard disks is used for controlling a data read-write process in the first stripe. The method comprises the steps that a first control panel receives a first IO instruction from a host and sends a second IO instruction to a first data panel. The first IO instruction is used for indicating to process the first strip and feeding back a processing result to the host, the second IO instruction is generated based on the first IO instruction, the second IO instruction is used for indicating to process the first data block in the first strip and feeding back the first processing result to the first control panel. Therefore, the method can unload the data read-write task from the host to the hard disk, thereby reducing the workload of the host and releasing the resources of the host.

Inventors

  • YU HUANHUAN
  • LI GEZI
  • LI CHENG
  • SHEN HUI
  • ZANG HUI

Assignees

  • 华为技术有限公司

Dates

Publication Date
20260512
Application Date
20241111

Claims (20)

  1. 1. A data processing system, comprising a plurality of hard disks, wherein the plurality of hard disks are used for providing a plurality of stripes, a data block and a check block in each stripe are respectively from different hard disks, the data block is a physical block for storing data, the check block is a physical block for storing check data, and the check data is used for checking the data; the plurality of stripes comprises a first stripe, wherein the data blocks and the check blocks in the first stripe are provided by a plurality of first hard disks, and the plurality of first hard disks belong to the plurality of hard disks; The first control panel is used for controlling the data reading and writing process in the first strip, and the first control panel is any hard disk in the plurality of first hard disks.
  2. 2. The system of claim 1, wherein the data processing system comprises a host, The first control panel is used for receiving a first input/output IO instruction from the host and sending a second IO instruction to the first data panel; The first IO instruction is used for indicating to process the first stripe and feeding back a processing result to the host, the second IO instruction is generated based on the first IO instruction and used for indicating to process a first data block in the first stripe and feeding back a first processing result to the first control panel, the first data panel belongs to the plurality of first hard disks, and the first data panel is different from the first control panel.
  3. 3. The system of claim 2, wherein the first control disk and the first data disk are disposed on a same device, or the first control disk and the first data disk are disposed on different devices, and the first control disk is specifically configured to send the second IO instruction to the first data disk through a high-speed interconnect bus.
  4. 4. A system according to claim 2 or 3, wherein, The first data disk is configured to read first data from a memory of the host after receiving the second IO instruction, write the first data into the first data block on the first data disk, and then send an instruction completion message to the first control disk.
  5. 5. A system according to claim 2 or 3, wherein, The first data disk is used for reading first data from the memory of the host after receiving the second IO instruction, acquiring data of a second data block in the first stripe, and calculating the first data and the data of the second data block based on a disk array RAID algorithm to obtain check data of a first check block in the first stripe; The data block used for storing the first data in the first stripe is the first data block, and the second data block is different from the first data block.
  6. 6. The system of claim 5, wherein the system further comprises a controller configured to control the controller, The first data disk is specifically configured to send a third IO instruction to a second data disk, and receive data of the second data block sent by the second data disk after the third IO instruction is completed; Wherein the second data disk belongs to the plurality of first hard disks, and the second data disk is different from the first data disk.
  7. 7. The system of claim 5 or 6, wherein the system comprises a plurality of sensors, The first data disk is further used for sending verification data of the first verification block to a third data disk; wherein the third data disk is different from the first data disk.
  8. 8. A system according to claim 2 or 3, wherein said first data block is from said first data disc, The first data disk is configured to read second data from the first data block after receiving the second IO instruction, write the second data into the memory of the host, and send an instruction completion message to the first control disk.
  9. 9. A system according to claim 2 or 3, wherein said first data block does not belong to said first data disc, The first data disk is used for acquiring data of a second data block in the first stripe and check data of a first check block in the first stripe after receiving the second IO instruction; calculating the data of the second data block and the check data of the first check block based on a RAID algorithm to obtain second data, writing the second data into a memory of the host, and then sending an instruction completion message to the first control disk; Wherein the second data is stored on the first data block, the second data block being different from the first data block.
  10. 10. The system of claim 2, wherein the system further comprises a controller configured to control the controller, The first control panel is further configured to receive a fourth IO instruction from the host before sending the second IO instruction to the first data panel, and combine the first IO instruction and the fourth IO instruction if the processing manner and the stripe indicated by the fourth IO instruction are the same as those indicated by the first IO instruction, and generate the second IO instruction according to a combination result; The fourth IO instruction is used for indicating to process the second stripe and feeding back a processing result to the host.
  11. 11. The system of claim 10, wherein the first control disk is configured to cache IO instructions from the host through an I/O queue, including the first IO instruction and the fourth IO instruction.
  12. 12. A data processing method as claimed in any one of claims 1 to 11, applied to a data processing system, the method comprising: The first control panel receives a first input/output IO instruction from a host in the data processing system, wherein the first IO instruction is used for indicating to process the first stripe and feeding back a processing result to the host; The first control panel sends a second IO instruction to the first data panel, wherein the second IO instruction is generated based on the first IO instruction, the second IO instruction is used for indicating to process a first data block in the first stripe and feeding back a first processing result to the first control panel, the first data panel belongs to the plurality of first hard disks, and the first data panel is different from the first control panel.
  13. 13. The method of claim 12, wherein the step of determining the position of the probe is performed, The first control panel and the first data panel are disposed on the same device, or the first control panel and the first data panel are disposed on different devices, The first control panel sends a second IO instruction to the first data panel, which comprises the following steps: And the first control panel sends the second IO instruction to the first data panel through a high-speed interconnection bus.
  14. 14. The method according to claim 12 or 13, characterized in that the method further comprises: After receiving the second IO instruction, the first data disk reads first data from the memory of the host, writes the first data into the first data block on the first data disk, and then sends an instruction completion message to the first control disk.
  15. 15. The method according to claim 12 or 13, characterized in that the method further comprises: The first data disk reads first data from the memory of the host after receiving the second IO instruction, acquires data of a second data block in the first stripe, calculates the first data and the data of the second data block based on a disk array RAID algorithm, and acquires check data of a first check block in the first stripe; The data block used for storing the first data in the first stripe is the first data block, and the second data block is different from the first data block.
  16. 16. The method of claim 15, wherein the step of determining the position of the probe is performed, The acquiring the data of the second data block in the first stripe includes: a third IO instruction is sent to a second data disk, and data of the second data block sent by the second data disk after the third IO instruction is completed is received; Wherein the second data disk belongs to the plurality of first hard disks, and the second data disk is different from the first data disk.
  17. 17. The method according to claim 15 or 16, characterized in that the method further comprises: The first data disk sends the check data of the first check block to a third data disk; wherein the third data disk belongs to the plurality of first hard disks, and the third data disk is different from the first data disk.
  18. 18. The method according to claim 12 or 13, wherein the first data block is from the first data disc, The method further comprises the steps of: And after receiving the second IO instruction, the first data disk reads second data from the first data block, writes the second data into the memory of the host, and then sends an instruction completion message to the first control disk.
  19. 19. Method according to claim 12 or 13, characterized in that the first data block does not belong to the first data disc, The method further comprises the steps of: After receiving the second IO instruction, the first data disk acquires data of a second data block in the first stripe and check data of a first check block in the first stripe; calculating the data of the second data block and the check data of the first check block based on a RAID algorithm to obtain second data, writing the second data into a memory of the host, and then sending an instruction completion message to the first control disk; Wherein the second data is stored on the first data block, the second data block being different from the first data block.
  20. 20. The method of claim 12, wherein the step of determining the position of the probe is performed, Before the first control disk sends the second IO instruction to the first data disk, the method further comprises: the first control panel receives a fourth IO instruction from the host, wherein the fourth IO instruction is used for indicating to process the second strip and feeding back a processing result to the host; and under the condition that the processing modes and the strips indicated by the fourth IO instruction and the first IO instruction are the same, the first control panel merges the first IO instruction and the fourth IO instruction, and generates the second IO instruction according to the merging result.

Description

Data processing method and system Technical Field The present application relates to the field of storage technologies, and in particular, to a data processing method and system. Background The server comprises a processor and a Solid state disk (Solid STATE DRIVES, SSD), wherein the SSD is used for storing data, and the processor is responsible for processing the data in the SSD, such as management, reading, writing, calculation and the like. Typically, the number of processors in a server is between 1-8, while the number of SSDs varies from tens to thousands. When the server receives a large number of tasks for reading and writing data from and to the SSD, all processors in the server share the tasks for reading and writing the data. For each processor, the processor can only execute the data read-write task allocated to the processor one by one. Because one data read-write task often involves data in multiple SSDs, a processor needs to control a data read-write process of each SSD involved in the data read-write task, specifically, the processor issues an input-output (inputoutput, IO) instruction to each SSD, and receives data returned by each SSD or a write-completion message. Therefore, a large amount of resources of the processor are occupied in the process of executing the data read-write task by the processor, so that the available resources of the processor are reduced, and the processor cannot be supported to execute more tasks. Disclosure of Invention In order to solve the technical problems, the application provides a data processing method and a system, which can unload the data read-write task from a host to a hard disk, thereby reducing the workload of the host and releasing the resources of the host. In a first aspect, a data processing system is provided that includes a plurality of hard disks. The plurality of hard disks is used to provide a plurality of stripes. The data blocks and the check blocks in each stripe are respectively from different hard disks, wherein the data blocks are physical blocks for storing data, and the check blocks are physical blocks for storing check data. The check data is used for checking the data. The plurality of strips includes a first strip. The data blocks and the check blocks in the first stripe are provided by a plurality of first hard disks. The plurality of first hard disks belong to a plurality of hard disks. The first control disk is used for controlling the data reading and writing process in the first strip, and is any hard disk in a plurality of first hard disks. In the above-mentioned scheme, after the stripe management is performed on the data stored on the hard disk in the data processing system, a control panel (i.e., a first control panel) is configured for the first stripe, and the first control panel controls the data reading and writing process in the first stripe. This may offload all data read-write tasks involving the first stripe that would have been performed by the host to the first control disk, in particular offload control of the data read-write process in the first stripe by the host to the first control disk, thereby reducing the workload of the host and freeing up resources of the host. In some possible implementations, the data processing system includes a host. The first control panel is configured to receive a first input/output IO instruction from the host, and send a second IO instruction to the first data panel. The first IO instruction is used for indicating to process the first stripe and feeding back a processing result to the host. The second IO instruction is generated based on the first IO instruction. And the second IO instruction is used for indicating to process the first data block in the first stripe and feeding back the first processing result to the first control panel. The first data disk belongs to the plurality of first hard disks, and the first data disk is different from the first control disk. The technical scheme provides an implementation mode that the first control panel controls the data reading and writing process in the first strip. Specifically, the first control panel controls the data read-write process in the first stripe by executing the data read-write task on the first stripe by the first control panel instead of the host, and controlling the data read-write operation of other first hard disks (i.e. the first data panel) by the first control panel. The first control panel replaces the host to execute the data read-write task of the first strip, the data read-write task of the first strip can be realized by the host sending the data read-write task of the first strip to the first control panel in the form of an IO instruction (specifically, the first IO instruction), and the mode that the first control panel completes the data read-write task is that the first IO instruction is completed. The first control panel controls the data read-write operation of the first data panel according to the