EP-4742630-A2 - METHOD FOR PROCESSING DATA BY USING INTERMEDIATE DEVICE, COMPUTER SYSTEM, AND INTERMEDIATE DEVICE

EP4742630A2EP 4742630 A2EP4742630 A2EP 4742630A2EP-4742630-A2

Abstract

Embodiments of this application provide a method for processing data by using an intermediate device, a computer system, and an intermediate device. The computer system includes a computing node, a storage node, and an intermediate device. A service runs in the computing node. The storage node stores data of the service. The service is provided with a storage space, and the service performs an operation on the data of the service in the storage node by accessing the storage space. For example, the storage space is a persistence LOG (PLOG) space. In embodiments of this application, metadata of the storage space is stored in the intermediate device. The intermediate device manages the metadata of the storage space, and implements a data operation between the computing node and the storage node based on the metadata of the storage space. For example, the intermediate device is a switch. According to the solutions provided in this application, a load of the computing node and a data access latency can be reduced.

Inventors

XIONG, Yanhui
YOU, JUN
LIU, YONG
WANG, Qiaoling
LIN, Qinliang

Assignees

Huawei Technologies Co., Ltd.

Dates

Publication Date: 20260513
Application Date: 20210629

Claims (20)

A data processing method, wherein the method is performed by a network connection device, the network connection device is physically connected between a computing node and a storage node, a service runs in the computing node, the storage node stores data of the service, the network connection device manage metadata corresponding to the data of the service, and the method comprises: receiving (S307), from the computing node, a write request for writing to-be-written data to the stoage node to offload processing of the write request from the computing node to the network connection device; determining (S309), based on the metadata, storage address of the to-be-written data in the storage node; and sending (S310) the to-be-written data to the storage node to store the to-be-written to the storage address of the storage node.
The method of claim 1, wherein the network connection device is further configured to manage a logical storage space corresponding to the data of the service.
The method of any one of claim 2, wherein the logical storage space corresponding to a plurality storage nodes, and the plurality storage nodes comprise different types of storage nodes.
The method of any one of claim 1 to 3, wherein the network connection device is further for storing the metadata in the network connection device.
The mothod of any one of claims 1 to 4, wherein the network connection device connets the computing node and the storage node by using a remote direct memory access (RDMA) connection.
The method of any one of claims 1 to 5, wherein before sending the to-be-written data to the storage node, the network connection device is further configured to divide the to-be-written data into a plurality of data fragments according to a preset erasure coding (EC) algorithm or a redundant array of independent disks (RAID) algorithm.
The method of any one of claim 1 to 6, further comprising performing network process to the to-be-written data.
A computer system, comprising a computing node, a storage node, and a network connection device; the computing node is configured to runn a service; the storage node is confitured to store data of the service; and the network connection device is configured to: physically connect the computing node and the storage node; manage metadata corresponding to the data of the service; receive, from the computing node, a write request for writing to-be-written data to the storage node to offload processing of the first write request from the computing node to the network connection device; implement processing of the the first write request to store the to-be-written data to the storage node based on the metadata.
The computer system of claim 8, wherein the network connection device is further configured to manage a first logical storage space corresponding to the data of the service.
The computer system of claim 9, wherein the first logical storage space corresponding to a plurality storage nodes, and the plurality storage nodes comprise different types of storage nodes.
The computer system of any one of claims 8 to 10, wherein the network coonction device is further configured to store the metadata in the network connection device.
The computer system of any one of claims 8 to 11, wherein the network connection between the at least one storage node and the computing node is formed by using a remote direct memory access (RDMA) connection.
The computer system of any one of claims 8 to 12, wherein the network connection device is further configured to divide the to-be-written data into a plurality of data fragments according to a preset erasure coding (EC) algorithm or a redundant array of independent disks (RAID) algorithm.
The computer system of any one of claims 1 to 13, wherein the network connection device is further configured to perform network process to the to-be-written data.
A network connection device, configured to: physically connect a computing node and a storage node; manage metadata corresponding to data of a service running in the computing node; receive, from the computing node, a write request for writing to-be-written data to the store node to offload processing of the write request from the computing node to the network connection device; implement processing of the the write request to store the to-be-written data to the storage node based on the metadata.
The network connection device of claim 15, wherein the network connection device is further configured to manage a first logical storage space corresponding to the data of the service.
The network connection device of claim 15 or 16, wherein the first logical storage space corresponding to a plurality storage nodes, and the plurality storage nodes comprise different types of storage nodes.
The network connection device of any one of claim 15 to 17, wherein the network conection device is further configured to store the metadata in the network connection device.
The network connection device of any one of claims 15 to 18, wherein the network connection between the at least one storage node and the computing node is formed by using a remote direct memory access (RDMA) connection.
The network connection device of any one of claims 15 to 19, wherein the network connection device is further configured to divide the to-be-written data into a plurality of data fragments according to a preset erasure coding (EC) algorithm or a redundant array of independent disks (RAID) algorithm.

Description

TECHNICAL FIELD This application relates to the field of computer technologies, and in particular, to a method for processing data by using an intermediate device, a computer system, and an intermediate device. BACKGROUND Usually, a multi-node computer system includes a plurality of computing nodes and storage nodes. To achieve concurrent access by the plurality of computing nodes to storage spaces for applications, the storage nodes manage the storage spaces for the applications in related technologies. Because the storage node manages the storage space of the application, a load of the computing node and a data processing latency increase when the computing node accesses application data stored in the storage node. For example, when the computing node needs to write data of an application to a storage space corresponding to the application, the computing node first requests the storage node to allocate, for the data, a write address in the storage space corresponding to the application. After obtaining the write address, the computing node sends, to the storage node, a write request for writing the data to the write address. In this way, when writing the data, the computing node first needs to apply to the storage node for allocation of the write address. After the write address is allocated, the computing node sends, to the storage node, the write request for writing the data to the write address. That is, the computing node needs to send at least two requests to the storage node. This leads to increase in the load of the computing node and a data write latency. SUMMARY Embodiments of this application provide a data processing method, a computer system, and an intermediate device. A storage space of an application is managed by an intermediate device connected to a computing node and a storage node, thereby effectively reducing a load of the computing node and reducing a data write latency. To achieve the foregoing objectives, a first aspect of this application provides a computer system. The computer system includes a computing node, a storage node, and an intermediate device. A service runs in the computing node. The storage node stores data of the service. The service is provided with a storage space, and the service performs an operation on the data of the service in the storage node by accessing a storage space. For example, the storage space is a persistence LOG (PLOG) space. In this embodiment of this application, metadata of the storage space is stored in the intermediate device. The intermediate device manages the metadata of the storage space, and implements a data operation between the computing node and the storage node based on the metadata of the storage space. The metadata of the storage space of the service is stored on the intermediate device, so that the intermediate device manages the storage space of the service based on the metadata. In this way, a load of the computing node and a latency in writing data can be reduced. For example, when the operation is a write operation, the computing node only needs to send the write request to the intermediate device. The intermediate device allocates an address in the storage space for the write request based on the metadata, and writes data to the storage device according to the allocated address. Because the computing node only needs to send the request to the intermediate device, a load of the computing node and a latency in writing data are reduced. In an implementation of the first aspect, the metadata of the storage space includes metadata of a logical storage space, the metadata of the logical storage space includes an identifier of the logical storage space, address information of the logical storage space, and address information of a physical storage space corresponding to the logical storage space; and the physical storage space belongs to a space in the storage node. By recording, in the metadata, the address information of the logical storage space of the application and the address information of the physical storage space in the storage node, the intermediate device may map an operation on the logical storage space to an operation on the physical storage space in the storage node. In this way, an operation on the service data stored in the physical storage space that is in the storage node is implemented. In an implementation of the first aspect, there are a plurality of intermediate devices between the computing node and the storage node, the computing node designates a first intermediate device in the plurality of intermediate devices for the logical storage space, and the first intermediate device is configured to store the metadata of the logical storage space. For example, the computing node may select the first intermediate device from the plurality of intermediate devices based on the identifier of the logical storage space. In this way, a plurality of storage spaces of the application are distributed to different intermedi