CN-121979898-A - Storage method and database system

CN121979898ACN 121979898 ACN121979898 ACN 121979898ACN-121979898-A

Abstract

The application provides a storage method and a database system, wherein the method comprises the steps of responding to modification operation of a plurality of data pages, generating a plurality of log records, writing the log records into a log storage node, updating the data pages based on the log records in the log storage node to obtain updated data pages, and writing the updated data pages into the data storage node. By the scheme of the application, the performance of the database system can be improved.

Inventors

WANG BAOHAN
PEI TINGWEI
XUE GANG
SHEN ZHENG
SHI PIXIAN
WANG JIANFEI

Assignees

中移(苏州)软件技术有限公司
中国移动通信集团有限公司

Dates

Publication Date: 20260505
Application Date: 20260123

Claims (10)

1. A storage method, characterized in that it is applied to a database system, the database system including a log storage node for storing log records and a data storage node for storing data pages, the method comprising: generating a plurality of log records in response to a modification operation of a plurality of data pages, wherein each log record corresponds to a modification operation of one data page; Writing the plurality of log records to the log storage node; Updating the plurality of data pages based on the plurality of log records in the log storage node to obtain updated plurality of data pages; and writing the updated plurality of data pages into the data storage node.
2. The method of claim 1, wherein the generating a plurality of log records comprises: A corresponding log record is generated for the modification operation for each of the plurality of data pages.
3. The method of claim 1, wherein the generating a plurality of log records comprises: The plurality of log records is generated in parallel.
4. A method according to any one of claims 1 to 3, wherein said writing said plurality of log records to said log storage node comprises: Writing the plurality of log records to the log storage node in a key-value format, wherein a key of each log record includes page information of a corresponding data page, and a value of the log record includes modification information of the data page.
5. A method according to any one of claims 1 to 3, wherein said writing said plurality of log records to said log storage node comprises: And writing the plurality of log records into the log storage node out of order, wherein the sequence of writing the plurality of log records into the log node is irrelevant to the sequence corresponding to the log serial numbers of the plurality of log records.
6. A method according to any one of claims 1 to 3, further comprising: receiving a data reading request sent by a client, wherein the data reading request is used for requesting to read a data page of a target version; reading the data page of the target version in the data storage node in the case that the data page of the target version is stored in the data storage node in response to the data read request; And returning the data page of the target version to the client.
7. The method of claim 6, wherein the method further comprises: Reading one or more log records of the data page from the log storage node, in the event that the target version of the data page is not stored in the data storage node, reading a base version of the data page in the data storage node; Updating the data page of the basic version based on one or more log records of the data page to obtain the data page of the target version; And returning the data page of the target version to the client.
8. The method of claim 7, wherein the updating the base version of the data page to obtain the target version of the data page comprises: and updating the data pages of the basic version according to the sequence of the log sequence numbers of the one or more log records to obtain the data pages of the target version.
9. A method according to any one of claims 1 to 3, further comprising: In the process of crash recovery, a plurality of log records after a preset consistency point are read from the log storage node, wherein the consistency point is used for indicating the starting position of log record reading in the process of updating a data page; recovering the data pages respectively corresponding to the plurality of log records according to the plurality of log records after the consistency point to obtain recovered data pages; And writing the recovered data page into the data storage node.
10. A database system, characterized in that it comprises a log storage node for storing log records and a data storage node for storing data pages, the database system being adapted to perform the method of any of claims 1-9.

Description

Storage method and database system Technical Field The present application relates to the field of database technologies, and in particular, to a storage method and a database system. Background With the rapid development of the information technology industry, the data volume generated in each field is increased in a mass manner, and the consistency of data storage and the high performance of data access have become core problems of a database system. In a database system, data storage consistency, i.e., data consistency (Data Consistency in english) is the requirement that data remain accurate, efficient, reliable at any time and meet data integrity constraints. Data consistency is a core property of the Database core engine, i.e., database management system (Database MANAGEMENT SYSTEM, DBMS). The data consistency ensures that the database system can still maintain the data accurate and meet the expectations under various scenes such as transaction processing, concurrent operation, system faults and the like. In order to achieve data consistency, the database system periodically writes modified data in the database system into a storage device (such as a disk) by introducing a Checkpoint (English is Checkpoint) design, so as to achieve data persistence. Checkpointing is an internal operation of a database system. Through the detection point operation, the database system refreshes dirty pages (modified data pages which are not written into the disk) in the memory to external storage (namely storage equipment), so that the integrity and consistency of data are realized. The data manipulation language (Data Manipulation Language, DML) is used as an important component of the database system language to implement operations such as insertion, modification, and deletion of data in the database system. A large number of DML instructions may generate a large number of dirty pages, and thus, during checkpointing operations performed by the database system, may trigger operations of writing a large number of dirty pages in the memory into the external storage, generating a large number of instantaneous Input/Output (I/O) loads, which affect the performance of the database. Disclosure of Invention In order to solve the related technical problems, the embodiment of the application provides a storage method and a database system. The technical scheme of the embodiment of the application is realized as follows: The embodiment of the application provides a storage method, which is applied to a database system, wherein the database system comprises a log storage node and a data storage node, the log storage node is used for storing log records, the data storage node is used for storing data pages, and the method comprises the following steps: generating a plurality of log records in response to a modification operation of a plurality of data pages, wherein each log record corresponds to a modification operation of one data page; Writing the plurality of log records to the log storage node; Updating the plurality of data pages based on the plurality of log records in the log storage node to obtain updated plurality of data pages; and writing the updated plurality of data pages into the data storage node. In the above solution, the generating a plurality of log records includes: A corresponding log record is generated for the modification operation for each of the plurality of data pages. In the above solution, the generating a plurality of log records includes: The plurality of log records is generated in parallel. In the above solution, the writing the plurality of log records into the log storage node includes: Writing the plurality of log records to the log storage node in a key-value format, wherein a key of each log record includes page information of a corresponding data page, and a value of the log record includes modification information of the data page. In the above solution, the writing the plurality of log records into the log storage node includes: And writing the plurality of log records into the log storage node out of order, wherein the sequence of writing the plurality of log records into the log node is irrelevant to the sequence corresponding to the log serial numbers of the plurality of log records. In the above scheme, the method further comprises: receiving a data reading request sent by a client, wherein the data reading request is used for requesting to read a data page of a target version; reading the data page of the target version in the data storage node in the case that the data page of the target version is stored in the data storage node in response to the data read request; And returning the data page of the target version to the client. In the above scheme, the method further comprises: Reading one or more log records of the data page from the log storage node, in the event that the target version of the data page is not stored in the data storage node, reading a base version of the data