CN-121636586-B - Instantaneous snapshot creation and reading method, device and equipment

CN121636586BCN 121636586 BCN121636586 BCN 121636586BCN-121636586-B

Abstract

The application discloses a method, a device and equipment for creating and reading an instantaneous snapshot, and relates to the technical field of data processing, wherein when a snapshot creation request is received, a current snapshot is created based on the snapshot creation request, and the current snapshot is inserted into a snapshot table; when a history snapshot reading request is received, a corresponding history snapshot time stamp is obtained from a snapshot table based on the history snapshot reading request, a target check point list is obtained by inquiring based on the history snapshot time stamp, and a data object existing in the history snapshot time stamp is loaded based on the target check point list to construct a target snapshot partition. The method solves the problems of long creation time caused by full data replication and low reading performance caused by full historical data scanning in the traditional snapshot technology.

Inventors

XU PENG
WANG ZHEN

Assignees

矩阵起源(深圳)信息科技有限公司

Dates

Publication Date: 20260508
Application Date: 20260204

Claims (8)

1. A method for instantaneous snapshot creation and reading, the method comprising: When a snapshot creation request is received, creating a current snapshot based on the snapshot creation request, and inserting the current snapshot into a snapshot table; When a history snapshot reading request is received, acquiring a corresponding history snapshot time stamp from the snapshot table based on the history snapshot reading request; querying based on the historical snapshot time stamp to obtain a target check point list, wherein the target check point list is an ordered set containing one or more check points, and the check points are persistent records of the database on all data object states in a certain time period; Loading data objects existing at the historical snapshot time stamp based on the target checkpoint list to construct a target snapshot partition; the loading the data object with the historical snapshot time stamp based on the target checkpoint list to construct a target snapshot partition further comprises: acquiring the total number of data objects, the data size of the data objects and the deleting line proportion of the data objects; Obtaining a vacuum degree score based on the data size and the deleted row proportion; when the vacuum degree score is larger than a preset score threshold, adding the data object into a queue to be merged; determining whether a data storage state meets a merging triggering condition according to the total number of the data objects, the data size of the data objects and the vacuum degree score; When the data storage state meets a merging triggering condition, carrying out data merging based on the queues to be merged; the loading the data object with the historical snapshot time stamp based on the target checkpoint list to construct a target snapshot partition further comprises: Obtaining a plurality of active snapshot time stamps based on the snapshot table; Obtaining a plurality of data object information based on the target checkpoints; Comparing the creation time stamp and the deletion time stamp of each data object information with each active snapshot time stamp to obtain a comparison result; determining a deletable object according to the comparison result; Adding the deletable object to a tombstone record.
2. The method of claim 1, wherein upon receiving a snapshot creation request, creating a current snapshot based on the snapshot creation request and inserting the current snapshot into a snapshot table comprises: generating a snapshot identifier and a snapshot time stamp when a snapshot creation request is received; determining a snapshot level and a snapshot name according to the snapshot creation request, wherein the snapshot level comprises a cluster level, an account level, a database level and a table level; Extracting corresponding object identification information based on the snapshot level; Determining a current snapshot according to the snapshot identification, the snapshot timestamp, the snapshot name, the snapshot level and the object identification information; And inserting the current snapshot into a snapshot table.
3. The method of claim 1, wherein upon receiving a historical snapshot read request, the obtaining a corresponding historical snapshot timestamp from the snapshot table based on the historical snapshot read request comprises: When a historical snapshot reading request is received, determining a historical snapshot name or a query timestamp according to the historical snapshot reading request; searching a historical snapshot from the snapshot table according to the historical snapshot name or the query timestamp; and extracting a historical snapshot time stamp from the historical snapshot.
4. The method of claim 1, wherein the querying based on the historical snapshot time stamp to obtain a target checkpoint list comprises: Sending the historical snapshot time stamp to a thing node, and acquiring a checkpoint list returned by the thing node based on the historical snapshot time stamp, wherein the checkpoint list comprises a plurality of checkpoints; And when the ending time of the last checkpoint of the checkpoint list is greater than or equal to the historical snapshot time stamp, taking the checkpoint list as a target checkpoint list.
5. The method of claim 1, wherein loading the data object existing at the historical snapshot time stamp to construct a target snapshot partition based on the target checkpoint list comprises: obtaining target checkpoints contained in each target checkpoint list, wherein the target checkpoints comprise a plurality of data object information, and the data object information comprises a data object ID, a creation time stamp and a deletion time stamp; when the creation time stamp of the data object information is smaller than or equal to the historical snapshot time stamp and the deletion time stamp of the data object information is larger than the historical snapshot time stamp, acquiring a data object corresponding to the data object ID of the data object information; and constructing a target snapshot partition according to the data object.
6. The method of claim 5, wherein prior to obtaining the target checkpoints contained in each of the target checkpoints lists, further comprising: Taking a data object ID corresponding to the target check point list and the historical snapshot time stamp as query conditions; Inquiring in a snapshot use cache based on the inquiring condition; when a corresponding snapshot partition is queried in the snapshot use cache, taking the snapshot partition as a target snapshot partition; And executing the step of acquiring the target checkpoints contained in each target checkpoint list when no corresponding snapshot partition is queried in the snapshot use cache.
7. An instantaneous snapshot creation and reading device, said device comprising: The creation module is used for creating a current snapshot based on the snapshot creation request and inserting the current snapshot into a snapshot table when the snapshot creation request is received; the reading module is used for acquiring a corresponding historical snapshot time stamp from the snapshot table based on the historical snapshot reading request when the historical snapshot reading request is received; The query module is used for querying based on the historical snapshot time stamp to obtain a target check point list, wherein the target check point list is an ordered set containing one or more check points, and the check points are persistent records of the database on all data object states in a certain time period; the loading module is used for loading the data objects existing in the historical snapshot time stamp based on the target check point list to construct a target snapshot partition; the loading module is also used for acquiring the total number of data objects, the data size of the data objects and the deleting line proportion of the data objects, obtaining vacuum degree scores based on the data size and the deleting line proportion, adding the data objects into a queue to be merged when the vacuum degree scores are larger than a preset scoring threshold value, determining whether a data storage state meets a merging triggering condition according to the total number of the data objects, the data size of the data objects and the vacuum degree scores, and merging data based on the queue to be merged when the data storage state meets the merging triggering condition; The loading module is further configured to obtain a plurality of active snapshot time stamps based on the snapshot table, obtain a plurality of data object information based on the target check point, compare the creation time stamp and the deletion time stamp of each data object information with each active snapshot time stamp to obtain a comparison result, determine a deletable object according to the comparison result, and add the deletable object into a tombstone record.
8. An instant snapshot creation and reading device, characterized in that it comprises a memory, a processor and a computer program stored on said memory and executable on said processor, said computer program being configured to implement the steps of the instant snapshot creation and reading method according to any one of claims 1 to 6.

Description

Instantaneous snapshot creation and reading method, device and equipment Technical Field The present application relates to the field of data processing technologies, and in particular, to a method, an apparatus, and a device for creating and reading an instant snapshot. Background Along with the continuous increase of enterprise data volume and the expansion of mixed load application scenes, a database system needs to be capable of realizing millisecond-level data state capturing and high-performance reading of historical data on the premise of not causing obvious performance influence on online service so as to support key functions such as data version management, real-time analysis and quick recovery and meet the requirements of high-frequency and fine-granularity data snapshot creation and query. The existing database snapshot technology mainly adopts a full-scale replication or incremental snapshot mode, when a snapshot is created, large-scale data scanning and replication are needed, so that the snapshot creation process takes a few minutes to a few hours, service continuity is seriously affected, meanwhile, the reading of historical data is needed to be realized through full-scale scanning and state reconstruction, the performance is low, and the real-time query requirement cannot be met. Therefore, the prior art has the problems of excessively long snapshot creation time and low snapshot reading performance, and cannot provide efficient and instant data history state access capability while ensuring service continuity. The foregoing is provided merely for the purpose of facilitating understanding of the technical solutions of the present application and is not intended to represent an admission that the foregoing is prior art. Disclosure of Invention The application mainly aims to provide a method, a device and equipment for creating and reading instant snapshots, which aim to solve the technical problem of how to improve the efficiency of creating and reading the snapshots. In order to achieve the above object, the present application provides a method for creating and reading an instant snapshot, the method comprising: When a snapshot creation request is received, creating a current snapshot based on the snapshot creation request, and inserting the current snapshot into a snapshot table; When a history snapshot reading request is received, acquiring a corresponding history snapshot time stamp from the snapshot table based on the history snapshot reading request; Inquiring based on the historical snapshot time stamp to obtain a target check point list; Based on the target checkpoint list, loading the data objects existing at the historical snapshot time stamp to construct a target snapshot partition. In an embodiment, when receiving the snapshot creation request, the creating a current snapshot based on the snapshot creation request and inserting the current snapshot into the snapshot table includes: generating a snapshot identifier and a snapshot time stamp when a snapshot creation request is received; determining a snapshot level and a snapshot name according to the snapshot creation request, wherein the snapshot level comprises a cluster level, an account level, a database level and a table level; Extracting corresponding object identification information based on the snapshot level; Determining a current snapshot according to the snapshot identification, the snapshot timestamp, the snapshot name, the snapshot level and the object identification information; And inserting the current snapshot into a snapshot table. In an embodiment, when receiving the history snapshot read request, obtaining the corresponding history snapshot timestamp from the snapshot table based on the history snapshot read request includes: When a historical snapshot reading request is received, determining a historical snapshot name or a query timestamp according to the historical snapshot reading request; searching a historical snapshot from the snapshot table according to the historical snapshot name or the query timestamp; and extracting a historical snapshot time stamp from the historical snapshot. In an embodiment, the querying based on the historical snapshot timestamp to obtain the target checkpoint list includes: Sending the historical snapshot time stamp to a thing node, and acquiring a checkpoint list returned by the thing node based on the historical snapshot time stamp, wherein the checkpoint list comprises a plurality of checkpoints; And when the ending time of the last checkpoint of the checkpoint list is greater than or equal to the historical snapshot time stamp, taking the checkpoint list as a target checkpoint list. In an embodiment, the loading the data object existing at the historical snapshot time stamp based on the target checkpoint list to construct a target snapshot partition includes: obtaining target checkpoints contained in each target checkpoint list, wherein the target checkpoints comprise a p