CN-122019269-A - Database log analysis recovery method, device, equipment, medium and program product

CN122019269ACN 122019269 ACN122019269 ACN 122019269ACN-122019269-A

Abstract

The application provides a database log analysis recovery method, a device, equipment, a medium and a program product, and relates to the field of data processing. The method comprises the steps of periodically generating a snapshot file of cache record data involved in a database log analysis process, recording a snapshot identifier corresponding to the snapshot file, updating version information of a data dictionary, binding the snapshot identifier with the version information of the data dictionary, matching corresponding target version information from the data dictionary according to target snapshot identifiers when the database log analysis is interrupted, recovering the state of the cache record data based on the target version information, and determining the starting position of the database log analysis.

Inventors

LI JIALUO
HE SU
WEI QIAN

Assignees

中电科金仓(北京)科技股份有限公司

Dates

Publication Date: 20260512
Application Date: 20251229

Claims (10)

1. The database log analysis and recovery method is characterized by comprising the following steps of: Periodically generating a snapshot file of cache record data involved in the database log analysis process, and recording a snapshot identifier corresponding to the snapshot file, wherein the snapshot file refers to a persistent copy of a memory data state generated through copy-on-write or redirection-on-write; updating version information of the data dictionary, and binding the snapshot identification with the version information of the data dictionary, wherein the version information is used for identifying the association relation between the snapshot file and the state of the data dictionary; when the analysis of the database log is interrupted, matching corresponding target version information from the data dictionary according to a target snapshot identifier, wherein the target snapshot identifier is the snapshot identifier closest to the analysis interruption time of the database log; and recovering the state of the cached record data based on the target version information, and determining the starting position of database log analysis.
2. The method of claim 1, wherein periodically generating a snapshot file of cached record data involved in a database log parsing process comprises: And periodically detecting whether long transactions exist in the database log analysis process, and if so, generating a snapshot file of the related cache record data, wherein the long transactions refer to transactions which have longer duration and are not submitted yet in database operation.
3. The method of claim 2, wherein the snapshot file comprises a delta snapshot file containing difference information, and wherein the generating the snapshot file of the related cached record data if there is a long transaction comprises: and if the long transaction exists, generating the increment snapshot file containing the difference information.
4. The method of claim 3, wherein the generating the delta snapshot file containing difference information comprises: Comparing the current source snapshot file with the cache record data of the snapshot file generated last time to obtain the difference information, wherein the current source snapshot file comprises all cache record data related in the current database log analysis process; And generating the increment snapshot file according to the difference information.
5. The method according to any one of claims 1-4, wherein updating version information of the data dictionary comprises: and allocating a unique version number for each snapshot file, and writing the version number into a corresponding field of the data dictionary to obtain updated version information of the data dictionary.
6. The method of claim 5, wherein the target snapshot identifier comprises a global snapshot identifier, wherein the matching corresponding target version information from the data dictionary according to the target snapshot identifier comprises: in the distributed database architecture, the global snapshot identification is obtained from a coordination node, wherein the coordination node is a node responsible for managing the global snapshot identification in the distributed database architecture; The target version information that matches the data dictionary is identified based on the global snapshot.
7. The method according to any one of claims 1-4, further comprising: mapping the snapshot file into a memory mapping file, and performing read-write operation through an operating system memory management mechanism.
8. A database log parsing and recovering apparatus, comprising: The system comprises a recording module, a recording module and a database log analyzing module, wherein the recording module is used for periodically generating a snapshot file of cache record data involved in the database log analyzing process and recording a snapshot identifier corresponding to the snapshot file; The binding module is used for updating version information of the data dictionary and binding the snapshot identification with the version information of the data dictionary, wherein the version information is used for identifying the association relation between the snapshot file and the state of the data dictionary; The matching module is used for matching corresponding target version information from the data dictionary according to a target snapshot identifier when the analysis of the database log is interrupted, wherein the target snapshot identifier is the snapshot identifier closest to the analysis interruption time of the database log; And the determining module is used for recovering the state of the cache record data based on the target version information and determining the starting position of database log analysis.
9. An electronic device is characterized by comprising a memory and a processor; The memory stores computer-executable instructions; The processor executing computer-executable instructions stored in the memory, causing the processor to perform the method of any one of claims 1-7.
10. A computer readable storage medium having stored therein computer executable instructions which when executed by a processor are adapted to carry out the method of any one of claims 1-7.

Description

Database log analysis recovery method, device, equipment, medium and program product Technical Field The present application relates to the field of data processing, and in particular, to a method, apparatus, device, medium, and program product for database log parsing recovery. Background In a database system, a log analysis technology is critical to data recovery, disaster recovery, backup and the like. The fault recovery and the master-slave synchronization are realized by analyzing the pre-written log (WAL, write Ahead Logging). However, long transactions can result in breakpoint information (e.g., oldestLSN (Oldest Log Sequence Number, the oldest log sequence number)) not being updated in time, forcing reparse from an earlier location upon recovery, extending recovery time and increasing storage burden. In the prior art, database log analysis is firstly split into transaction records which are temporarily stored in a memory, and DML (Data Manipulation Language ) is generated by combining a data dictionary after being submitted. Breakpoint resume dependencies OldestLSN and CommitLSN (Commit Log Sequence Number ) determine the starting point to trace back to BackupLSN (Backup Log Sequence Number ) to extract RedoLSN (Redo Log Sequence Number, redo log sequence number) of the most recent dictionary version less than OldestLSN. However, in the prior art, oldestLSN cannot be advanced due to long transactions, and the recovery needs to be started from a premature LSN (Log Sequence Number ) position for analysis, so that the recovery time is greatly increased. Disclosure of Invention The application provides a database log analysis and recovery method, device, equipment, medium and program product, which are used for solving the technical problems that OldestLSN cannot be advanced due to long transactions, and recovery time consumption is greatly increased because analysis is needed from a premature LSN position during recovery. In a first aspect, the present application provides a method for recovering database log parsing, the method comprising: Periodically generating a snapshot file of cache record data involved in the database log analysis process, and recording a snapshot identifier corresponding to the snapshot file, wherein the snapshot file refers to a persistent copy of a memory data state generated through copy-on-write or redirection-on-write; updating version information of the data dictionary, and binding the snapshot identification with the version information of the data dictionary, wherein the version information is used for identifying the association relation between the snapshot file and the state of the data dictionary; when the analysis of the database log is interrupted, matching corresponding target version information from the data dictionary according to a target snapshot identifier, wherein the target snapshot identifier is the snapshot identifier closest to the analysis interruption time of the database log; and recovering the state of the cached record data based on the target version information, and determining the starting position of database log analysis. In one possible implementation manner, the periodically generating the snapshot file of the cache record data involved in the database log parsing process includes: And periodically detecting whether long transactions exist in the database log analysis process, and if so, generating a snapshot file of the related cache record data, wherein the long transactions refer to transactions which have longer duration and are not submitted yet in database operation. In one possible implementation manner, the snapshot file includes an incremental snapshot file including difference information, and the generating the snapshot file of the related cached record data if there is a long transaction includes: and if the long transaction exists, generating the increment snapshot file containing the difference information. In a possible implementation manner, the generating the incremental snapshot file including the difference information includes: Comparing the current source snapshot file with the cache record data of the snapshot file generated last time to obtain the difference information, wherein the current source snapshot file comprises all cache record data related in the current database log analysis process; And generating the increment snapshot file according to the difference information. In one possible implementation manner, the updating the version information of the data dictionary includes: and allocating a unique version number for each snapshot file, and writing the version number into a corresponding field of the data dictionary to obtain updated version information of the data dictionary. In a possible implementation manner, the target snapshot identifier includes a global snapshot identifier, and the matching corresponding target version information from the data dictionary according to the target snapshot identifier i