CN-122019534-A - Data marking method, apparatus, device, storage medium and program product

CN122019534ACN 122019534 ACN122019534 ACN 122019534ACN-122019534-A

Abstract

The embodiment of the application provides a data marking method, a device, equipment, a storage medium and a program product, and relates to the field of database management. The method comprises the steps of generating a first data object according to transaction change data of a source data node and corresponding transaction context information, wherein the first data object comprises transaction change data and transaction context information, the transaction context information comprises at least one of transaction identification, source data node identification, transaction generation time and database ip address, generating a second data object comprising marking information according to marking rule information and the first data object, marking the transaction change data according to the transaction context information by the marking information, and sending the second data object to a target data node. The method provided by the application improves the analyzability of the transaction change data synchronized by the source database.

Inventors

HU RUI
HE SU
WEI QIAN

Assignees

中电科金仓(北京)科技股份有限公司

Dates

Publication Date: 20260512
Application Date: 20251229

Claims (10)

1. A method of marking data, the method comprising: Generating a first data object according to transaction change data of a source data node and corresponding transaction context information, wherein the first data object comprises the transaction change data and the transaction context information, and the transaction context information comprises at least one of a transaction identifier, a source data node identifier, transaction generation time and a database network address; generating a second data object containing marking information according to marking rule information and the first data object, wherein the marking information is a mark for changing data of the transaction according to the transaction context information; And sending the second data object to a target end data node.
2. The method of claim 1, wherein generating a second data object containing marking information based on marking rule information and the first data object comprises: generating a corresponding marking rule mapping table according to the marking rule information, wherein the marking rule mapping table is used for indicating the mapping relation between marking dimensions and marking rules; and generating marking information according to the marking rule mapping table and the transaction context information, wherein the marking information comprises a data source mark, an operation type mark and/or a transaction time mark.
3. The method according to claim 2, wherein the marking information further comprises a transaction generation timestamp and/or a synchronous link processing timestamp.
4. The method of claim 2, wherein the marking rule information includes at least one of a data source identification rule, an operation type identification rule, and a business semantic identification rule.
5. The method of claim 1, wherein the second data object is an SQL statement, and wherein the sending the second data object to the target data node comprises: and submitting the second data object to the target data node through a database transaction mechanism.
6. The method of claim 1, wherein prior to generating the first data object based on the transaction change data and the corresponding transaction context information of the source data node, the method further comprises: Determining a corresponding reporting strategy according to the node state information of the source data node, wherein the reporting strategy is used for indicating the frequency, granularity and/or priority of acquiring transaction change data; And acquiring transaction change data of the source data node based on the reporting strategy.
7. The method of claim 6, wherein the determining a corresponding reporting policy according to the node status information of the source data node includes at least one of: Determining the frequency of acquiring transaction change data according to the resource utilization rate of the source data node; Determining the granularity of acquiring transaction change data according to the input/output rate of the source data node; and determining the priority of the transaction change data according to the type of the transaction change data.
8. The method of claim 1, wherein the transaction context information is obtained in a cache space for caching high frequency transaction context information.
9. A data marking apparatus, the data marking apparatus comprising: the generation module is used for generating a first data object according to transaction change data of the source data node and corresponding transaction context information, wherein the first data object comprises the transaction change data and the transaction context information, and the transaction context information comprises at least one of a transaction identifier, a source data node identifier, transaction generation time and a database ip address; the marking module is used for generating a second data object containing marking information according to marking rule information and the first data object, wherein the marking information is a marking of the transaction change data according to the transaction context information; and the sending module is used for sending the second data object to the target end data node.
10. An electronic device comprising a processor and a memory communicatively coupled to the processor; The memory stores computer-executable instructions; The processor executes computer-executable instructions stored in the memory to implement the method of any one of claims 1 to 8.

Description

Data marking method, apparatus, device, storage medium and program product Technical Field The present application relates to the field of database management, and in particular, to a data marking method, apparatus, device, storage medium, and program product. Background In enterprise-level data management, real-time synchronization of multi-source heterogeneous data is a core requirement for guaranteeing data consistency, supporting business decisions and realizing system integration. In the scenario that a plurality of source databases or data sources synchronize data to a single target database, the data sources are complex, the paths of change are interleaved, and extremely high requirements are placed on the traceability and transparency of the data. For example, in the financial, medical, logistical, etc. industries, multiple business systems (e.g., order systems, inventory systems, customer relationship management systems) may need to synchronize data in real-time to a unified data warehouse or analysis platform to support cross-system business collaboration, risk control, and data analysis. However, in the above implementation, the target database receives the transaction change data of the source database, and it is difficult to analyze the transaction change data. Disclosure of Invention The application provides a data marking method, a device, equipment, a storage medium and a program product, which are used for solving the technical problem of high difficulty in analyzing transaction change data. According to the first aspect, the application provides a data marking method, which comprises the steps of generating a first data object according to transaction change data of a source data node and corresponding transaction context information, wherein the first data object comprises the transaction change data and the transaction context information, and the transaction context information comprises at least one of a transaction identifier, a source data node identifier, transaction generation time and a database network address; generating a second data object containing marking information according to marking rule information and the first data object, wherein the marking information is used for marking transaction change data according to transaction context information; and sending the second data object to the target data node. In a possible implementation manner of the first aspect, generating a second data object containing marking information according to the marking rule information and the first data object includes: generating a corresponding marking rule mapping table according to the marking rule information, wherein the marking rule mapping table is used for indicating the mapping relation between marking dimensions and marking rules; tag information is generated from the tag rule map and the transaction context information, the tag information including a data source tag, an operation type tag, and/or a transaction time tag. In a possible implementation manner of the first aspect, the marking information further includes a transaction generation timestamp and/or a synchronous link processing timestamp. In a possible implementation manner of the first aspect, the marking rule information includes at least one of a data source identification rule, an operation type identification rule, and a service semantic identification rule. In a possible implementation manner of the first aspect, the second data object is an SQL statement, and the sending the second data object to the target data node includes: And submitting the second data object to the target data node through a database transaction mechanism. In a possible implementation manner of the first aspect, before generating the first data object according to the transaction change data of the source data node and the corresponding transaction context information, the method further includes: determining a corresponding reporting strategy according to node state information of the source data node, wherein the reporting strategy is used for indicating the frequency, granularity and/or priority of acquiring transaction change data; based on the reporting strategy, transaction change data of the source data node are obtained. In a possible implementation manner of the first aspect, according to node state information of the source data node, a corresponding reporting policy is determined, including at least one of the following: determining the frequency of acquiring transaction change data according to the resource utilization rate of the source data node; Determining and acquiring granularity of transaction change data according to the input/output rate of the source data node; and determining the priority of the transaction change data according to the type of the transaction change data. In a possible implementation manner of the first aspect, the transaction context information is obtained in a buffer space, and the buffer space is