CN-122001938-A - Multi-strategy data synchronization and trigger control method and system for cloud primary micro-service of universal financial ecological connector

CN122001938ACN 122001938 ACN122001938 ACN 122001938ACN-122001938-A

Abstract

The invention relates to the technical field of cloud primary computing and distributed data synchronous control, in particular to a multi-strategy data synchronous and trigger control method and system of cloud primary micro-service of a universal financial ecological connector, comprising the steps of S1 deploying the universal financial connector in a cloud primary environment, and comprising synchronous arrangement micro-service, source end adaptation micro-service, target end adaptation micro-service, trigger control micro-service, state storage service and event bus; S2, synchronously arranging and distributing an execution model according to a task identification selection strategy; S3, the source end sends data through event bus according to the model, the target end writes and records check points in two stages, S4 the source/target end checks the data into state storage service, and submits to the second stage if the check data is consistent, S5 triggers control to generate instructions by change/resource/fault/voice recognition/voice control event, synchronously programs migration and triggers start/stop/recovery/current limiting/switching/replay. The invention realizes the cooperative control of synchronous delivery and trigger control.

Inventors

CHEN YILING
YAO MINGMING
LIANG BO

Assignees

上海东方屹腾科技有限公司

Dates

Publication Date: 20260508
Application Date: 20260326

Claims (10)

1. A multi-strategy data synchronization and trigger control method of cloud primary micro-service of a universal financial ecological connector is characterized by comprising the following steps: The method comprises the steps that S1, a universal financial ecological connector is deployed in a cloud primary micro-service running environment, wherein the universal financial ecological connector comprises synchronous arrangement micro-service, source end adaptation micro-service, target end adaptation micro-service, trigger control micro-service, state storage service and an event bus; S2, determining strategy information for the synchronous task identifier from a synchronous strategy set aiming at the synchronous task identifier by the synchronous arrangement micro-service, generating a strategy execution model based on the strategy information, and distributing the strategy execution model to the source end adaptation micro-service, the target end adaptation micro-service and the trigger control micro-service; The target end adaptation micro-service performs two-stage writing on the data to be synchronized, wherein the two-stage writing comprises first-stage writing and second-stage writing, and records checkpoints respectively corresponding to the first-stage writing and the second-stage writing; S4, the source end adaptation micro-service generates source end verification data corresponding to the data to be synchronized, the target end adaptation micro-service generates target end verification data corresponding to the first-stage writing, and the source end verification data and the target end verification data are written into the state storage service; And S5, the trigger control micro-service generates a trigger instruction based on a trigger input event, wherein the trigger input event at least comprises one or more of a change event, a resource index event and a fault event, and the synchronization orchestration micro-service executes state migration and triggers actions on the synchronous task identifier according to the strategy execution model, and the actions at least comprise one or more of starting synchronization, suspending synchronization, restoring synchronization, current limiting synchronization, switching synchronization strategies and replaying check points.
2. The method of claim 1, wherein the policy information comprises a policy declaration object including at least a set of states, a set of state transition conditions, and a set of actions, wherein the synchronization orchestration micro-service generates the policy enforcement model based on the policy declaration object compilation, and wherein the policy enforcement model is used to define a running state of the synchronization task identity and a trigger condition for the action.
3. The method of claim 2, wherein the first phase write is a pre-write and the second phase write is a commit write, wherein the checkpoints include a pre-write checkpoint and a commit checkpoint, and wherein the target-side adapter micro-service writes the pre-write checkpoint to the state storage service after performing the pre-write and writes the commit checkpoint to the state storage service after performing the commit write.
4. A method according to claim 3, wherein the source verification data comprises a source consistency fingerprint The target verification data comprises target consistency fingerprints And the preset consistency condition includes: Wherein, the And presetting a fingerprint deviation threshold value.
5. A method according to claim 3, wherein the source adaptation microservice generates a causal sequence number for a change event And comparing the causal sequence number The target end adaptation microservice is transmitted together with the change event and is based on the causal serial number Performing deduplication and ordered pre-write on change events, wherein: Wherein, the Numbering the partitions; sequence numbers of the inner source ends of the partitions; Is a preset integer greater than the maximum source end sequence number in the partition.
6. The method of claim 2, wherein the synchronization orchestration micro-service generates a policy consistency check value for the policy declaration object And checking the policy consistency value The source end adaptation micro-service, the target end adaptation micro-service and the trigger control micro-service are based on the strategy consistency check value when receiving the strategy execution model Performing a consistency check, wherein: Wherein, the Declaring the first of the objects for the policy The strategy parameters are subjected to preset encoding to obtain non-negative integers; Is the number of policy parameters; Is a preset modulus.
7. The method of claim 1, wherein the trigger control micro service calculates the trigger decision value according to the following calculation formula And the trigger decision value is set Compare to a set of thresholds to generate a trigger instruction: Wherein, the The number of messages to be consumed for the event bus associated with the synchronous task identification; the average time delay is written into for the first stage of the micro service of the target terminal adaptation; adapting a resource utilization rate associated with the micro service to the target end in the cloud primary micro service running environment; the first stage writing failure times associated with the synchronous task identification in a preset statistical window are obtained; To the point of Is a preset weight coefficient.
8. The method of claim 7, wherein when the trigger instruction comprises a current limited synchronization, the synchronization orchestration micro-service update sending rate parameter And issuing to the source-side adaptive micro service, wherein: Wherein, the Is the minimum transmission rate; Is the maximum transmission rate; Is the reference transmission rate; Is the rate attenuation coefficient; The number of messages to be consumed for the event bus.
9. The method of claim 7, wherein the policy enforcement model includes a transaction calendar window constraint including a transaction freeze zone When the current time is The method meets the following conditions: When the synchronization orchestration micro-service eliminates the switching synchronization strategy and the replay check point from the executable actions, and selects actions to be executed from the rest actions according to the trigger instruction, wherein, Is the time stamp integer corresponding to the current time.
10. The system for multi-strategy data synchronization and triggering control of cloud primary micro-service of the universal financial ecological connector is characterized by comprising the following components: Synchronously arranging a micro-service, a source end adaptation micro-service, a target end adaptation micro-service, a trigger control micro-service, a state storage service and an event bus; the synchronous arranging micro-service is used for determining strategy information, compiling and generating a strategy execution model, distributing the strategy execution model to the source end adaptation micro-service, the target end adaptation micro-service and the trigger control micro-service, triggering actions according to the trigger instruction and writing trigger action results into the state storage service; The source end adaptation microservice is used for outputting data to be synchronized according to the strategy execution model and generating source end verification data; The target end adaptation microservice is used for performing two-stage writing on the data to be synchronized and generating target end verification data, and performing second-stage writing and recording a commit check point when the source end verification data and the target end verification data meet preset consistency conditions; The trigger control micro-service is used for generating a trigger instruction based on a trigger input event; the state storage service is used for storing the strategy information, the strategy execution model association information, the source end verification data, the target end verification data and the commit checkpoint; The event bus is used for transmitting the data to be synchronized.

Description

Multi-strategy data synchronization and trigger control method and system for cloud primary micro-service of universal financial ecological connector Technical Field The invention relates to the technical field of cloud primary computing and distributed data synchronous control, in particular to a multi-strategy data synchronous and trigger control method and system of cloud primary micro-service of a universal financial ecological connector. Background With the continuous expansion of the ecology of general financial business, the requirements of cross-system data sharing and consistency linkage exist among the systems of bank cores, payment clearing, wind control anti-fraud, credit investigation, marketing, accounting and account checking, external cooperation institutions and the like. In order to meet real-time or quasi-real-time data consumption, account checking, risk triggering and business linkage, synchronization of source system data change to one or more target systems is usually realized through a universal financial ecological connector in engineering practice, and control actions such as pause, recovery, current limiting, replay and the like are triggered according to business or running states when necessary. In recent years, the deployment form of the connector gradually evolves towards cloud native micro services, generally, an event bus is used for carrying transmission of data to be synchronized, and a plurality of micro services cooperatively complete data extraction, transmission and landing. In the prior art, a modified data capturing and copying scheme oriented to a distributed data source is disclosed. For example, the scheme of publication number CN110249321a discloses a system and method for capturing change data from a distributed data source for use by heterogeneous targets, which primarily surrounds change data capture and data replication, describes mechanisms including initial loading, duplicate data deletion, and recovery, and can use the captured change data for targets such as databases or message queues. The scheme can solve the problems of 'distributed source end change acquisition and heterogeneous target output' to a certain extent, and is suitable for a general scene of data replication and synchronization. However, under the cloud primary micro-service architecture, the universal financial ecological connector generally has the engineering characteristics that a synchronous link is split into a plurality of micro-services to cooperate in parallel, the link state can dynamically change along with elastic expansion, retry and fault recovery, data to be synchronized is transmitted among different micro-services through an event bus, partial failure retry and delay difference caused by consumption backlog can occur in target-end writing, meanwhile, synchronous control is not only dependent on a data change event, but also can be influenced by resource indexes, fault events and the like, and control actions such as start, pause, recovery, current limiting, switching strategies, replay check points and the like need to be executed on synchronous tasks. When the conditions are overlapped, starting from a change capturing and copying mechanism focused by the prior art (for example, the disclosure of CN110249321 a), although the capabilities of obtaining changed data, removing duplication, recovering and the like can be provided, there may still be a situation that a clear unified constraint mechanism is lacking in aspects such as "synchronous control action, how a consistent constraint relationship is formed between a check point record and an actual writing state of a target end after cloud native micro-service splitting", and "how a verifiable control is performed on a final submitting time of writing of the target end under the condition that multi-micro-service collaboration and retry coexist". Therefore, the main technical problem to be solved by the invention is how to realize the commit control based on the consistency judgment of the verification data of the source end and the target end in the target end writing stage in the pan-financial ecological connector of the cloud primary micro-service architecture, and to enable the commit control to be consistent with the synchronous action driven by the trigger instruction and the check point record, so that the cooperative controllability of the data synchronization and the trigger control is realized. Disclosure of Invention In order to overcome the technical defects, the invention aims to provide a multi-strategy data synchronization and trigger control method and system of cloud primary micro-service of a universal financial ecological connector. According to the method, constraint synchronous execution is distributed through a strategy execution model, two-stage writing is adopted at a target end, the source end/target end verification data meet the preset consistency condition to serve as a second stage submitting ga