CN-112749172-B - Data synchronization method and system between cache and database

CN112749172BCN 112749172 BCN112749172 BCN 112749172BCN-112749172-B

Abstract

The invention provides a data synchronization method and a data synchronization system between a cache and a database, wherein the data synchronization system comprises an application service, a RabbitMQ middleware, the cache and the database, the application service updates target data in the database, the application service deletes the target data in the cache, the RabbitMQ middleware triggers the application service to delete the target data in the cache again under the condition of receiving cache deletion failure information sent by the application service until the target data in the cache is successfully deleted, and the data consistency between the cache and the database is ensured in a strategy of updating the data and deleting the cache.

Inventors

XU JIAHONG
CHEN HUABING
HUANG JINLONG
ZENG KE

Assignees

深圳市茁壮网络股份有限公司

Dates

Publication Date: 20260505
Application Date: 20191029

Claims (2)

1. The data synchronization method between the cache and the database is characterized by being applied to a data synchronization system between the cache and the database, wherein the data synchronization system comprises an application service, a RabbitMQ middleware, the cache, a Kafka message queue and the database, the application service comprises a business code sub-service and a non-business code sub-service, and the non-business code sub-service does not need to be associated with business, and the method comprises the following steps: The business code sub-service updates the target data in the database; The database writes the target data into a binlog; The Kafka message queue subscribes to the target data in the binlog log; the non-business code sub-service acquires the target data updated in the database by the business code sub-service by reading the Kafka message queue, and deletes the target data in the cache, and the non-business code sub-service deletes the business code of the cache operation independent of the application service; triggering the application service to delete the target data in the cache again under the condition that the RabbitMQ middleware receives the cache deletion failure information sent by the application service until the target data in the cache is successfully deleted; The database comprises a master library and at least one slave library, the application services comprise a first application service and a second application service, and after the first application service updates the target data in the master library, the method further comprises: The first application service sets an update mark in the cache, wherein the update mark comprises a service code number, a database identifier, a table identifier, a primary key identifier and an expiration time, and the primary library and the secondary library realize synchronous update of the target data within the expiration time; The second application service reads the update mark of the target data in the cache; if the update flag of the target data in the cache is not expired, the second application service reads the target data in the main library; if the update flag of the target data in the cache has expired, the second application service reads the target data in the slave library; Before the non-business code sub-service deletes the target data in the cache, the method further includes: The non-business code sub-service sends a delay message to the RabbitMQ middleware, wherein the delay time of the delay message is the same as the expiration time of the update mark; Triggering the non-business code sub-service to delete the target data in the cache when the RabbitMQ middleware receives the delay message; The database comprises a master library and at least one slave library, the application services comprise a first application service and a second application service, and after the first application service updates the target data in the master library, the method further comprises: The first application service sets an update mark at a local client, wherein the update mark comprises a service code number, a database identifier, a table identifier, a primary key identifier and an expiration time, and the primary library and the secondary library realize synchronous update of the target data within the expiration time; if the update flag of the target data in the local client is not expired, the first application service reads the target data in the main library; if the update flag of the target data in the local client has expired, the first application service reads the target data in the slave library; And the second application service sets the target data read from the main library into a cache to ensure that the new value is in the cache, and ensures that the data in the main library, the slave library and the cache are consistent after the slave library synchronous data is completed.
2. The data synchronization system between the cache and the database is characterized by comprising an application service, a RabbitMQ middleware, a cache, a Kafka message queue and a database, wherein the application service comprises a business code sub-service and a non-business code sub-service, and the non-business code sub-service does not need to be associated with business; The business code sub-service updates the target data in the database; The database writes the target data into a binlog; The Kafka message queue subscribes to the target data in the binlog log; the non-business code sub-service acquires the target data updated in the database by the business code sub-service by reading the Kafka message queue, and deletes the target data in the cache, and the non-business code sub-service deletes the business code of the cache operation independent of the application service; The RabbitMQ middleware is configured to trigger the application service to delete the target data in the cache again until the target data in the cache is deleted successfully when receiving the cache deletion failure information sent by the application service; the database comprises a master library and at least one slave library, and the application service comprises a first application service and a second application service; The first application service is configured to set an update flag in the cache after updating the target data in the master library, where the update flag includes a service code number, a database identifier, a table identifier, a master key identifier, and an expiration time, and the master library and the slave library implement synchronous update of the target data within the expiration time; the second application service is used for reading the update mark of the target data in the cache, and reading the target data in the master library if the update mark of the target data in the cache is not expired; The non-business code sub-service is further configured to send a delay message to the rabitimq middleware before deleting the target data in the cache, where a delay time of the delay message is the same as an expiration time of the update flag; The RabbitMQ middleware is further configured to trigger the non-service code sub-service to delete the target data in the cache when the delay message is received; the database comprises a master library and at least one slave library, and the application service comprises a first application service and a second application service; The first application service is configured to set an update flag at a local client after updating the target data in the master library, where the update flag includes a service code number, a database identifier, a table identifier, a master key identifier, and an expiration time, and the master library and the slave library implement synchronous update of the target data within the expiration time; the first application service is further configured to read the target data in the master library if the update flag of the target data in the local client is not expired; if the update mark of the target data in the local client has expired, reading the target data in the slave library; And the second application service sets the target data read from the main library into a cache to ensure that the new value is in the cache, and ensures that the data in the main library, the slave library and the cache are consistent after the slave library synchronous data is completed.

Description

Data synchronization method and system between cache and database Technical Field The invention relates to the technical field of databases, in particular to a data synchronization method and a data synchronization system between a cache and a database. Background In the technical field of databases, in order to reduce the load pressure of the database and improve the response efficiency to users, the data of the database are generally synchronized into a cache, and the users can obtain corresponding data by accessing the cache. How to ensure the consistency of the database and the cache during data updating is a technical problem to be solved in the field. In the prior art, in the strategy of updating the database and deleting the cache, if the cache deletion fails, the program considers that data exists in the cache at the moment, the user acquires returned data from the cache, and the data acquired by the user at the moment is inconsistent with the data in the database because the database data is updated, namely the database is inconsistent with the cache. Disclosure of Invention In view of this, the present invention provides a method for synchronizing data between a cache and a database, which ensures data consistency between the cache and the database in a policy of updating data before deleting the cache. In order to achieve the above purpose, the specific technical scheme provided by the invention is as follows: A data synchronization method between a cache and a database, applied to a data synchronization system between the cache and the database, the data synchronization system comprising an application service, a rabitmq middleware, the cache and the database, the method comprising: The application service updates the target data in the database; The application service deletes the target data in the cache; and under the condition that the RabbitMQ middleware receives the cache deletion failure information sent by the application service, triggering the application service to delete the target data in the cache again until the target data in the cache is successfully deleted. Optionally, the application service includes a business code sub-service and a non-business code sub-service; The application service updates the target data in the database, specifically: The business code sub-service updates the target data in the database; the application service deletes the target data in the cache, specifically: And the non-business code sub-service acquires the target data updated in the database by the business code sub-service, and deletes the target data in the cache. Optionally, the data synchronization system further includes a Kafka message queue, and after the business code sub-service updates the target data in the database, the method further includes: The database writes the target data into a binlog; The Kafka message queue subscribes to the target data in the binlog log; the non-business code sub-service obtaining the target data updated in the database by the business code sub-service, comprising: and the non-business code sub-service acquires the target data updated in the database by the business code sub-service through reading the Kafka message queue. Optionally, the database includes a master library and at least one slave library, the application service includes a first application service and a second application service, and after the first application service updates the target data in the master library, the method further includes: The first application service sets an update mark in the cache, wherein the update mark comprises a service code number, a database identifier, a table identifier, a primary key identifier and an expiration time, and the primary library and the secondary library realize synchronous update of the target data within the expiration time; The second application service reads the update mark of the target data in the cache; if the update flag of the target data in the cache is not expired, the second application service reads the target data in the main library; And if the update flag of the target data in the cache is expired, the second application server reads the target data in the slave library. Optionally, the database includes a master library and at least one slave library, the application service includes a first application service and a second application service, and after the first application service updates the target data in the master library, the method further includes: The first application server sets an update mark at a local client, wherein the update mark comprises a service code number, a database identifier, a table identifier, a primary key identifier and an expiration time, and the primary library and the secondary library realize synchronous update of the target data within the expiration time; If the update flag of the target data in the local client is not expired, the first application server reads the target data in the m