Search

CN-122019513-A - Data processing method, device, electronic equipment and computer storage medium

CN122019513ACN 122019513 ACN122019513 ACN 122019513ACN-122019513-A

Abstract

The application provides a data processing method, a device, electronic equipment and a computer storage medium, wherein the method comprises the steps of obtaining historical data corresponding to target service from a preset cache when target data corresponding to the target service is received, determining whether the target data is full data, determining to-be-updated data according to the target data and the historical data when the target data is the full data, wherein the to-be-updated data comprises to-be-deleted data, and updating the historical data according to the to-be-updated data to obtain updated historical data. The embodiment provided by the scheme can improve the consistency of the data and save the storage resources.

Inventors

  • LIANG JINGBIAO

Assignees

  • 三六零数字安全科技集团有限公司

Dates

Publication Date
20260512
Application Date
20241111

Claims (10)

  1. 1. A method of data processing, comprising: When target data corresponding to target service is received, acquiring historical data corresponding to the target service from a preset cache; Determining whether the target data is full data; Under the condition that the target data is full data, determining data to be updated according to the target data and the historical data, wherein the data to be updated comprises data to be deleted; And updating the historical data according to the data to be updated to obtain updated historical data.
  2. 2. The method of claim 1, wherein after receiving the target data corresponding to the target service, the method further comprises: judging whether the target data is data corresponding to the target service received for the first time; if not, executing the step of acquiring the historical data corresponding to the target service from the preset cache; If yes, storing the target data into the preset cache, and taking the target data as history data corresponding to the target service, which is obtained when new target data corresponding to the target service is received next time.
  3. 3. The method of claim 1, wherein the determining whether the target data is full data comprises: acquiring a configuration information file corresponding to the target service; inquiring a data identifier corresponding to the target service from the configuration information file; And determining whether the target data is full data according to the data identification.
  4. 4. The method of claim 1, wherein the target data includes at least one first sub-data and a first identification of each first sub-data, the history data includes at least one second data and a second identification of each second sub-data, and the determining the data to be updated based on the target data and the history data includes: Determining a first difference set of at least one first identifier included in the target data and at least one second identifier included in the history data, the first difference set being a set of second identifiers belonging to the history data and not belonging to the target data; judging whether the first difference set is an empty set or not; And under the condition that the first difference set is not an empty set, determining second sub-data corresponding to a second identifier in the first difference set as data to be deleted.
  5. 5. The method of claim 1, wherein the target data includes at least one first sub-data and a first identifier of each first sub-data, the history data includes at least one second data and a second identifier of each second sub-data, the data to be updated further includes data to be modified, and the determining the data to be updated according to the target data and the history data includes: Determining an intersection of at least one first identifier included in the target data and at least one second identifier included in the history data, the intersection being a set of common identifiers belonging to both the target data and the history data; for each shared identifier included in the intersection, determining whether first sub-data corresponding to the shared identifier is consistent with second sub-data corresponding to the shared identifier; And under the condition that the first sub data corresponding to the shared identifier is inconsistent with the second sub data corresponding to the shared identifier, determining the second sub data corresponding to the shared identifier as data to be modified.
  6. 6. The method of claim 4, wherein updating the history data according to the data to be updated to obtain updated history data comprises: Deleting the data to be deleted included in the data to be updated from the historical data to obtain updated historical data.
  7. 7. The method of claim 4, wherein updating the history data according to the data to be updated to obtain updated history data comprises: generating a label to be deleted for the data to be deleted contained in the historical data to obtain updated historical data; the method further comprises the steps of: and deleting the second data corresponding to the label to be deleted in the historical data based on a preset period.
  8. 8. A data processing apparatus, comprising: The acquisition module is used for acquiring historical data corresponding to the target service from a preset cache when receiving the target data corresponding to the target service; A first determining module, configured to determine whether the target data is full data; the second determining module is used for determining data to be updated according to the target data and the historical data when the target data is full data, wherein the data to be updated comprises data to be deleted; And the updating module is used for updating the historical data according to the data to be updated to obtain updated historical data.
  9. 9. An electronic device is characterized by comprising a processor and a memory; the memory stores a computer program adapted to be loaded by the processor and to perform the steps of the method according to any one of claims 1 to 7.
  10. 10. A computer storage medium storing a plurality of instructions adapted to be loaded by a processor and to perform the steps of the method according to any one of claims 1 to 7.

Description

Data processing method, device, electronic equipment and computer storage medium Technical Field The present application relates to the field of data synchronization technologies, and in particular, to a data processing method, a data processing device, an electronic device, and a computer storage medium. Background In the development process of an application program, data of some third party platforms are usually accessed, for example, a third party account number, a third party organization architecture, third party message pushing, third party on-shelf product display and the like are used. When the third party platform provides data, the full amount of data is usually provided, or the full amount of data is provided for the first time and then incremental data is provided. However, in the data providing process, if the third party platform cannot provide the identification or the deletion record of the deleted data, the application program is difficult to know which data are no longer present, so that accumulation and redundancy of historical data are caused, and waste of system storage resources is caused. Disclosure of Invention The embodiment of the application provides a data processing method, a device, electronic equipment and a computer storage medium, which can avoid data expansion, maintain the simplicity and maintainability of a data structure and save system storage resources. The technical scheme is as follows: In a first aspect, an embodiment of the present application provides a data processing method, including: when target data corresponding to target service is received, acquiring historical data corresponding to the target service from a preset cache; determining whether the target data is full data; Determining data to be updated according to the target data and the historical data when the target data is full data, wherein the data to be updated comprises data to be deleted; And updating the historical data according to the data to be updated to obtain updated historical data. In one possible implementation manner, after the receiving the target data corresponding to the target service, the method further includes: Judging whether the target data is data corresponding to the target service received for the first time; If not, executing the step of acquiring the historical data corresponding to the target service from the preset cache; If yes, storing the target data into the preset cache, and taking the target data as history data corresponding to the target service, which is obtained when new target data corresponding to the target service is received next time. In one possible implementation manner, the determining whether the target data is full data includes: Acquiring a configuration information file corresponding to the target service; inquiring a data identifier corresponding to the target service from the configuration information file; and determining whether the target data is full data according to the data identification. In one possible implementation manner, the target data includes at least one first sub data and a first identifier of each first sub data, the history data includes at least one second data and a second identifier of each second sub data, and the determining the data to be updated according to the target data and the history data includes: Determining a first difference set of at least one first identifier included in the target data and at least one second identifier included in the history data, wherein the first difference set is a set of second identifiers which belong to the history data and do not belong to the target data; judging whether the first difference set is an empty set or not; and determining second sub-data corresponding to the second identifier in the first difference set as data to be deleted under the condition that the first difference set is not an empty set. In one possible implementation manner, the target data includes at least one first sub data and a first identifier of each first sub data, the history data includes at least one second data and a second identifier of each second sub data, the data to be updated further includes data to be modified, and the determining the data to be updated according to the target data and the history data includes: Determining an intersection of at least one first identifier included in the target data and at least one second identifier included in the history data, the intersection being a set of common identifiers belonging to both the target data and the history data; For each common identifier included in the intersection, determining whether the first sub-data corresponding to the common identifier is identical to the second sub-data corresponding to the common identifier; And determining the second sub-data corresponding to the common identifier as data to be modified under the condition that the first sub-data corresponding to the common identifier is inconsistent with the second sub-