Search

CN-117093560-B - Data processing method and related device

CN117093560BCN 117093560 BCN117093560 BCN 117093560BCN-117093560-B

Abstract

The application discloses a data processing method and a related device, which dynamically adjust the data format of service data in a shared memory, solve the problem that the data format is not matched due to the addition of the service data, and improve the use experience of a target object. The method comprises the steps of obtaining a first data description file, wherein the first data description file describes a data format of first service data, the first service data is the service data obtained when a loading service program is in a first state, determining data difference information between the first data description file and a second data description file when the version of the first data description file is higher than that of the second data description file, the second data description file describes a data format of second service data stored in a shared memory, the second service data is the service data obtained before the loading service program is in the first state, and adjusting the data format of the second service data based on the data difference information and the data format of the first service data.

Inventors

  • YANG YANG

Assignees

  • 腾讯科技（深圳）有限公司

Dates

Publication Date
20260505
Application Date
20220513

Claims (15)

  1. 1. A method of data processing, comprising: Acquiring a first data description file, wherein the first data description file is used for describing a data format of first service data, and the first service data is the service data obtained when a loading service program is in a first state; When the version of the first data description file is higher than that of the second data description file, determining data difference information between the first data description file and the second data description file, wherein the second data description file is used for describing a data format of second service data stored in a shared memory, the second service data is service data obtained before the service program is loaded in the first state, and the first service data is different from the second service data; traversing a message structure in the second data description file to obtain fields in the message structure; acquiring a field type of a current field in a data format of the second service data from the field in the message structure; Based on the data difference information, adjusting the field data corresponding to the field type of the current field according to the field length in the data format of the first service data, including: when the field type of the current field comprises an array type, traversing subfields contained in a field corresponding to the array type to acquire the field type of the current subfields; When the field type of the current sub-field is the basic type, based on the data difference information, adjusting the field data corresponding to the basic type in the second service data according to a first target field length, wherein the first target field length is the minimum value of the field length of the basic type in the data format of the first service data and the field length of the basic type in the second service data, or And when the field type of the current subfield is a character string type, adjusting the field data corresponding to the character string type in the second service data according to a second target field length based on the data difference information, wherein the second target field length is the minimum value of the field length of the character string type in the data format of the first service data and the field length corresponding to the character string type in the second service data.
  2. 2. The method of claim 1, wherein the obtaining the first data description file comprises: When the second data description file is stored in the shared memory, acquiring file configuration information of the second data description file; And when the file configuration information of the target data description file is different from the file configuration information of the second data description file, acquiring the first data description file from the configuration file, wherein the target data description file is the data description file when the service program is currently operated.
  3. 3. The method of claim 2, wherein the file configuration information of the target data description file is different from the file configuration information of the second data description file, comprising: Acquiring a version number of the target data description file from file configuration information of the target data description file; acquiring a version number of the second data description file from file configuration information of the second data description file; And when the version number of the target data description file is different from that of the second data description file, determining that the file configuration information of the target data description file is different from that of the second data description file.
  4. 4. The method of claim 1, wherein the adjusting the field data corresponding to the field type of the current field according to the field length in the data format of the first service data based on the data difference information includes: And when the field type of the current field is a basic type, based on the data difference information, adjusting the field data corresponding to the basic type according to a first field length, wherein the first field length is the minimum value of the field length of the basic type in the data format of the first service data and the field length of the basic type in the second service data.
  5. 5. The method of claim 1, wherein the adjusting the field data corresponding to the field type of the current field according to the field length in the data format of the first service data based on the data difference information includes: and when the field type of the current field is a character string type, adjusting the field data corresponding to the character string type according to a second field length based on the data difference information, wherein the second field length is the minimum value of the field length of the character string type in the data format of the first service data and the field length corresponding to the character string type in the second service data.
  6. 6. A method according to any one of claims 1 to 3, wherein said determining data difference information between the first data description file and the second data description file comprises: acquiring file configuration information of the first data description file and file configuration information of the second data description file, wherein the file configuration information of the first data description file comprises version information and data information of the first data description file, and the file configuration information of the first data description file comprises version information and data information of the first data description file; Data difference information between the first data description file and the second data description file is determined based on file configuration information of the first data description file and file configuration information of the second data description file.
  7. 7. A data processing apparatus, comprising: The system comprises an acquisition unit, a service loading unit and a service loading unit, wherein the acquisition unit is used for acquiring a first data description file, the first data description file is used for describing a data format of first service data, and the first service data is the service data obtained when a loading service program is in a first state; The processing unit is used for determining data difference information between the first data description file and the second data description file when the version of the first data description file is higher than that of the second data description file, wherein the second data description file is used for describing a data format of second service data stored in a shared memory, the second service data is service data obtained before the service program is loaded in the first state, and the first service data is different from the second service data; The processing unit is used for traversing the message structure in the second data description file to acquire fields in the message structure; acquiring a field type of a current field in a data format of the second service data from the field in the message structure; the processing unit is configured to adjust field data corresponding to a field type of the current field according to a field length in a data format of the first service data based on the data difference information, and includes: when the field type of the current field comprises an array type, traversing subfields contained in a field corresponding to the array type to acquire the field type of the current subfields; When the field type of the current sub-field is the basic type, based on the data difference information, adjusting the field data corresponding to the basic type in the second service data according to a first target field length, wherein the first target field length is the minimum value of the field length of the basic type in the data format of the first service data and the field length of the basic type in the second service data, or And when the field type of the current subfield is a character string type, adjusting the field data corresponding to the character string type in the second service data according to a second target field length based on the data difference information, wherein the second target field length is the minimum value of the field length of the character string type in the data format of the first service data and the field length corresponding to the character string type in the second service data.
  8. 8. The data processing apparatus of claim 7, wherein, The acquiring unit is used for acquiring file configuration information of the second data description file when the second data description file is stored in the shared memory; And the processing unit is used for acquiring the first data description file from the configuration file when the file configuration information of the target data description file is different from the file configuration information of the second data description file, wherein the target data description file is the data description file when the service program is currently operated.
  9. 9. The data processing apparatus according to claim 8, wherein the obtaining unit is configured to obtain a version number of the target data description file from file configuration information of the target data description file; And the processing unit is used for determining that the file configuration information of the target data description file is different from the file configuration information of the second data description file when the version number of the target data description file is different from the version number of the second data description file.
  10. 10. The data processing device according to claim 7, wherein the processing unit is configured to, when a field type of the current field is a base type, perform adjustment processing on field data corresponding to the base type according to a first field length based on the data difference information, where the first field length is a minimum value of a field length of the base type in a data format of the first service data and a field length of the base type in the second service data.
  11. 11. The data processing device according to claim 7, wherein the processing unit is configured to, when the field type of the current field is a string type, perform adjustment processing on field data corresponding to the string type according to a second field length based on the data difference information, where the second field length is a minimum value of a field length of a string type in a data format of the first service data and a field length of the string type corresponding to the second service data.
  12. 12. The data processing apparatus according to any one of claims 7 to 9, wherein the acquiring unit is configured to acquire file configuration information of the first data description file and file configuration information of the second data description file, the file configuration information of the first data description file including version information and data information of the first data description file; the processing unit is used for determining data difference information between the first data description file and the second data description file based on the file configuration information of the first data description file and the file configuration information of the second data description file.
  13. 13. A data processing apparatus comprising an input/output (I/O) interface, a processor and a memory, the memory having program instructions stored therein; The processor is configured to execute the program instructions stored in the memory to perform the method of any one of claims 1 to 6.
  14. 14. A computer readable storage medium comprising instructions which, when run on a computer device, cause the computer device to perform the method of any of claims 1 to 6.
  15. 15. A computer program product comprising instructions which, when run on a computer device, cause the computer device to perform the method of any of claims 1 to 6.

Description

Data processing method and related device Technical Field The embodiment of the application relates to the technical field of computers, in particular to a data processing method and a related device. Background Shared memory is an efficient way of inter-process communication. During the running process of the service program, the data is written into the shared memory, and the data can be stored in a lasting manner through the shared memory instead of being written into a local disk or a remote database. Therefore, when the service program is abnormal, the abnormal service program can be pulled up faster, corresponding service capability is provided for one or more application programs, the use experience of the object is not affected, and the loss of unsynchronized data can be reduced when the service program is abnormal. In the related scheme, when the service program is started for the first time, the memory space needs to be applied to the system, and grid cutting is performed on the applied memory space. Thus, when the service program is started later, whether the shared memory associated with the service program exists or not is checked, data is written into the shared memory when the shared memory exists, and meanwhile, whether the grid size of the shared memory is consistent with the service data size of the service program or not is checked. However, since the shared memory is generally a memory with a fixed size, only a shared memory with a fixed length can be applied, and the applied shared memory cannot be dynamically increased. In addition, the service data nodes are fixed in size, so that the service data is difficult to adjust during on-line operation, a certain space can be reserved in the shared memory, and the reserved space is used for caching the service data. However, the scheme of storing service data by reserving a space of a shared memory is required to strictly keep the data format of the service data before being adjusted unchanged, and new service data can only be added after the service data which is not adjusted, so that the problem that the service program dynamically adds the service data in daily operation to cause mismatching of the data format can occur, and the use experience of a target object is affected. Disclosure of Invention The embodiment of the application provides a data processing method and a related device, which can dynamically adjust the data format of service data in a shared memory, thereby solving the problem that the data format is not matched due to the dynamic addition of the service data and improving the use experience of a target object. In a first aspect, an embodiment of the present application provides a method for processing data. The data processing method comprises the steps of obtaining a first data description file, wherein the first data description file is used for describing a data format of first service data, the first service data is the service data obtained when a loading service program is in a first state, determining data difference information between the first data description file and a second data description file when the version of the first data description file is higher than that of the second data description file, the second data description file is used for describing a data format of second service data stored in a shared memory, the second service data is the service data obtained before the loading service program is in the first state, the first service data is different from the second service data, and adjusting the data format of the second service data based on the data difference information and the data format of the first service data. In a second aspect, an embodiment of the present application provides a data processing apparatus. The data processing apparatus includes an acquisition unit and a processing unit. The system comprises an acquisition unit, a service loading unit and a service loading unit, wherein the acquisition unit is used for acquiring a first data description file, the first data description file is used for describing a data format of first service data, and the first service data is the service data obtained when the service loading program is in a first state. The system comprises a first data description file, a processing unit and a processing unit, wherein the first data description file is used for describing the data format of second service data stored in a shared memory, the second service data is the service data obtained before a loading service program is in a first state, the first service data is different from the second service data, and the processing unit is used for adjusting the data format of the second service data based on the data difference information and the data format of the first service data. In some optional embodiments, the obtaining unit is configured to obtain the file configuration information of the second data description file when the second data de