Search

CN-121996368-A - Data processing method and system

CN121996368ACN 121996368 ACN121996368 ACN 121996368ACN-121996368-A

Abstract

The embodiment of the specification provides a data processing method and a data processing system, wherein the data processing method comprises the steps of determining a detection task of a target application and obtaining data to be detected corresponding to the detection task. The data identification and aggregation fields are configured for the data to be detected, a target static data table is constructed based on the data to be detected, the data identification and the aggregation fields, static data corresponding to each application attribute in the target static data table has a fixed storage position, data ordering is not needed to be carried out again when data aggregation is carried out subsequently, the data processing overhead is reduced, and the data processing efficiency is improved. And constructing a data aggregation tree based on the detection items in the target static data table and the attribute data of the detection items, and executing detection tasks based on the data aggregation tree. And the data aggregation tree is used for realizing the aggregation of the data in the target static data table, so that the flexibility of the data aggregation is improved, and the hierarchical data aggregation is realized. And guiding the execution of the subsequent detection task, and optimizing the execution process of the detection task.

Inventors

  • GONG YUAN
  • LIU KAI

Assignees

  • 阿里云计算有限公司

Dates

Publication Date
20260508
Application Date
20241108

Claims (13)

  1. 1. A data processing method, comprising: Determining a detection task of a target application, and acquiring data to be detected corresponding to the detection task; Configuring a data identifier and an aggregation field for the data to be detected, and constructing a target static data table based on the data to be detected, the data identifier and the aggregation field; and constructing a data aggregation tree based on the detection items in the target static data table and the attribute data of the detection items, and executing the detection task based on the data aggregation tree.
  2. 2. The data processing method according to claim 1, wherein the determining the detection task of the target application includes: Determining a task queue corresponding to the target application; And reading the detection task of the target application in the task queue.
  3. 3. The method of claim 1, further comprising, after the obtaining the data to be detected corresponding to the detection task: determining test attribute data associated with the data to be detected through a data callback; And generating detection guidance information of the data aggregation tree based on the test attribute data.
  4. 4. The data processing method according to claim 1, wherein the configuring the data identifier and the aggregation field for the data to be detected, and constructing the target static data table based on the data to be detected, the data identifier, and the aggregation field, includes: configuring the data identifier for the data to be detected, and constructing an initial static data table based on the data to be detected and the data identifier; and configuring an aggregation field for the data records in the initial static data table to obtain a target static data table.
  5. 5. The data processing method according to claim 1, wherein configuring an aggregation field for the data record in the initial static data table to obtain a target static data table includes: Determining an aggregation mode of the data records in the initial static data table; and configuring the aggregation field for the data record according to the aggregation mode to obtain the target static data table.
  6. 6. The data processing method according to claim 1, the constructing a data aggregation tree based on the detection items in the target static data table and the attribute data of the detection items, comprising: Determining a detection item in the target static data table and aggregation field data contained in the detection item; And determining attribute data of the detection item according to the aggregation field data, and constructing a data aggregation tree based on the detection item and the attribute data.
  7. 7. The data processing method according to claim 1, the determining attribute data of the detection item from the aggregation field data, comprising: determining an aggregation mode level corresponding to the aggregation field data; And determining a node structure level according to the aggregation mode level, and taking the node structure level as the attribute data of the detection item.
  8. 8. The data processing method of claim 7, the constructing a data aggregation tree based on the detection item and the attribute data, comprising: Extracting detection data from the detection items, and determining tree nodes corresponding to the detection data; the data aggregation tree is constructed based on the tree nodes and the node structure hierarchy.
  9. 9. The data processing method of claim 1, the performing the detection task based on the data aggregation tree, comprising: determining a target tree node contained in the data aggregation tree; and reading node data of the target data node, and executing the detection task based on the node data.
  10. 10. A data processing system comprises a server and a client; the client is used for generating an application detection request based on a target application and sending the application detection request to the server; The method comprises the steps of determining a detection task of a target application, acquiring data to be detected corresponding to the detection task, configuring a data identification and an aggregation field for the data to be detected, constructing a target static data table based on the data to be detected, the data identification and the aggregation field, constructing a data aggregation tree based on detection items in the target static data table and attribute data of the detection items, executing the detection task based on the data aggregation tree, and sending a task execution result to the client.
  11. 11. A computing device, comprising: A memory and a processor; the memory is adapted to store a computer program or instructions, the processor being adapted to execute the computer program or instructions, which when executed by the processor, perform the steps of the method according to any one of claims 1-9.
  12. 12. A computer readable storage medium storing a computer program or instructions which, when executed by a processor, performs the steps of the method of any one of claims 1-9.
  13. 13. A computer program product comprising a computer program or instructions which, when executed by a processor, implement the steps of the method of any of claims 1 to 9.

Description

Data processing method and system Technical Field Embodiments of the present disclosure relate to the field of computer technologies, and in particular, to a data processing method and system. Background Monitoring and performance tracking are important during the process of software development or application running. The observable indicators generated during the running of software development or applications are typically quantitative data, represented in numerical form, that can provide critical information about the performance, health, and behavior of the system. After the information is aggregated, the method can be used for detecting application programs and monitoring performance of functional modules in the software development process. In the prior art, openTelemetry (OTel) SKD index aggregation logic with an open source is generally adopted to conduct data aggregation and performance monitoring in the running process of software development and application. However, the core data structure inside this data aggregation method is a hash table keyed by a set of attributes. When the same hash is generated for the same attribute set for the key value, the attribute set is inevitably ordered once according to the keys, so that the cost of data processing is increased, and the data processing efficiency is reduced. In addition, according to the index aggregation method, different indexes are in an isolation state, so that subsequent index aggregation is not facilitated. Therefore, a more efficient data processing method is needed to solve the above-mentioned problems. Disclosure of Invention In view of this, the present embodiments provide a data processing method. One or more embodiments of the present specification are also directed to a data processing system, a data processing apparatus, a computing device, a computer readable storage medium, and a computer program product that address the deficiencies of the prior art. According to a first aspect of embodiments of the present specification, there is provided a data processing method, including: Determining a detection task of a target application, and acquiring data to be detected corresponding to the detection task; Configuring a data identifier and an aggregation field for the data to be detected, and constructing a target static data table based on the data to be detected, the data identifier and the aggregation field; and constructing a data aggregation tree based on the detection items in the target static data table and the attribute data of the detection items, and executing the detection task based on the data aggregation tree. According to a second aspect of embodiments of the present specification, there is provided a data processing system, comprising a server and a client; the client is used for generating an application detection request based on a target application and sending the application detection request to the server; The method comprises the steps of determining a detection task of a target application, acquiring data to be detected corresponding to the detection task, configuring a data identification and an aggregation field for the data to be detected, constructing a target static data table based on the data to be detected, the data identification and the aggregation field, constructing a data aggregation tree based on detection items in the target static data table and attribute data of the detection items, executing the detection task based on the data aggregation tree, and sending a task execution result to the client. According to a third aspect of embodiments of the present specification, there is provided a data processing apparatus comprising: The determining module is configured to determine a detection task of the target application and acquire data to be detected corresponding to the detection task; The configuration module is configured to configure a data identifier and an aggregation field for the data to be detected, and construct a target static data table based on the data to be detected, the data identifier and the aggregation field; And the execution module is configured to construct a data aggregation tree based on the detection items in the target static data table and the attribute data of the detection items, and execute the detection task based on the data aggregation tree. According to a fourth aspect of embodiments of the present specification, there is provided a computing device comprising: A memory and a processor; the memory is configured to store computer executable instructions that, when executed by the processor, perform the steps of the data processing method described above. According to a fifth aspect of embodiments of the present specification, there is provided a computer readable storage medium storing computer executable instructions which, when executed by a processor, implement the steps of the data processing method described above. According to a sixth aspect of embodiments of