Search

CN-114281648-B - Data acquisition method and device, electronic equipment and storage medium

CN114281648BCN 114281648 BCN114281648 BCN 114281648BCN-114281648-B

Abstract

The invention provides a data acquisition method, a device, electronic equipment and a storage medium, wherein the method is applied to mobile terminal equipment and comprises the steps of obtaining task information of a target task from cloud equipment, wherein the task information comprises embedded point information of the target task, data screening conditions and a time window, the cloud equipment is used for storing the task information, determining data embedded points of the target task according to the embedded point information, obtaining embedded point data meeting the data screening conditions from the data embedded points and serving as data to be processed, aggregating the data to be processed according to the time window to obtain an aggregation result, and storing the aggregation result as a data file of the target task. Therefore, the data files corresponding to the target tasks can be obtained by aggregating the data according to the task information, and if the data required by the target tasks need to be modified, the task information only needs to be modified at the cloud device, so that the data obtaining mode is flexible and the limitation is small.

Inventors

  • LIU HAIJUN

Assignees

  • 北京奇艺世纪科技有限公司

Dates

Publication Date
20260508
Application Date
20211223

Claims (12)

  1. 1. A data acquisition method, applied to a mobile terminal device, the method comprising: task information of a target task is obtained from cloud equipment, wherein the task information comprises embedded point information, data screening conditions, a time window, grouping data items, aggregation data items, query time and a second aggregation rule of the target task, and the cloud equipment is used for configuring and storing the task information; Determining data embedded points of the target tasks according to the embedded point information, and acquiring embedded point data meeting the data screening conditions from the data embedded points according to distribution indexes as data to be processed, wherein each piece of data to be processed comprises a plurality of data items; according to the time window, aggregating the data to be processed to obtain an aggregation result, including: Grouping and aggregating the data to be processed in each time window based on the grouping data item and the aggregation data item to obtain an aggregation result corresponding to each grouping; the grouping data items are used for grouping the data to be processed, and the aggregation data items are used for aggregating the data to be processed; Based on the second aggregation rule, secondarily aggregating the data to be queried in the query time in the aggregation result to obtain a query result; And storing the aggregation result and the query result as data files of the target task, wherein the data files are effective in the life cycle of the application program.
  2. 2. The method according to claim 1, wherein the target task includes a plurality of the data buried points of the target task are determined according to the buried point information, and buried point data conforming to the data screening condition is acquired from the data buried points according to a distribution index as data to be processed, including: Generating a distribution index of each data embedded point according to the embedded point information of the target task; Monitoring the data embedded points, and determining target tasks corresponding to the monitored embedded point data according to the distribution indexes; Judging whether the monitored buried data accords with the data screening condition of any target task corresponding to the data buried point, and if so, taking the monitored buried data as the data to be processed of any target task.
  3. 3. The method of claim 1, wherein aggregating the data to be processed according to the time window to obtain an aggregate result comprises: taking the acquisition time of the task information as the starting time, and determining the deadlines of a plurality of time windows of the target task; And when the deadline of any time window is reached, aggregating the data to be processed acquired in any time window to obtain an aggregation result.
  4. 4. The method of claim 1, wherein the task information further includes a first aggregation rule, and the aggregating the data to be processed according to the time window to obtain an aggregation result includes: according to the time window, the data to be processed are aggregated according to the first aggregation rule, and an aggregation result is obtained; The first aggregation rule comprises at least one of data number statistics, maximum value calculation and minimum value calculation.
  5. 5. The method according to claim 1, wherein the task information further includes a save period, and the storing the aggregation result as a data file of the target task includes: And after the time difference between the current time and the time of last storing the aggregation result into the data file of the target task exceeds the storage period, storing the non-stored aggregation result as the data file of the target task.
  6. 6. The method according to claim 1, wherein the task information further includes a preset operation, the preset operation is used for triggering storage of the aggregation result, and the storing the aggregation result as the data file of the target task includes: responding to the preset operation, and detecting whether the aggregation result of the target task is updated or not; and if the target task is updated, storing the updated aggregation result as a data file of the target task.
  7. 7. The method of claim 1, wherein after aggregating the data to be processed according to the time window to obtain an aggregate result, the method further comprises: acquiring data to be queried in the query time from the aggregation result; And aggregating the data to be queried according to the second aggregation rule to obtain a query result.
  8. 8. The method of claim 7, wherein each data in the aggregated result has a corresponding key value pair, and the aggregating the data to be queried according to the second aggregation rule to obtain a query result includes: And according to the second aggregation rule, aggregating the key value pairs of the data to be queried to obtain a query result.
  9. 9. The method according to claim 1, wherein grouping and aggregating the data to be processed in each time window based on the grouping data item and the aggregate data item to obtain an aggregate result corresponding to each group includes: Grouping the data to be processed in each time window based on the grouping data items; And aggregating the aggregate data items of the data to be processed in each group to obtain an aggregation result corresponding to the group.
  10. 10. A data acquisition apparatus, for use with a mobile terminal device, the apparatus comprising: the information acquisition module is used for acquiring task information of a target task from cloud equipment, wherein the task information comprises embedded point information, data screening conditions, a time window, grouping data items, aggregation data items, query time and a second aggregation rule of the target task, and the cloud equipment is used for configuring and storing the task information; the data acquisition module is used for determining the data embedded point of the target task according to the embedded point information, and acquiring embedded point data meeting the data screening conditions from the data embedded point according to a distribution index as data to be processed, wherein each data to be processed comprises a plurality of data items; The aggregation module is used for aggregating the data to be processed according to the time windows to obtain an aggregation result, and comprises the steps of grouping and aggregating the data to be processed in each time window based on grouping data items and aggregation data items to obtain an aggregation result corresponding to each grouping, wherein the grouping data items are used for grouping the data to be processed, the aggregation data items are used for aggregating the data to be processed, the second aggregation is carried out on the data to be queried in the query time in the aggregation result based on the second aggregation rule to obtain a query result, and the aggregation result and the query result are stored as data files of the target task, wherein the data files are effective in the life cycle of an application program.
  11. 11. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the data acquisition method according to any one of claims 1 to 9 when the program is executed.
  12. 12. A computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the steps of the data acquisition method according to any one of claims 1 to 9.

Description

Data acquisition method and device, electronic equipment and storage medium Technical Field The present invention relates to the field of internet technologies, and in particular, to a data acquisition method, apparatus, electronic device, and storage medium. Background Under the scene of collecting the user behavior data, the mobile terminal has the advantages of natural user behavior data, and can directly interact with the user to acquire the behavior data of the user. At present, the user behavior data is usually obtained in a hard-coded form at the mobile terminal, that is, the information of the user behavior data to be obtained is directly embedded into the source code of a program or other executable objects, and the user behavior data is obtained in the execution process of the program or the code. However, since the hard-coded data can be modified only by editing the source code and recompiling the executable file, the current acquisition mode of the user behavior data is inflexible, needs to be developed according to specific requirements, and then is iteratively updated along with release of the version, so that the data is difficult to effectively use, has very large limitation and has very poor flexibility. Disclosure of Invention In order to solve the technical problems, the invention discloses a data acquisition method, a data acquisition device, electronic equipment and a storage medium. The invention provides a data acquisition method which is applied to mobile terminal equipment, and comprises the following steps: Task information of a target task is obtained from cloud equipment, wherein the task information comprises embedded point information, data screening conditions and a time window of the target task, and the cloud equipment is used for storing the task information; Determining a data embedded point of the target task according to the embedded point information, and acquiring embedded point data meeting the data screening conditions from the data embedded point as data to be processed; and aggregating the data to be processed according to the time window to obtain an aggregation result, and storing the aggregation result as a data file of the target task. Optionally, the target task includes a plurality of buried points, and determining a data buried point of the target task according to the buried point information, and acquiring buried point data meeting the data screening condition from the data buried point, as data to be processed, including: Generating a distribution index of each data embedded point according to the embedded point information of the target task, wherein the distribution index is used for indicating the corresponding relation between each data embedded point and at least one target task; Monitoring the data embedded points, and determining target tasks corresponding to the monitored embedded point data according to the distribution indexes; Judging whether the monitored buried data accords with the data screening condition of any target task corresponding to the data buried point, and if so, taking the monitored buried data as the data to be processed of any target task. Optionally, the aggregating the data to be processed according to the time window to obtain an aggregation result includes: taking the acquisition time of the task information as the starting time, and determining the deadlines of a plurality of time windows of the target task; And when the deadline of any time window is reached, aggregating the data to be processed acquired in any time window to obtain an aggregation result. Optionally, the task information further includes a first aggregation rule, and the aggregating the data to be processed according to the time window to obtain an aggregation result includes: according to the time window, the data to be processed are aggregated according to the first aggregation rule, and an aggregation result is obtained; The first aggregation rule comprises at least one of data number statistics, maximum value calculation and minimum value calculation. Optionally, the task information further includes a save period, and the storing the aggregation result as a data file of the target task includes: And after the time difference between the current time and the time of last storing the aggregation result into the data file of the target task exceeds the storage period, storing the non-stored aggregation result as the data file of the target task. Optionally, the task information further includes a preset operation, where the preset operation is used to trigger storage of the aggregation result, and the storing the aggregation result as the data file of the target task includes: responding to the preset operation, and detecting whether the aggregation result of the target task is updated or not; and if the target task is updated, storing the updated aggregation result as a data file of the target task. Optionally, the task information further includes a