Search

CN-121984884-A - Method and system for processing edge calculation dynamic load prediction data of Internet of things

CN121984884ACN 121984884 ACN121984884 ACN 121984884ACN-121984884-A

Abstract

The invention discloses a method and a system for processing dynamic load prediction data of edge calculation of the Internet of things, and belongs to the technical field of edge calculation. According to the method, edge node load prediction is completed through collecting edge node operation, network link quality and task characteristic parameters of the Internet of things, a time sequence prediction model is adopted, relevant parameters are integrated to construct a task dynamic decision state space, calculation task route distribution is completed based on the state space, a heterogeneous resource scheduling multi-objective optimization model is built, a scheduling scheme is obtained through solving, original data of the Internet of things is subjected to mixed compression according to the scheme, and a pre-fetching rule is built by combining compressed data access statistics and time-space correlation attributes to complete data pre-fetching and cache management. The method can realize dynamic adaptation and reasonable resource utilization of the computing task, reduce the occupation of data transmission bandwidth, improve the cache hit rate of the edge node and the running stability of the system, and adapt to the real-time processing requirement of the edge computing scene of the Internet of things.

Inventors

  • GUO HAO
  • SUN RUJIA

Assignees

  • 四川华鲲振宇智能科技有限责任公司

Dates

Publication Date
20260505
Application Date
20260407

Claims (10)

  1. 1. The method for processing the edge calculation dynamic load prediction data of the Internet of things is characterized by comprising the following steps of: S1, acquiring edge node operation parameters, network link quality parameters and task feature parameters of the Internet of things, adopting a long-and-short-term memory network model to conduct time sequence prediction on edge node loads to obtain load prediction results, integrating the edge node operation parameters, the network link quality parameters, the task feature parameters of the Internet of things and the load prediction results, and constructing a state space of task dynamic decision; S2, inputting a state space into a depth Q network algorithm, completing calculation task route distribution, establishing a multi-objective optimization model of heterogeneous calculation resource scheduling, and solving the multi-objective optimization model by adopting a non-dominant ordering genetic algorithm with elite strategy to obtain a heterogeneous calculation resource scheduling scheme; S3, performing wavelet decomposition processing on original data of the Internet of things according to a heterogeneous computing resource scheduling scheme, performing self-adaptive threshold quantization processing on the data after the wavelet decomposition processing, and performing dynamic Huffman coding processing on the data after the self-adaptive threshold quantization processing to obtain mixed compressed data; S4, carrying out access frequency statistics on the mixed compressed data to obtain access frequency statistics, constructing a data prefetching rule by combining the access frequency statistics and the time-space correlation attribute of the mixed compressed data, and completing data prefetching operation and cache copy updating operation according to the data prefetching rule and a heterogeneous computing resource scheduling scheme.
  2. 2. The method according to claim 1, characterized in that step S1 comprises the sub-steps of: s1.1, acquiring edge node operation parameters, network link quality parameters and task feature parameters of the Internet of things, wherein the edge node operation parameters comprise edge node central processing unit utilization parameters and edge node graphic processing unit utilization parameters, the network link quality parameters comprise network link service quality parameter matrixes, and the task feature parameters of the Internet of things comprise task feature vectors of the Internet of things; S1.2, adopting a long-short-period memory network model to conduct time sequence prediction on the edge node load, wherein the long-short-period memory network model comprises an input layer, a hidden layer and an output layer, the input layer receives historical load time sequence data of the edge node, the hidden layer comprises a forgetting gate, an input gate and an output gate, the hidden layer conducts feature extraction and time sequence fitting on the input time sequence data, and the output layer outputs a load prediction result of the edge node; S1.3, integrating the operation parameters of the edge nodes, the network link quality parameters, the task characteristic parameters of the Internet of things and the load prediction result, and constructing a state space for task dynamic decision.
  3. 3. The method according to claim 1, characterized in that step S2 comprises the sub-steps of: S2.1, inputting a state space into a depth Q network algorithm, wherein the depth Q network algorithm comprises a state input layer, a convolution layer, a full connection layer and an action output layer, the state input layer receives state space data, the convolution layer and the full connection layer extract characteristics of the state space data, the action output layer outputs route allocation actions of calculation tasks, and allocation of the calculation tasks among local edge nodes, adjacent edge clusters and cloud centers is completed; s2.2, taking the processing time consumption and the running power consumption of the computing task as optimization targets, establishing a multi-target optimization model of heterogeneous computing resource scheduling, and setting constraint conditions of the multi-target optimization model as a load upper limit threshold of the edge node; s2.3, solving a multi-objective optimization model by adopting a non-dominant ordering genetic algorithm with elite strategy to obtain a pareto front, and determining a heterogeneous computing resource scheduling scheme based on the pareto front.
  4. 4. The method according to claim 1, characterized in that step S3 comprises the sub-steps of: s3.1, performing multi-level wavelet decomposition processing on original data of the Internet of things by adopting a biorthogonal wavelet base according to a heterogeneous computing resource scheduling scheme to obtain a low-frequency coefficient after the wavelet decomposition processing and a high-frequency coefficient after the wavelet decomposition processing; S3.2, carrying out quantization processing on the high-frequency coefficient after wavelet decomposition processing by adopting Donoho self-adaptive thresholds, reserving the high-frequency coefficient meeting the threshold requirement, and finishing the self-adaptive threshold quantization processing; S3.3, performing entropy coding on the low-frequency coefficient subjected to the self-adaptive threshold quantization and the high-frequency coefficient subjected to the self-adaptive threshold quantization by adopting dynamic Huffman coding, generating a compressed code stream, and obtaining mixed compressed data.
  5. 5. The method according to claim 1, characterized in that step S4 comprises the sub-steps of: S4.1, counting historical access records of the mixed compressed data to obtain access frequency statistics of the mixed compressed data, wherein the access frequency statistics comprise access frequency mean values and access frequency standard deviations; S4.2, constructing a data prefetching rule of space-time correlation based on the spatial correlation attribute of the mixed compressed data and the time correlation attribute of the mixed compressed data and combining the access frequency statistic; S4.3, according to a data prefetching rule and a heterogeneous computing resource scheduling scheme, the prefetching operation of target data is completed, and the updating operation of the cache copy and the eliminating operation of the cache copy are completed based on the access frequency statistics.
  6. 6. The method according to claim 1, wherein in step S1, secondary exponential smoothing processing is performed on the edge node operation parameters, first, exponential smoothing calculation is performed on the time sequence of the edge node operation parameters to obtain a primary smoothed value, then, secondary exponential smoothing calculation is performed on the primary smoothed value to obtain a secondary smoothed value, the time sequence of the edge node operation parameters is corrected based on the secondary smoothed value, normalization processing is performed on the corrected time sequence of the edge node operation parameters, and load time sequence prediction is performed on the normalized time sequence of the edge node by using a long-short-term memory network model to obtain a load prediction result.
  7. 7. The method of claim 1, wherein in step S2, the heterogeneous computing resources include a central processor, a graphics processor, and a field programmable gate array, the optimization objective of the multi-objective optimization model includes a total processing time of the computing task and a total running power consumption of the computing task, the total processing time being a weighted sum of the central processor processing time, the graphics processor processing time, and the field programmable gate array processing time, the total running power consumption being a cumulative sum of the central processor running power consumption, the graphics processor running power consumption, and the field programmable gate array running power consumption, the constraint condition of the multi-objective optimization model being an upper load threshold of the edge node, the real-time load value of the edge node not exceeding the upper load threshold.
  8. 8. The method of claim 1, wherein in step S3, when wavelet decomposition is performed on original data of the internet of things, symmetric boundary extension processing is performed on the original data of the internet of things, a boundary sequence of the original data of the internet of things is supplemented, multi-level wavelet decomposition processing is performed on the original data of the internet of things after boundary extension by adopting a biorthogonal wavelet basis, low-frequency coefficients and multiple groups of high-frequency coefficients of a corresponding level are obtained, block processing is performed on the high-frequency coefficients obtained by the decomposition, quantization processing is performed on each group of high-frequency coefficients after the block processing by adopting Donoho self-adaptive threshold values, and high-frequency coefficients which do not meet the threshold value requirements are filtered.
  9. 9. The method of claim 1, wherein in step S4, when a space-time associated data prefetching rule is constructed, spatial association degree calculation is performed on the mixed compressed data, spatial position association weights among different mixed compressed data are calculated, then time association degree calculation is performed on the mixed compressed data, time access sequence association weights among different mixed compressed data are calculated, a space position association weight, a time access sequence association weight and access frequency statistic are combined, a space-time associated data prefetching rule is constructed, prefetched target data is determined based on the data prefetching rule, the target data is cached in a storage unit corresponding to an edge node in advance, and updating operation of a cached copy and elimination operation of the cached copy are completed based on the access frequency statistic.
  10. 10. An internet of things edge computing dynamic load prediction data processing system for executing the method of any of claims 1-9, comprising a load prediction and state space construction module, a task routing and resource scheduling module, a data hybrid compression module, and a distributed cache management module; The system comprises a load prediction and state space construction module, a task routing and resource scheduling module, a data hybrid compression module, a distributed cache management module and a cache copy update module, wherein the load prediction and state space construction module is used for completing edge node operation parameter acquisition, network link quality parameter acquisition, task characteristic parameter acquisition of the Internet of things, edge node load time sequence prediction and state space construction of task dynamic decision, the task routing and resource scheduling module is used for completing calculation task routing distribution, multi-objective optimization model construction of heterogeneous calculation resource scheduling and multi-objective optimization model solution, the data hybrid compression module is used for completing boundary extension processing, wavelet decomposition processing, self-adaptive threshold quantization processing and dynamic Huffman coding processing of original data of the Internet of things, and the distributed cache management module is used for completing access frequency statistics of hybrid compressed data, time-space related data prefetching rule construction, data prefetching operation and cache copy update operation.

Description

Method and system for processing edge calculation dynamic load prediction data of Internet of things Technical Field The invention relates to the technical field of edge calculation, in particular to a method and a system for processing dynamic load prediction data of edge calculation of the Internet of things. Background Along with the continuous deepening of the scale landing and industrial application of the Internet of things technology, the deployment scale of the terminal nodes of the Internet of things is continuously enlarged, and massive real-time data generated in various scenes are provided with higher requirements on the real-time performance of data processing, the high efficiency of transmission and the rationality of computing resource scheduling. The edge computing is used as a distributed computing architecture deployed near a data generation end at the network edge side, can effectively make up a transmission delay short board of the traditional centralized cloud computing architecture in the scene of the Internet of things, becomes a core technical direction in the data processing field of the Internet of things, and realizes wide application and landing in a plurality of scenes such as the Internet of things, intelligent equipment collaboration, city perception networks and the like. At present, the technical development and industry practice in the field of edge computing continue to advance, around the core technical links of edge computing task scheduling, heterogeneous computing resource management, data compression transmission of the internet of things, distributed cache optimization and the like, various mature technology implementation paths and engineering application schemes, time sequence prediction algorithms, reinforcement learning decision algorithms, wavelet transformation signal processing technologies, intelligent cache management technologies and other related technologies have been formed, deep fusion has also been gradually realized with an edge computing architecture, abundant technical support is provided for edge side data processing in the scene of the internet of things, and meanwhile, the optimization requirements for operation stability, resource utilization efficiency and data processing instantaneity of an edge computing system in the industry are also continuously improved, and continuous iteration and innovation development of the related technologies are promoted. In the technical practice of edge computing data processing of the internet of things, the technical limitation of multiple dimensions still exists, the prior art scheme is difficult to conduct time sequence prejudgement on load change of edge nodes, the operation state of the edge nodes, the quality of network links, task attributes of the internet of things and other multidimensional information cannot be effectively integrated, complete state basis adapting to dynamic decision requirements is built, and the task allocation scheme is difficult to adapt to dynamic change characteristics of an edge computing environment. Meanwhile, in the existing scheme, in the process of computing task route allocation, dynamic adaptation of tasks and computing nodes is difficult to achieve, balanced optimization cannot be achieved among a plurality of associated optimization targets in the scheduling process of heterogeneous computing resources, a resource scheduling scheme adapting to task requirements and node states is difficult to form, and utilization efficiency of the computing resources is easy to be insufficient. In addition, in the existing data processing scheme of the Internet of things, the processing efficiency and the data fidelity are difficult to consider in the data compression processing process, the effective reduction of the data redundancy cannot be realized through a progressive processing flow, the network bandwidth occupation in the data transmission process is increased, the existing distributed cache management scheme also is difficult to combine the access rule of the data and the internal association attribute to construct the adaptive prefetching rule, the dynamic update and the reasonable elimination of the cache copy cannot be realized, the end-to-end response time delay of the data access is difficult to effectively reduce, and the operation requirement of the edge computing scene of the Internet of things cannot be comprehensively adapted. Disclosure of Invention The invention aims to overcome the defects of the prior art and provides a method and a system for processing edge computing dynamic load prediction data of the Internet of things. The aim of the invention is realized by the following technical scheme: the invention provides a method for processing edge calculation dynamic load prediction data of the Internet of things, which comprises the following steps: S1, acquiring edge node operation parameters, network link quality parameters and task feature parameters of the I