Search

CN-121985047-A - Data lightweight transmission method for Internet of things equipment based on edge calculation

CN121985047ACN 121985047 ACN121985047 ACN 121985047ACN-121985047-A

Abstract

The invention discloses an internet of things equipment data lightweight transmission method based on edge calculation, and particularly relates to the technical field of data transmission, comprising the following steps of firstly, establishing a dynamic sliding window at an edge node, filtering a data source by calculating deviation between a new data point and a historical trend and combining a self-adaptive threshold value dynamically adjusted based on historical fluctuation and network conditions, and introducing an optimization mechanism correction weight coefficient based on report missing rate and false report rate feedback; step three, carrying out value evaluation on the data packet from service priority, information entropy and timeliness dimension, and effectively eliminating time and space dimension data redundancy through a three-level linkage mechanism, realizing network resource refined distribution and greatly reducing uplink data quantity and bandwidth occupation.

Inventors

  • WANG AORAN

Assignees

  • 合肥新理科技有限公司

Dates

Publication Date
20260505
Application Date
20260407

Claims (9)

  1. 1. The method for lightweight transmission of the data of the Internet of things equipment based on edge calculation is characterized by comprising the following steps: Firstly, establishing a dynamic sliding window for Internet of things equipment on an edge computing node, primarily filtering a data source by calculating the deviation between a new data point and a historical trend and combining an adaptive threshold dynamically adjusted based on historical fluctuation and network conditions, and only marking the data point which exceeds the adaptive threshold or has a numerical mutation with the historical trend as a key data point and storing the key data point into a queue to be uploaded; Maintaining a device relation map by the edge node, identifying a spatially related sensor group, performing spatial redundancy analysis when a plurality of sensors in the group generate key data points in similar time windows, and performing fusion processing on data meeting spatial redundancy conditions to generate representative point data or statistical feature data to replace the original plurality of key data points; And thirdly, the edge node evaluates the value of the data packet to be uploaded, calculates the total value score from three dimensions of service priority, information entropy and timeliness, divides the total value score into different value grades according to the total value score, and uploads the data packet with different grades by adopting a differentiated compression algorithm and a differentiated transmission protocol.
  2. 2. The method for lightweight transmission of data of internet of things equipment based on edge calculation according to claim 1, wherein in the first step: The dynamic sliding window comprises a main sampling window and an auxiliary verification window, the main sampling window is used for trend analysis, the size of the main sampling window is dynamically adjusted according to the data stability, and the size of the auxiliary verification window is fixed for mutation detection; the self-adaptive threshold is calculated dynamically according to the fluctuation amplitude of the historical data and the current network congestion condition, wherein the historical fluctuation weight and the network congestion weight coefficient are periodically corrected through an optimization mechanism based on feedback of the false alarm rate and the false alarm rate.
  3. 3. The method for lightweight transmission of data of internet of things equipment based on edge calculation according to claim 2, wherein the optimization mechanism based on the feedback of the missing report rate and the false report rate comprises: periodically acquiring the missing report times and the false report times of the previous period from the cloud by the edge node, and calculating the missing report rate and the false report rate; setting a target missing report rate and a target false report rate, and calculating the adjustment quantity of the weight coefficient by adopting a proportional control law; and updating the weight coefficient according to the adjustment quantity, and limiting the updated coefficient within a preset range for calculating the self-adaptive threshold value of the next period.
  4. 4. The method for lightweight transmission of data of internet of things equipment based on edge calculation according to claim 1, wherein the method for automatically constructing the equipment relationship map in the second step comprises the following steps: In the initial stage of system deployment, an edge node collects original data and reports the data to a cloud end, the cloud end calculates the data sequence correlation of any two sensors by using a Pearson correlation coefficient, and if the absolute value of the correlation coefficient is larger than a preset threshold, the two sensors are judged to be highly correlated and are classified into the same correlation group; the atlas supports periodic dynamic updates or manual corrections by an administrator.
  5. 5. The method for lightweight transmission of data of internet of things equipment based on edge computing according to claim 4, wherein the spatial redundancy analysis comprises: When a plurality of sensors in the group generate key data points in a similar time window, calculating the difference value between the readings of each sensor, and judging that space redundancy exists if the maximum difference value is smaller than a preset space redundancy threshold value; the fusion processing comprises representative point uploading or statistical feature uploading, wherein the representative point uploading selects sensor data with highest signal-to-noise ratio or most sufficient battery power in the group, and the statistical feature uploading calculates and uploads an average value, a median, a maximum value or a minimum value of the data in the group; and after the fusion processing is completed, generating a data list to be uploaded for recording the corresponding information of the fusion data and the original sensor.
  6. 6. The method for lightweight transmission of data of internet of things equipment based on edge calculation according to claim 1, wherein in the third step: the value evaluation total score is calculated through a multi-factor weighted summation model, and normalization processing is carried out on each dimension: Mapping the service priority into a numerical value, directly using the normalized value by the information entropy, and calculating the freshness according to the time difference between the data generation time and the current time and the maximum validity period of the data; And multiplying the corresponding weight coefficients and summing to obtain total score, and dividing the high, medium and low value grades according to the total score.
  7. 7. The method for lightweight transmission of data of internet of things equipment based on edge calculation according to claim 6, wherein the weight coefficient is dynamically optimized by a weight adjustment mechanism based on historical transmission performance feedback, comprising: The edge node periodically acquires the actual utilization rate of each level of data of the previous period from the cloud, and calculates the deviation from the target utilization rate; And calculating the adjustment quantity of each weight coefficient by adopting a proportional control law, updating the weight coefficient, and then carrying out normalization processing and amplitude limiting for value evaluation of the next period.
  8. 8. The method for lightweight transmission of data of internet of things equipment based on edge computing according to claim 6, wherein the differentiated compression algorithm and transmission protocol comprises: For high-value data, adopting a lossless compression algorithm and transmitting the high-value data in real time through a reliable TCP connection; for medium value data, adopting a lossy compression algorithm and carrying out batch transmission through a UDP protocol; and for low-value data, after the edge nodes perform aggregation compression, selecting a network idle period to perform batch transmission through a non-real-time channel.
  9. 9. The method for lightweight transmission of data of an internet of things device based on edge computing according to claim 5, further comprising an edge node state holding mechanism: The queue to be uploaded and the data list are subjected to persistence through nonvolatile storage, and when the edge node is restarted, a persistent data recovery breakpoint is loaded; meanwhile, the queue to be uploaded adopts a ring buffer area design, when the buffer area is full, an alarm is triggered, and low-value data are discarded preferentially so as to ensure the stable operation of the system.

Description

Data lightweight transmission method for Internet of things equipment based on edge calculation Technical Field The invention relates to the technical field of data transmission, in particular to an internet of things equipment data lightweight transmission method based on edge calculation. Background With the rapid development and wide application of the internet of things technology, hundreds of millions of sensors and devices continuously generate mass data, and a serious challenge is provided for network transmission and cloud processing capacity. In the traditional 'end-cloud' direct connection transmission architecture, all original data are directly uploaded to a cloud, so that a large amount of network bandwidth is occupied, transmission delay and network congestion are caused, and heavy storage and calculation burden is brought to a cloud data center. To alleviate this problem, edge computing technologies have been developed to implement lightweight transmission of data by deploying computing nodes on the edge side of the network near the data source, and preprocessing the data. However, the existing edge computing light-weight method is generally simpler, for example, a fixed period aggregation report or a static threshold trigger report is adopted, and time correlation, space correlation and value density difference of the data are not fully considered, so that the light-weight effect is limited, or the precision loss is large during data restoration. In the data acquisition layer, the prior art mostly adopts fixed frequency sampling or simple threshold filtering, and the time correlation of the data is difficult to effectively identify. For example, a fixed period report can reduce the data volume, but cannot capture a transient abnormal peak value occurring in the period, so that a key event is lost, while a static threshold trigger report can generate a large amount of data which is normal but has abnormal trend when facing a slow drift fault, so that erroneous judgment or missed judgment is caused. In addition, in the multi-sensor deployment scenario, most of the existing schemes independently process each sensor data, and spatial redundancy of the multi-sensor data in the same area is not fully considered, for example, readings of a plurality of temperature sensors in the same warehouse are highly consistent but all uploaded, so that serious spatial dimension data redundancy is caused. Meanwhile, in the data transmission layer, the prior art generally adopts the same transmission strategy for all data, and the differentiation processing is not carried out according to the service value, the information content and the timeliness of the data, so that when a network is congested, high-value key data can be lost or delayed due to bandwidth occupation of low-value heartbeat logs and other data. Aiming at the problems in the prior art, a method for carrying out intelligent light weight from data acquisition, data processing to data transmission is needed to accurately identify and retain key information, eliminate data redundancy of time dimension and space dimension and realize fine distribution of network resources according to data value, so that network bandwidth occupation and cloud storage pressure are reduced to the greatest extent on the premise of guaranteeing data integrity and business requirements. Disclosure of Invention In order to overcome the defects in the prior art, the embodiment of the invention provides an internet of things equipment data lightweight transmission method based on edge calculation. In order to achieve the above purpose, the present invention provides the following technical solutions: The lightweight transmission method for the data of the Internet of things equipment based on edge calculation comprises the following steps: Firstly, establishing a dynamic sliding window for Internet of things equipment on an edge computing node, primarily filtering a data source by calculating the deviation between a new data point and a historical trend and combining a self-adaptive threshold dynamically adjusted based on historical fluctuation and network conditions, and only marking the data point with meaningful change or mutation as a key data point and storing the key data point into a queue to be uploaded; Secondly, maintaining a device relation map by the edge node, identifying a spatially related sensor group, performing spatial redundancy analysis when a plurality of sensors in the group generate key data points in similar time windows, and performing fusion processing on data meeting spatial redundancy conditions to generate representative point data or statistical feature data to replace the original plurality of key data points; And thirdly, the edge node evaluates the value of the data packet to be uploaded, calculates the total value score from three dimensions of service priority, information entropy and timeliness, divides the total value score into different